Self-Organizing Map Neural Network Semantic Text Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for semantic text processing, particularly in search engines, are limited in depth of analysis when dealing with longer and more complex word sequences due to reliance on statistical indexes and thesaurus files, which hinder the effective use of neural networks for understanding text meaning and predicting relevant keywords.
Innovation Solution
A method involving a self-organizing map neural network to create a pattern dictionary that maps keywords to semantic patterns, allowing for the translation of keyword sequences into pattern sequences, which are then used to train a second neural network for enhanced semantic classification and prediction tasks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If statistical indexes and thesaurus files are used for semantic text processing, then existing methods can process text, but the depth of semantic analysis is limited when dealing with longer and more complex word sequences
Solution Approach 1:
The patent replaces traditional statistical indexes and thesaurus files with a neural network-based system. The neural network learns semantic relationships from training data and can process complex word sequences without relying on pre-defined statistical structures, thereby increasing the depth of semantic analysis while managing system complexity through automated learning rather than manual configuration
Solution Approach 2:
The system transforms text processing from using fixed statistical parameters (frequency counts, co-occurrence statistics) to using dynamic neural network representations. The neural network adjusts its internal representations based on the input text, allowing it to capture semantic meanings that go beyond static statistical patterns, thus improving analysis depth for complex sequences
2Reliability
If neural networks are used for pattern recognition in complex data, then recognition capability improves, but training requires massive amounts of training data and complex configuration
Solution Approach 1:
The patent performs preliminary action by pre-training a neural network on a large corpus of training text data before applying it to specific tasks. This pre-training phase captures general semantic relationships and patterns, so that when the network is later applied to new, smaller datasets for specific applications, it requires less additional training data and computational resources, thereby reducing overall training complexity while maintaining high recognition capability
Solution Approach 2:
The neural network is designed to perform multiple functions: it can classify text, extract entities, understand semantic relationships, and generate completions, all using the same underlying model trained on general text data. This multi-functionality reduces the need for separate specialized models for each task, thereby reducing overall training complexity while improving reliability across different applications
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The invention relates to a computer-implemented method of training a neural network, comprising: training a first neural network (4) of a self organizing map type with a first set (2) of first text documents (3) each containing one or more keywords (7) in a semantic context to map each document (3) to a point (Xi/Yj) in the self organizing map (5) by semantic clustering; determining, for each keyword (7) occurring in the first set (2), all points (Xi/Yj) in the self organizing map (5) to which first documents (3) containing said keyword (7) are mapped, as a pattern (6) and storing said pattern (6) for said keyword (7) in a pattern dictionary (9); forming at least one sequence (11) of keywords (7) from a second set (12) of second text documents (13) each containing one or more keywords (7) in a semantic context; translating said at least one sequence (11) of keywords (7) into at least one sequence (14) of patterns (6) by using said pattern dictionary (9); and training a second neural network (15) with said at least one sequence (14) of patterns (6). The invention further relates to computer-readable media and classification, prediction and translation machines based on neural networks.