Language semantic analysis method and system based on improved PINN network
Through the improved PINN network and physical information feature vectors, combined with optimization algorithms to adjust parameters, the problem of insufficient accuracy in traditional neural network semantic analysis when processing complex texts is solved, achieving higher semantic analysis accuracy and model adaptability.
Patent Information
- Application Number
- CN202510132290.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-06
- Publication Date
- 2025-06-10
AI Technical Summary
When traditional neural network semantic analysis processes texts with stream of consciousness sentences, abnormal grammatical structures and dialects, the accuracy needs to be further improved.
Using an improved PINN network, we will build a neural network structure including input layer, hidden layer and output layer, and introduce physical information feature vectors and improved loss functions, and adjust parameters in combination with optimization algorithms to better learn the feature laws of text.
It improves the accuracy of semantic analysis and the adaptability of the model, can better handle complex and obscure text semantics, and enhances the interpretability and credibility of the analysis results.
Smart Images
Figure CN120124634A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of semantic analysis, and in particular to a language semantic analysis method and system based on an improved PINN network. Background Art
[0002] As an iconic figure in Australian literature, Helen Garner's works are known for their profound themes, delicate emotions and unique narrative style. The language is full of regional cultural characteristics. It not only integrates Australian dialect vocabulary, but also uses a large number of metaphors and symbols to make the text semantically rich and obscure. This unique language style provides rich materials for literary research, but also brings great challenges to semantic analysis. Traditional rule-based semantic analysis methods rely on manually preset complex grammatical and semantic rules, which are relatively limited when dealing with stream-of-consciousness sentences, unconventional grammatical structures and dialect obscurities that frequently appear in Garner's novels. The limited coverage of rules leads to loopholes in the analysis results.
[0003] As a product of the development of deep learning, physical information neural network has many key advantages; it can integrate multiple information, explore the hidden imprint of the times behind the text, and make semantic understanding more comprehensive and in-depth; at the same time, it can effectively improve the adaptability and generalization ability of the model, allowing the model to learn the changing laws of language in different situations, and when faced with new texts, it can quickly adapt to its unique style, accurately judge semantics, and overcome the limitations of traditional neural networks; enhance the interpretability of the results, make the analysis results more credible and convincing, and open up new perspectives for literary research. Summary of the invention
[0004] In view of the shortcomings of the existing methods, the present invention solves the problem that the accuracy of traditional neural network semantic analysis needs to be further improved.
[0005] The technical solution adopted by the present invention is: the language semantic analysis method based on the improved PINN network includes the following steps:
[0006] Step 1: Obtain the novel text, and perform word segmentation and annotation;
[0007] Step 2: construct a PINN network including an input layer, a hidden layer and an output layer, take the physical information feature vector as input and the paragraph sentiment as output, and train the PINN network;
[0008] As a preferred embodiment of the present invention, the physical information feature vector includes: a language description feature vector, an action description feature vector, an expression description feature vector, and an environment and object description feature vector.
[0009] Step 3: Use the novel text to be analyzed to verify the trained PINN network and output the author's topic distribution and the sentiment trend of the novel text;
[0010] As a preferred embodiment of the present invention, the loss function of the PINN network is improved, and the formula is:
[0011] L new = L + λL phy (5)
[0012] Where λ is a hyperparameter, L is the cross-entropy loss, and L phy is the loss of the physical information constraint term;
[0013]
[0014] Where j is the dimension index of the physical information feature vector p and the model output vector o.
[0015] As a preferred embodiment of the present invention, L 1 regularization is performed on the cross-entropy loss to obtain the improved cross-entropy loss ω i is the parameter of the model, and α 1 is the regularization parameter.
[0016] As a preferred embodiment of the present invention, L 2 regularization is performed on the cross-entropy loss to obtain the improved cross-entropy loss, ω i is the parameter of the model, and β is the regularization parameter.
[0017] As a preferred embodiment of the present invention, the SGD is used to update and adjust the PINN network parameters.
[0018] As a preferred embodiment of the present invention, the loss function of the PINN network further includes: the mean square error loss function.
[0019] As a preferred embodiment of the present invention, the ReLU activation function is adopted for the hidden layer of the PINN network.
[0020] As a preferred embodiment of the present invention, the Softmax activation function is adopted for the output layer of the PINN network.
[0021] As a preferred embodiment of the present invention, the language semantic analysis system based on the improved PINN network includes: a memory for storing instructions executable by a processor; and a processor for executing the instructions to implement the language semantic analysis method based on the improved PINN network.
[0022] As a preferred embodiment of the present invention, a computer-readable medium storing computer program code, and the computer program code implements the language semantic analysis method based on the improved PINN network when executed by a processor.
[0023] Advantages of the present invention:
[0024] 1. Introduce the theory of physics-informed neural networks. By designing the neural network structure, adopting a loss function with physical information constraints, and combining optimization algorithms to adjust parameters to learn the characteristic laws of works, fully integrate physical information and text information, and promote the innovation and development of literary research. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 is a flowchart of the method for language semantic analysis based on the improved PINN network of the present invention;
[0026] Figure 2 is a structural diagram of the improved PINN network;
[0027] Figure 3 is a semantic role labeling (SRL) chart generated from the passage of "The Monkey's Paw" of the present invention;
[0028] Figure 4 is a semantic similarity matrix generated from the passage of "The Monkey's Paw" of the present invention;
[0029] Figure 5 is an emotional analysis chart of the passage of "The Monkey's Paw" of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0030] The present invention will be further described below in conjunction with the drawings and embodiments. This figure is a simplified schematic diagram, which only illustrates the basic structure of the present invention in a schematic manner. Therefore, it only shows the components related to the present invention.
[0031] As Figure 1 shown, the method for language semantic analysis based on the improved PINN network includes the following steps:
[0032] Step 1: Obtain the novel text, and perform word segmentation and annotation;
[0033] In this embodiment, the novel text of Helen Garner is taken as an example for illustration. The novel text of Helen Garner is collected from public platforms, resources, and professional literary databases to ensure integrity and no copyright issues; after cleaning the text, natural language processing tools are used for word segmentation and part-of-speech annotation;
[0034] As Figure 2As shown in the figure, in Step 2, a physics-informed neural network structure (PINN network) is constructed; the input layer receives the preprocessed text data and physics information annotation data, and concatenates and inputs the physics information feature vector and the vocabulary vector; the hidden layer adopts several layers of neural network structures, and through several experiments, the performance of the model under different combinations of the number of hidden layer nodes is compared, overfitting is avoided, and the ReLU activation function is adopted. The output layer depends on the semantic analysis task and uses the Softmax activation function to output the probability distribution;
[0035] Specifically, the input layer receives the information from text preprocessing, including the preprocessed text data and physics information annotation data; the physics information feature vector, such as the feature vector related to the novel creation era, social and cultural environment, etc., is concatenated with the vocabulary vector and passed as input to the network, enabling the network to consider both the text itself and the physics information simultaneously and enhancing the semantic understanding ability;
[0036] The physics information feature vector includes: language description feature vector, action description feature vector, expression description feature vector, environment and object description feature vector;
[0037] For example, the language description feature vector maps the words with similar semantics to the positions close to each other in the vector space. For the sentence "The cat sat on the mat", for the central word "cat", the model will try to predict the surrounding words such as "The", "sat", "on", "mat", etc.; through training, the vocabulary vector of "cat" will be obtained, and it will be found that the word vectors of "cat" and "dog" are relatively close in the vector space because they are semantically similar.
[0038] The hidden layer adopts the ReLU activation function, and the formula is:
[0039] f(x) = max(0, x) (1)
[0040] When the input of the ReLU function is greater than 0, the output is equal to the input; when the input is less than or equal to 0, the output is 0; this non-linear characteristic can help the neural network learn non-linear relationships, and at the same time, compared with some other activation functions, it can alleviate the gradient vanishing problem and improve the training efficiency and network performance.
[0041] The output layer adopts the Softmax activation function, and the formula is:
[0042]
[0043] where, y i is the raw score of the i-th output unit, and K is the total number of output units.
[0044] Train the model using the labeled dataset, adopt the cross-entropy loss function, add a physical information constraint term, and combine the stochastic gradient descent optimization algorithm to continuously adjust the parameters of the neural network so that the model can accurately learn the features and rules of the works;
[0045] The mean squared error loss function can also be adopted;
[0046] Specifically, train the model using the labeled dataset and adopt the cross-entropy formula:
[0047]
[0048] where y i is the true label, is the probability predicted by the model; it measures the difference between the predicted probability distribution and the true distribution. The closer the predicted probability distribution is to the true distribution, the smaller the cross-entropy.
[0049] Add L 1 or L 2 regularization term to the standard cross-loss function to obtain the MCE loss, which can prevent the model parameters from being too large or too small and indirectly impose boundary conditions; L 1 The regularization term is L 2 The regularization term is where ω i are the parameters of the model, and α 1 and β are regularization parameters; add them to the loss function L to obtain or If the model parameter ω has multiple dimensions, by adjusting the value of the regularization parameter, the size range of the model parameter can be controlled.
[0050] Secondly, add a physical information constraint term, calculate a certain distance metric between the physical information feature vector and the model output, use the Euclidean distance as the constraint term, and the loss function formula of the physical information constraint term is:
[0051]
[0052] where j is the dimension index of the physical information feature vector p and the model output vector o.
[0053] The final loss function is a combination of cross-entropy (CE) and Euclidean distance constraint term (PIC):
[0054] L new = L + λL phy (5)
[0055] where λ is a hyperparameter used to balance the importance of the original loss function and the physical information constraint term;
[0056] Continuously adjust the parameters of the PINN network in combination with the optimization algorithm;
[0057] The SGD update formula is:
[0058]
[0059] where θ is the parameter of the model and α is the learning rate, is the gradient of the loss function with respect to the parameter; it randomly selects a sample each time to calculate the gradient and update the parameter, making the loss function gradually decrease.
[0060] Step 3: Input the preprocessed text to be analyzed into the trained model, output results such as the topic distribution and sentiment trend, and visually display them to the user through a visualization tool;
[0061] The topic distribution is the theme to which the novel belongs, such as fantasy, martial arts, etc.; the sentiment trend is the change in the sentiment of the paragraph, such as changing from a positive sentiment to a negative sentiment, etc.; the topic distributions of all novels by the same author can be classified.
[0062] Example:
[0063] To verify the feasibility and effectiveness of the improved PINN network model in analyzing the emotional characteristics of novel texts, select a chapter from Helen Garner's novel "The Monkey's Paw", and the text analysis is as follows:
[0064] This novel was created in the early 20th century, when society was filled with a physical background of fear of the unknown and the desire to explore. The improved PINN network model accurately judged that the emotional tendency of this text is negative and with a hint of awe.
[0065] Input layer information extraction;
[0066] Feature description in language: In the text, the old man said "I don't know what to do", which directly shows his inner confusion and helplessness. This is a clear emotional feature input, and corresponding to the neural network, it can be used as an important feature vector;
[0067] Feature description of action: "His hand nervously rubbed the dry monkey's paw", the nervous action of the hand is a non-verbal emotional expression, which can be used as an auxiliary emotional feature input in the neural network, reflecting the old man's inner uneasiness and anxiety;
[0068] Feature description of expression: "His eyes were shining with fear and hesitation", the description of the expression further strengthens the emotional information; the two emotional state features of fear and hesitation can be used as specific emotional labels in the neural network, corresponding to different neuron activation patterns;
[0069] Environmental and object description features: "The monkey's paw just lay quietly on the table, as if carrying some mysterious magic power, emitting an unsettling atmosphere" and "In the darkness, it seemed that countless pairs of eyes were peeping at it". The descriptions of the monkey's paw and the dark environment create an eerie and terrifying atmosphere; these environmental information can serve as background factors for emotion generation in the neural network, affecting the overall emotion judgment;
[0070] Analysis and processing of contradictory emotion features in the hidden layer: The integration of contradictory emotions, the old man "both afraid and curious, on the one hand eager to change his poor situation, on the other hand worried that it will trigger unpredictable disasters". This contradictory emotion will undergo complex calculations and integrations in the hidden layer of the neural network; the two pairs of contradictory emotion factors of fear and curiosity, eagerness and worry will interact with each other in the connection and weight adjustment of the hidden layer neurons, forming a unique emotion feature vector;
[0071] Reinforcement and inhibition of emotions: The terrifying atmosphere brought by the environmental description and the description of the mysterious magic power of the monkey's paw will strengthen the old man's fear emotion in the hidden layer; while the eagerness to change the poor situation suppresses part of the fear to a certain extent and conflicts with the emotion of worrying about disasters at the same time. The complex relationships between these emotions are simulated and processed in the hidden layer through the activation and inhibition of neurons;
[0072] The overall emotion presented in the output layer is mainly confusion and entanglement; based on the above analysis of the input layer and the hidden layer, the main emotions presented in the output layer are confusion and entanglement; facing the unknown power represented by the monkey's paw, the old man is unable to determine how to make a choice. This confused and entangled emotion is output in a quantifiable emotion intensity through the calculation and processing of the entire neural network;
[0073] Interweaving of fear and eagerness: Although the two emotions of fear and eagerness are contradictory, they also coexist and interweave in the output layer; fear stems from the worry about unknown disasters, and eagerness comes from the urgent need to change the status quo. They jointly constitute the old man's complex emotional state and are clearly shown through the output of the neural network.
[0074] The semantic role labeling (SRL) chart generated by the "The Monkey's Paw" paragraph is as Figure 3 shown; the semantic similarity matrix generated by the "The Monkey's Paw" paragraph is as Figure 4 shown, and the emotion analysis generated by the "The Monkey's Paw" paragraph is as Figure 5 shown. Experimental results:
[0075] In an experiment on emotion classification of novels, the following two loss functions are used for comparison: namely, the traditional standard cross-entropy loss function CE and the loss function (MCE+PIC) that combines the improved cross-entropy loss and physical information constraint terms;
[0076] Table 1 Experimental Results
[0077] Loss function Precision Recall F1 score CE 78％ 75％ 76.5％ MCE + PIC 83％ 81％ 82％
[0078] Based on the text analysis obtained from the theory of physics-informed neural networks, the present invention updates the search for relationships such as sentiment, theme, and characters by means of SGD, thereby predicting the language semantic analysis of the text. Considering that the text semantics are rich and implicit, a physical information constraint term is added to calculate the uncertainty range of the prediction result, which helps to better improve the credibility of the prediction result and provides a more valuable reference for literary research.
[0079] Taking the above-described ideal embodiment of the present invention as an inspiration, through the above description, relevant staff can completely make various changes and modifications without departing from the technical idea of the present invention. The technical scope of the present invention is not limited to the content in the specification, and its technical scope must be determined according to the scope of the claims.
Claims
1. A language semantic analysis method based on an improved PINN network, characterized in that: The following steps are involved: Step 1: Obtain the novel text, and perform word segmentation and annotation; Step 2: construct a PINN network including an input layer, a hidden layer and an output layer, take the physical information feature vector as input and the paragraph sentiment as output, and train the PINN network; Step 3: Use the novel text to be analyzed to verify the trained PINN network and output the author's topic distribution and the sentiment trend of the novel text.
2. The language semantic analysis method based on the improved PINN network according to claim 1 is characterized in that: The loss function of the PINN network is improved, and the formula is: THE new =L+λL phy (5) Among them, λ is a hyperparameter, L is the cross entropy loss, and L phy is the loss of physical information constraints; Among them, j is the dimension index of the physical information feature vector p and the model output vector o.
3. The language semantic analysis method based on the improved PINN network according to claim 2 is characterized in that: Perform L1 regularization on the cross entropy loss to obtain the improved cross entropy loss ω i is the parameter of the model, and α1 is the regularization parameter.
4. The language semantic analysis method based on the improved PINN network according to claim 2 is characterized in that: The cross entropy loss is regularized by L2 to obtain the improved cross entropy loss. ω i is the parameter of the model and β is the regularization parameter.
5. The language semantic analysis method based on the improved PINN network according to claim 1 is characterized in that: SGD is used to update and adjust the PINN network parameters.
6. The language semantic analysis method based on the improved PINN network according to claim 1 is characterized in that: The loss function of the PINN network also includes: mean square error loss function.
7. The language semantic analysis method based on the improved PINN network according to claim 1 is characterized in that: The hidden layer of the PINN network uses the ReLU activation function.
8. The language semantic analysis method based on the improved PINN network according to claim 1 is characterized in that: The output layer of the PINN network uses the Softmax activation function.
9. The language semantic analysis method based on the improved PINN network according to claim 1 is characterized in that: The physical information feature vectors include: language description feature vectors, action description feature vectors, expression description feature vectors and environment and object description feature vectors.
10. A language semantic analysis system based on an improved PINN network, characterized in that: include: a memory for storing instructions executable by a processor; A processor, configured to execute instructions to implement the language semantic analysis method based on the improved PINN network as described in any one of claims 1 to 9.
11. A computer readable medium storing a computer program code, characterized in that: When the computer program code is executed by a processor, the language semantic analysis method based on the improved PINN network is implemented as described in any one of claims 1 to 9.