High-performance semantic recognition method

Abstract

The invention discloses a high-performance semantic recognition method, and belongs to the field of natural language semantic recognition. The method comprises the following steps: firstly, converting a statement into a corresponding word vector sequence according to a word order, and then inputting word vectors into a semantic recognition network to obtain a recognition result. The semantic recognition network comprises a plurality of semantic recognition units which are arranged in sequence; the semantic recognition unit comprises a hidden layer neuron, a value prediction network, an output neuron and a classifier. And during calculation, determining whether each unit is reserved or not according to the predicted value. And after all word vectors are processed, taking a classification result output by the classifier in the last semantic recognition unit marked as'retention 'as a semantic recognition result of the statement. According to the method, the operand can be reduced under the condition that the semantic features are not lost, and the semantic recognition performance is remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the field of natural language semantic recognition, and in particular relates to a high-performance semantic recognition method. Background Art

[0002] Natural language semantic recognition (classification) refers to the process of analyzing text or speech content using computer technology to identify its semantic category or intent. This technology has widespread applications in areas such as intelligent customer service, search engines, ad recommendations, and voice assistants. Efficient semantic recognition can significantly enhance the human-computer interaction experience, but its performance is highly dependent on the computational effort and efficiency of the underlying algorithms.

[0003] Currently, mainstream semantic recognition solutions are primarily based on deep learning models, such as the Transformer architecture or support vector machines (LSVMs). Chinese invention patent application publication number CN120218083A discloses a big data-based text semantic recognition system and method. This system collects text data through a data acquisition module and uses a trained semantic recognition model (such as a Transformer) to perform semantic analysis on newly input text. However, such models suffer from large parameter counts and low computational efficiency. Specifically, such methods typically calculate all input features (such as every character or word in a sentence) equally, without distinguishing between critical and redundant information. For example, in the sentence "I want to eat at a certain restaurant in a certain snack street," the keyword that truly determines the semantic category ("catering") is "to eat," while modifiers such as "I want" and "certain snack street, certain restaurant" have little impact on the classification result. However, existing models still require the complete encoding and calculation of these non-critical features, resulting in increased network layers and computational complexity, which in turn reduces overall performance. Summary of the Invention

[0004] The present invention proposes a high-performance semantic recognition method, the purpose of which is to improve the performance of semantic recognition.

[0005] The technical solutions of the present invention are as follows:

[0006] A high-performance semantic recognition method that first converts sentences into corresponding word vector sequences according to word order, and then inputs the word vectors into a semantic recognition network to obtain recognition results;

[0007] The semantic recognition network includes a plurality of sequentially arranged semantic recognition units; the semantic recognition units include hidden layer neurons, a value prediction network, output neurons, and a classifier; the hidden layer neurons obtain a state vector based on a first input vector and a second input vector of the current semantic recognition unit; the value prediction network obtains a value prediction value based on the state vector; the output neurons obtain a state transition output value based on the state vector; and the classifier obtains a classification result based on the state transition output value;

[0008] When inputting, for Semantic recognition unit, :if , then its first input vector is the first word vector of the word vector sequence, and the second input vector is the second word vector of the word vector sequence; if , then its first input vector is the state vector obtained in the previous semantic recognition unit, and the second input vector is the first word vectors;

[0009] The first semantic recognition unit is marked as "retained"; starting from the second semantic recognition unit, after each semantic recognition unit obtains the value prediction value of this unit, it is compared with the value prediction value of the previous semantic recognition unit. If the value prediction value of this semantic recognition unit is greater than or equal to the value prediction value of the previous semantic recognition unit, then the semantic recognition unit is marked as "retained", otherwise it is marked as "discarded"; after all word vectors are processed, the classification result output by the classifier in the last semantic recognition unit marked as "retained" is used as the semantic recognition result of the sentence.

[0010] As a further improvement of the high-performance semantic recognition method: the number of neurons in the hidden layer is consistent with the dimensions of the first input vector and the second input vector, and both are .

[0011] As a further improvement of the high-performance semantic recognition method, the hidden layer neurons obtain the state vector The process is:

[0012] Step A-1: ​​Each neuron in the hidden layer performs the following calculations: neurons, and the intermediate feature vector is obtained by No. element value :

[0013] ;

[0014] in, is the hyperbolic tangent function, is the first The weights of neurons, is the first input vector of the current semantic recognition unit elements, is the first The bias of each neuron;

[0015] Step A-2: Each neuron in the hidden layer performs the following calculations: a state vector of the neuron is obtained by the first element value of the state vector :

[0016] ;

[0017] wherein, is the first element of the second input vector of the current semantic recognition unit;

[0018] Thus, the complete state vector is obtained.

[0019] As a further improvement of the high-performance semantic recognition method:

[0020] The value prediction network is a feedforward neural network, whose input is the state vector and whose output is the value prediction value;

[0021] The output neuron converts the state vector into a state transition output value through linear transformation and activation function;

[0022] The classifier is a function, whose input is the state transition output value , and whose output is a probability vector, wherein the value of each element represents the probability that the current state vector belongs to the corresponding category, and the sum of all elements is 1.

[0023] As a further improvement of the high-performance semantic recognition method, the training method of the semantic recognition network is:

[0024] Step T-1, prepare a sample sentence data set and label each sample sentence with a true label;

[0025] Step T-2, train the hidden layer neurons of each semantic recognition unit based on a random skip mechanism;

[0026] Step T-3, train the value prediction network of each semantic recognition unit based on a value judgment and a random double decision mechanism.

[0027] As a further improvement of the high-performance semantic recognition method: in step T-2, iterate the following steps several times, each time randomly selecting a different sample sentence:

[0028] Step T-2-1, convert the sample sentence into a corresponding sample word vector sequence according to the word order;

[0029] Step T-2-2, input the sample word vector in the sample word vector sequence into the semantic recognition network for calculation; each semantic recognition unit determines whether to skip the unit based on the random skip mechanism before starting calculation;

[0030] Step T-2-3, take the classification result output by the classifier in the last semantic recognition unit that is not skipped as the prediction result this time, then calculate the loss based on the prediction result and the true label corresponding to the sample sentence, and update the network parameters of the hidden layer neurons of the semantic recognition unit that is not skipped this time based on the loss.

[0031] As a further improvement of the high-performance semantic recognition method, the default input rule in step T-2-2 is: for the first semantic recognition unit, : if , the first input vector is the first sample word vector in the sample word vector sequence, and the second input vector is the second sample word vector in the sample word vector sequence; if , the first input vector is the state vector obtained in the previous semantic recognition unit, and the second input vector is the sample word vector in the sample word vector sequence;

[0032] In step T-2-2, after each semantic recognition unit determines whether to skip, it takes corresponding processing mode according to the determination:

[0033] Case a1, the current semantic recognition unit determines "not to skip" and the current semantic recognition unit is the first semantic recognition unit, then input according to the default input rule, and then perform calculation;

[0034] Case a2, the current semantic recognition unit determines "not to skip" and the current semantic recognition unit is the second or subsequent semantic recognition unit, then set the first input vector to the state vector obtained in the previous semantic recognition unit that determines "not to skip", if all the previous semantic recognition units are "skipped", use the state vector of the first semantic recognition unit as the first input vector, input the second input vector according to the default input rule, and then perform calculation;

[0035] Case a3, the current semantic recognition unit determines "to skip" and the current semantic recognition unit is the first semantic recognition unit, then input the first input vector according to the default input rule, set the second input vector to the zero vector, and then perform calculation;

[0036] Case a4, the current semantic recognition unit determines "to skip" and the current semantic recognition unit is the second or subsequent semantic recognition unit, then directly skip the calculation of the unit.

[0037] ​As a further improvement of the high-performance semantic recognition method, the specific steps of step T-3 are:

[0038] Step T-3-1, Initialize Probability ;

[0039] Step T-3-2: Iterate the following steps several times, randomly selecting different sample sentences each time:

[0040] Step T-3-2-1: Convert the sample sentence into the corresponding sample word vector sequence according to word order;

[0041] Step T-3-2-2: Input the sample word vectors in the sample word vector sequence into the semantic recognition network for calculation to determine whether each semantic recognition unit is retained;

[0042] Step T-3-2-3: All semantic recognition units marked as "retained" are sequentially formed into a valid unit sequence, recorded as , Indicates the Semantic recognition units marked as "retained" are selected; then, the valid unit sequence is traversed, the value error of each semantic recognition unit is calculated, and the network parameters of the value prediction network of the corresponding semantic recognition unit are updated based on the value error;

[0043] Step T-3-3: Determine Probability Is it 0? If it is 0, end step T-3, otherwise reduce the probability and ensure , then skip to step T-3-2.

[0044] As a further improvement of the high performance semantic recognition method: In step T-3-2-2, the default input rule is: Semantic recognition unit, :if , then its first input vector is the first sample word vector of the sample word vector sequence, and the second input vector is the second sample word vector of the sample word vector sequence; if , then its first input vector is the state vector obtained in the previous semantic recognition unit, and the second input vector is the first Sample word vectors;

[0045] In step T-3-2-2, each semantic recognition unit first calculates the probability Determines whether the current semantic recognition unit adopts value function decision or random decision. The probability of value function decision is , the probability of random decision is , and then take corresponding measures according to the decision:

[0046] Case b1: If the current semantic recognition unit decides to adopt the value function decision and the current semantic recognition unit is the first semantic recognition unit, then input is performed according to the default input rule, and then calculation is performed, and the current semantic recognition unit is marked as "retained";

[0047] Case b2: If the current semantic recognition unit decides to adopt the value function decision and the current semantic recognition unit is the second or subsequent semantic recognition unit, then its first input vector is set to the state vector obtained in the previous semantic recognition unit marked as "retained", and the second input vector is input according to the default input rule. Then, the calculation is performed, and the value prediction value obtained by the current semantic recognition unit is compared with the value prediction value obtained in the previous semantic recognition unit marked as "retained". If the current value prediction value is greater than or equal to the previous value prediction value, the current semantic recognition unit is marked as "retained", otherwise it is marked as "discarded";

[0048] Case b3: If the current semantic recognition unit decides to adopt random decision and the decision result is "not skipped" and the current semantic recognition unit is the first semantic recognition unit, then input is performed according to the default input rule, and then calculation is performed, and the current semantic recognition unit is marked as "retained";

[0049] Case b4: If the current semantic recognition unit decides to use random decision-making, the decision result is "not skipped", and the current semantic recognition unit is the second or subsequent semantic recognition unit, then its first input vector is set to the state vector obtained from the previous semantic recognition unit marked as "retained", and the second input vector is input according to the default input rule. Then, the calculation is performed and the current semantic recognition unit is marked as "retained".

[0050] Case b5: The current semantic recognition unit decides to adopt random decision, the decision result is "skip", and the current semantic recognition unit is the first semantic recognition unit. Then the first input vector is input according to the default input rule, the second input vector is set to the zero vector, and then the calculation is performed, and the current semantic recognition unit is marked as "retained";

[0051] Case b6: If the current semantic recognition unit decides to adopt random decision, the decision result is "skip", and the current semantic recognition unit is the second or subsequent semantic recognition unit, no calculation is performed, and the current semantic recognition unit is marked as "discarded".

[0052] As a further improvement of the high-performance semantic recognition method: in step T-3-2-3, for the first The value error of a semantic recognition unit is calculated as follows:

[0053] ;

[0054] ;

[0055] in, The first unit in the valid unit sequence The value prediction value output by the value prediction network in the semantic recognition unit, The first unit in the valid unit sequence The value training value of a semantic recognition unit, The first unit in the valid unit sequence The state transition output value of the semantic recognition unit, is the value prediction value output by the value prediction network in the last semantic recognition unit in the valid unit sequence, is the discount factor, .

[0056] Compared with the prior art, the present invention has the following positive effects:

[0057] 1. The present invention adopts a chain decision mechanism based on value prediction comparison, retaining only semantic units with increasing value, realizing progressive feature compression of sentences and dynamic adjustment of calculation paths. In the calculation process, the compressed deep semantic features are retained through the state vector transmission of hidden neurons, and the results are finally output through the classifier of the last retained unit. Compared with the traditional Transformer or LSVM architecture, the amount of computation is reduced while ensuring that semantic features are not lost, thereby significantly improving the performance of semantic recognition.

[0058] 2. The training of this invention is divided into two phases: the first phase focuses on the feature extraction capabilities of hidden layer neurons and ensures breadth through a random skipping mechanism, laying the foundation for the second phase of training the value prediction network. In the second phase, a dual decision-making mechanism of value judgment and randomness is introduced. By adjusting the dynamic decay of probability, a smooth transition from exploration to utilization is achieved. The early stage focuses on exploration to avoid falling into local optimality. Then, the value prediction network gradually takes over the decision-making leadership, and finally, the value decision-making is completely dominant to ensure the stability of convergence.

[0059] 3. In the second phase of training, the value prediction network is optimized through a temporal difference learning mechanism. The training value of each semantic recognition unit is determined by the current classification confidence and the discounted accumulation of future semantic values. This design enables the model to dynamically balance immediate classification accuracy with long-range semantic dependencies, significantly improving training efficiency and significantly enhancing robustness to truncated samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 This is the architecture diagram of the semantic recognition unit in the present invention. DETAILED DESCRIPTION

[0061] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the embodiments described are only a part of the embodiments of the present invention, rather than all the embodiments.

[0062] A high-performance semantic recognition method, the steps are as follows: first, convert the sentence into a corresponding word vector sequence according to the word order, and then input the word vector into the semantic recognition network to obtain the recognition result.

[0063] The semantic recognition network includes a plurality of sequentially arranged semantic recognition units.

[0064] like Figure 1 The semantic recognition unit includes hidden neurons, a value prediction network, output neurons, and a classifier. The hidden neurons obtain a state vector based on the first input vector and the second input vector of the current semantic recognition unit. The value prediction network obtains a value prediction value based on the state vector. The output neurons obtain a state transition output value based on the state vector. The classifier obtains a classification result based on the state transition output value.

[0065] When inputting, for Semantic recognition unit, :if , then its first input vector is the first word vector of the word vector sequence, and the second input vector is the second word vector of the word vector sequence; if , then its first input vector is the state vector obtained in the previous semantic recognition unit, and the second input vector is the first word vectors.

[0066] The first semantic recognition unit is marked as "retained". Starting from the second semantic recognition unit, after each semantic recognition unit obtains the value prediction value of this unit, it is compared with the value prediction value of the previous semantic recognition unit. If the value prediction value of this semantic recognition unit is greater than or equal to the value prediction value of the previous semantic recognition unit, the semantic recognition unit is marked as "retained", otherwise it is marked as "discarded". After all word vectors are processed, the classification result output by the classifier in the last semantic recognition unit marked as "retained" is used as the semantic recognition result of the sentence.

[0067] The specific structure of the semantic recognition unit is:

[0068] 1. Hidden layer neurons

[0069] The number of neurons in the hidden layer is consistent with the dimensions of the first and second input vectors, and both are , the hidden layer neurons get the state vector The process is:

[0070] Step A-1: ​​Each neuron in the hidden layer performs the following calculations: neurons, and the intermediate feature vector is obtained by No. element value :

[0071] ;

[0072] in, is the hyperbolic tangent function, is the first The weights of neurons, is the first input vector of the current semantic recognition unit elements, is the first The bias of a neuron.

[0073] Step A-2: Each neuron in the hidden layer performs the following calculations: neurons, the state vector is obtained by No. element value :

[0074] ;

[0075] in, is the second input vector of the current semantic recognition unit elements.

[0076] At this point, the complete state vector is obtained .

[0077] 2. Value Prediction Network

[0078] In this embodiment, the value prediction network is a feedforward neural network (FNN), whose input is a state vector and output is a value prediction value.

[0079] 3. Output Neuron

[0080] The output neuron transforms the state vector Convert to state transition output value .

[0081] 4. Classifier

[0082] The classifier is Function, whose input is the state transition output value , output Is a probability vector, where the value of each element represents the probability that the current state vector belongs to the corresponding category, and the sum of all elements is 1.

[0083] Furthermore, the semantic recognition network is trained as follows:

[0084] Step T-1: Prepare a sample sentence dataset and mark the true label for each sample sentence.

[0085] Preferably, a continuous portion of the original sentence can be randomly intercepted as a sample sentence with a fixed length L. For example, the original sentence "We go to Renmin Road for lunch" can be randomly intercepted with a fixed length L=5 to extract sample sentences such as "Go to Renmin for lunch" and "Eat at Renmin Road".

[0086] Step T-2: training the hidden layer neurons of each semantic recognition unit based on a random skipping mechanism.

[0087] During training, follow these steps several times (e.g., 1000 times), randomly selecting different sample sentences each time:

[0088] Step T-2-1: Convert the sample sentence into the corresponding sample word vector sequence according to word order;

[0089] Step T-2-2: Input the sample word vectors in the sample word vector sequence into the semantic recognition network for calculation.

[0090] The default input rule is: Semantic recognition unit, :if , then its first input vector is the first sample word vector of the sample word vector sequence, and the second input vector is the second sample word vector of the sample word vector sequence; if , then its first input vector is the state vector obtained in the previous semantic recognition unit, and the second input vector is the first Sample word vectors.

[0091] Before starting calculation, each semantic recognition unit decides whether to skip the unit based on a random skipping mechanism, with a skipping probability of 50%:

[0092] Case a1: If the current semantic recognition unit decides not to skip and the current semantic recognition unit is the first semantic recognition unit, input is performed according to the default input rule, and then calculation is performed.

[0093] Case a2: If the current semantic recognition unit decides to "not skip" and the current semantic recognition unit is the second or subsequent semantic recognition unit, then its first input vector is set to the state vector obtained in the previous semantic recognition unit that decided to "not skip". If the previous semantic recognition units are all "skipped", the state vector of the first semantic recognition unit is used as the first input vector, and the second input vector is input according to the default input rule, and then the calculation is performed.

[0094] Case a3: The current semantic recognition unit decides to "skip" and the current semantic recognition unit is the first semantic recognition unit, then the first input vector is input according to the default input rule, the second input vector is set to the zero vector, and then the calculation is performed.

[0095] Case a4: If the current semantic recognition unit decides to "skip" and the current semantic recognition unit is the second or subsequent semantic recognition unit, the calculation of this unit will be skipped directly.

[0096] Step T-2-3: Use the classification result output by the classifier in the last non-skipped semantic recognition unit as the prediction result this time, then calculate the loss based on the prediction result and the true label corresponding to the sample sentence, and then update the network parameters of the hidden layer neurons of the non-skipped semantic recognition unit based on the loss.

[0097] The loss may be a cross entropy loss, and the updating method may be a gradient descent method.

[0098] Step T-3: Train the value prediction network of each semantic recognition unit based on the dual decision-making mechanism of value judgment and randomness.

[0099] The dual decision-making mechanism of value judgment and randomness includes value function decision and random decision. Both decisions are used to determine whether each semantic recognition unit participates in the calculation during training. As the training progresses, the proportion of random decisions becomes lower and lower, and the proportion of value function decisions becomes higher and higher.

[0100] The specific steps are:

[0101] Step T-3-1, Initialize Probability In this embodiment, the probability The initial value is 98%.

[0102] Step T-3-2: Iterate the following steps several times (e.g., 1000 times), randomly selecting a different sample sentence each time:

[0103] Step T-3-2-1: Convert the sample sentence into the corresponding sample word vector sequence according to word order;

[0104] Step T-3-2-2: Input the sample word vectors in the sample word vector sequence into the semantic recognition network for calculation.

[0105] The default input rule is: Semantic recognition unit, :if , then its first input vector is the first sample word vector of the sample word vector sequence, and the second input vector is the second sample word vector of the sample word vector sequence; if , then its first input vector is the state vector obtained in the previous semantic recognition unit, and the second input vector is the first Sample word vectors.

[0106] Before starting calculation, each semantic recognition unit first calculates Determines whether the current semantic recognition unit adopts value function decision or random decision. The probability of value function decision is , the probability of random decision is :

[0107] Case b1: If the current semantic recognition unit decides to adopt the value function decision and the current semantic recognition unit is the first semantic recognition unit, input is performed according to the default input rule, and then calculation is performed, and the current semantic recognition unit is marked as "retained".

[0108] Case b2: If the current semantic recognition unit decides to adopt the value function decision and the current semantic recognition unit is the second or subsequent semantic recognition unit, then its first input vector is set to the state vector obtained in the previous semantic recognition unit marked as "retain", and the second input vector is input according to the default input rule. Then the calculation is performed, and the value prediction value obtained by the current semantic recognition unit is compared with the value prediction value obtained in the previous semantic recognition unit marked as "retain". If the current value prediction value is greater than or equal to the previous value prediction value, the current semantic recognition unit is marked as "retain", otherwise it is marked as "discarded".

[0109] Case b3: The current semantic recognition unit decides to adopt random decision, the decision result is "not skip" and the current semantic recognition unit is the first semantic recognition unit, then input is performed according to the default input rule, and then calculation is performed, and the current semantic recognition unit is marked as "retained".

[0110] Case b4: The current semantic recognition unit decides to adopt a random decision, the decision result is "do not skip" and the current semantic recognition unit is the second or subsequent semantic recognition unit, then its first input vector is set to the state vector obtained in the previous semantic recognition unit marked as "retain", the second input vector is input according to the default input rule, and then the calculation is performed, and the current semantic recognition unit is marked as "retain".

[0111] Case b5: The current semantic recognition unit decides to adopt a random decision, the decision result is "skip" and the current semantic recognition unit is the first semantic recognition unit. The first input vector is input according to the default input rule, the second input vector is set to the zero vector, and then the calculation is performed, and the current semantic recognition unit is marked as "retained".

[0112] Case b6: If the current semantic recognition unit decides to adopt random decision, the decision result is "skip", and the current semantic recognition unit is the second or subsequent semantic recognition unit, no calculation is performed, and the current semantic recognition unit is marked as "discarded".

[0113] When random decision making is used, the probability of "skipping" and "not skipping" is 50%.

[0114] Step T-3-2-3: All semantic recognition units marked as "retained" are sequentially formed into a valid unit sequence, recorded as , Indicates the Then, the valid unit sequence is traversed, the value error of each semantic recognition unit is calculated, and the network parameters of the value prediction network of the corresponding semantic recognition unit are updated based on the value error.

[0115] Specifically, for the first The value error of a semantic recognition unit is calculated as follows:

[0116] ;

[0117] ;

[0118] in, The first unit in the valid unit sequence The value prediction value output by the value prediction network in the semantic recognition unit, The first unit in the valid unit sequence The value training value of a semantic recognition unit, The first unit in the valid unit sequence The state transition output value of the semantic recognition unit, is the value prediction value output by the value prediction network in the last semantic recognition unit in the valid unit sequence, is the discount factor, .

[0119] Step T-3-3: Determine the probability Is it 0? If it is 0, end step T-3, otherwise reduce the probability and ensure , then skip to step T-3-2.

[0120] During network calculations, you can decide whether to complete all calculations of the value prediction network, output neurons, and classifiers as needed. For example, during normal reasoning, only the last remaining semantic recognition unit needs to perform output neuron and classifier calculations.

[0121] It should be noted that it is obvious to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. The scope of the present invention is defined by the claims rather than the foregoing description.

Claims

1. A high-performance semantic recognition method that first converts a sentence into a corresponding word vector sequence according to word order, and then inputs the word vector into a semantic recognition network to obtain a recognition result. The method is characterized by: The semantic recognition network includes a plurality of sequentially arranged semantic recognition units; the semantic recognition units include hidden layer neurons, a value prediction network, output neurons, and a classifier; the hidden layer neurons obtain a state vector based on a first input vector and a second input vector of the current semantic recognition unit; the value prediction network obtains a value prediction value based on the state vector; the output neurons obtain a state transition output value based on the state vector; and the classifier obtains a classification result based on the state transition output value; When inputting, for Semantic recognition unit, :if , then its first input vector is the first word vector of the word vector sequence, and the second input vector is the second word vector of the word vector sequence; if , then its first input vector is the state vector obtained in the previous semantic recognition unit, and the second input vector is the first word vectors; The first semantic recognition unit is marked as "retained"; starting from the second semantic recognition unit, after each semantic recognition unit obtains the value prediction value of this unit, it is compared with the value prediction value of the previous semantic recognition unit. If the value prediction value of this semantic recognition unit is greater than or equal to the value prediction value of the previous semantic recognition unit, the semantic recognition unit is marked as "retained", otherwise it is marked as "discarded"; after all word vectors are processed, the classification result output by the classifier in the last semantic recognition unit marked as "retained" is used as the semantic recognition result of the sentence.

2. The high-performance semantic recognition method according to claim 1, characterized in that: The number of neurons in the hidden layer is consistent with the dimensions of the first and second input vectors, and both are .

3. The high-performance semantic recognition method according to claim 2, characterized in that: The hidden layer neurons get the state vector The process is: Step A-1: ​​Each neuron in the hidden layer performs the following calculations: neurons, and the intermediate feature vector is obtained by No. element value : ； in, is the hyperbolic tangent function, is the first The weight of the neurons, is the first input vector of the current semantic recognition unit elements, is the first The bias of each neuron; Step A-2: Each neuron in the hidden layer performs the following calculations: neurons, the state vector is obtained by No. element value : ； in, is the second input vector of the current semantic recognition unit elements; At this point, the complete state vector is obtained .

4. The high-performance semantic recognition method according to claim 1, wherein: The value prediction network is a feedforward neural network, whose input is a state vector and output is a value prediction value; The output neuron transforms the state vector Convert to state transition output value ; The classifier is Function, whose input is the state transition output value , output Is a probability vector, where the value of each element represents the probability that the current state vector belongs to the corresponding category, and the sum of all elements is 1.

5. The high-performance semantic recognition method according to claim 1, wherein: The training method of the semantic recognition network is: Step T-1: Prepare a sample sentence dataset and mark the true label for each sample sentence; Step T-2: training the hidden neurons of each semantic recognition unit based on a random skipping mechanism; Step T-3: Train the value prediction network of each semantic recognition unit based on the dual decision-making mechanism of value judgment and randomness.

6. The high-performance semantic recognition method according to claim 5, characterized in that: In step T-2, the following steps are iterated several times, randomly selecting different sample sentences each time: Step T-2-1: Convert the sample sentence into the corresponding sample word vector sequence according to word order; Step T-2-2: Input the sample word vectors in the sample word vector sequence into the semantic recognition network for calculation; before starting the calculation, each semantic recognition unit decides whether to skip the unit based on the random skip mechanism; Step T-2-3: Use the classification result output by the classifier in the last non-skipped semantic recognition unit as the prediction result this time, then calculate the loss based on the prediction result and the true label corresponding to the sample sentence, and then update the network parameters of the hidden layer neurons of the non-skipped semantic recognition unit based on the loss.

7. The high-performance semantic recognition method according to claim 6, characterized in that: In step T-2-2, the default input rule is: Semantic recognition unit, :if , then its first input vector is the first sample word vector of the sample word vector sequence, and the second input vector is the second sample word vector of the sample word vector sequence; if , then its first input vector is the state vector obtained in the previous semantic recognition unit, and the second input vector is the first Sample word vectors; In step T-2-2, after each semantic recognition unit decides whether to skip, it takes the corresponding processing method according to the decision: Case a1: If the current semantic recognition unit decides not to skip and is the first semantic recognition unit, input is performed according to the default input rule and then calculation is performed; Case a2: If the current semantic recognition unit decides to "not skip" and is the second or subsequent semantic recognition unit, its first input vector is set to the state vector obtained by the previous semantic recognition unit that decided to "not skip". If all previous semantic recognition units decided to "skip", the state vector of the first semantic recognition unit is used as the first input vector. The second input vector is input according to the default input rule, and then the calculation is performed; Case a3: If the current semantic recognition unit decides to "skip" and the current semantic recognition unit is the first semantic recognition unit, the first input vector is input according to the default input rule, the second input vector is set to the zero vector, and then the calculation is performed; Case a4: If the current semantic recognition unit decides to "skip" and the current semantic recognition unit is the second or subsequent semantic recognition unit, the calculation of this unit will be skipped directly.

8. The high-performance semantic recognition method according to claim 5, characterized in that: The specific steps of step T-3 are: Step T-3-1, Initialize Probability ; Step T-3-2: Iterate the following steps several times, randomly selecting different sample sentences each time: Step T-3-2-1: Convert the sample sentence into the corresponding sample word vector sequence according to word order; Step T-3-2-2: Input the sample word vectors in the sample word vector sequence into the semantic recognition network for calculation to determine whether each semantic recognition unit is retained; Step T-3-2-3: All semantic recognition units marked as "retained" are sequentially formed into a valid unit sequence, recorded as , Indicates the Semantic recognition units marked as "retained" are selected; then, the valid unit sequence is traversed, the value error of each semantic recognition unit is calculated, and the network parameters of the value prediction network of the corresponding semantic recognition unit are updated based on the value error; Step T-3-3: Determine Probability Is it 0? If it is 0, end step T-3, otherwise reduce the probability and ensure , then skip to step T-3-2.

9. The high-performance semantic recognition method according to claim 8, characterized in that: In step T-3-2-2, the default input rule is: Semantic recognition unit, :if , then its first input vector is the first sample word vector of the sample word vector sequence, and the second input vector is the second sample word vector of the sample word vector sequence; if , then its first input vector is the state vector obtained in the previous semantic recognition unit, and the second input vector is the first Sample word vectors; In step T-3-2-2, each semantic recognition unit first calculates the probability Determines whether the current semantic recognition unit adopts value function decision or random decision. The probability of value function decision is , the probability of random decision is , and then take corresponding measures according to the decision: Case b1: If the current semantic recognition unit decides to use the value function decision and the current semantic recognition unit is the first semantic recognition unit, then the input is input according to the default input rule, and then the calculation is performed, and the current semantic recognition unit is marked as "retained"; Case b2: If the current semantic recognition unit decides to adopt the value function decision and the current semantic recognition unit is the second or subsequent semantic recognition unit, then its first input vector is set to the state vector obtained in the previous semantic recognition unit marked as "retain", and the second input vector is input according to the default input rule. Then, the calculation is performed, and the value prediction value obtained by the current semantic recognition unit is compared with the value prediction value obtained by the previous semantic recognition unit marked as "retain". If the current value prediction value is greater than or equal to the previous value prediction value, the current semantic recognition unit is marked as "retain", otherwise it is marked as "discard"; Case b3: If the current semantic recognition unit decides to use random decision-making and the decision result is "not skipped" and the current semantic recognition unit is the first semantic recognition unit, then input is performed according to the default input rule, and then calculation is performed, and the current semantic recognition unit is marked as "retained"; Case b4: If the current semantic recognition unit decides to use random decision-making, the decision result is "not skipped", and the current semantic recognition unit is the second or subsequent semantic recognition unit, then its first input vector is set to the state vector obtained from the previous semantic recognition unit marked as "retained". The second input vector is input according to the default input rule, and then the calculation is performed. The current semantic recognition unit is marked as "retained"; Case b5: The current semantic recognition unit decides to use random decision, the decision result is "skip", and the current semantic recognition unit is the first semantic recognition unit. Then the first input vector is input according to the default input rule, the second input vector is set to the zero vector, and then the calculation is performed. The current semantic recognition unit is marked as "retained"; Case b6: If the current semantic recognition unit decides to adopt random decision, the decision result is "skip", and the current semantic recognition unit is the second or subsequent semantic recognition unit, no calculation is performed and the current semantic recognition unit is marked as "discarded".

10. The high-performance semantic recognition method according to claim 8, characterized in that: In step T-3-2-3, for the first The value error of a semantic recognition unit is calculated as follows: ； ； in, The first unit in the valid unit sequence The value prediction value output by the value prediction network in the semantic recognition unit, The first unit in the valid unit sequence The value training value of a semantic recognition unit, The first unit in the valid unit sequence The state transition output value of the semantic recognition unit, is the value prediction value output by the value prediction network in the last semantic recognition unit in the valid unit sequence, is the discount factor, .

Images