A high-performance semantic recognition method

Abstract

This invention discloses a high-performance semantic recognition method, belonging to the field of natural language semantic recognition. The method first converts a sentence into a corresponding word vector sequence according to word order, and then inputs the word vectors into a semantic recognition network to obtain the recognition result. The semantic recognition network includes several sequentially arranged semantic recognition units; each semantic recognition unit includes hidden neurons, a value prediction network, an output neuron, and a classifier. During computation, the value prediction value determines whether each unit is retained. After all word vectors have been processed, the classification result output by the classifier in the last semantic recognition unit marked as "retained" is taken as the semantic recognition result of the sentence. This invention can reduce the computational load while ensuring that semantic features are not lost, significantly improving the performance of semantic recognition.

Description

Technical Field

[0001] This invention belongs to the field of natural language semantic recognition, and specifically relates to a high-performance semantic recognition method. Background Technology

[0002] Natural language semantic recognition (classification) refers to the process of analyzing text or speech content using computer technology to identify its semantic category or intent. This technology has wide applications in fields such as intelligent customer service, search engines, advertising recommendations, and voice assistants. Efficient semantic recognition can significantly improve the human-computer interaction experience, but its performance is highly dependent on the computational load and efficiency of the underlying algorithms.

[0003] Currently, mainstream semantic recognition solutions are mainly based on deep learning models, such as the Transformer architecture or Support Vector Machine (LSVM). Chinese invention patent application CN120218083A discloses a text semantic recognition system and method based on big data. This system collects text data through a data acquisition module and uses a trained semantic recognition model (such as Transformer) to perform semantic analysis on newly input text. However, such models suffer from a large number of parameters and low computational efficiency. Specifically, these methods typically calculate all input features (such as each character or word in a sentence) equally, without distinguishing between key and redundant information. For example, in the sentence "I want to go to a certain snack street and a certain restaurant to eat," the keyword that truly determines the semantic category ("dining") is "go to eat," while modifiers such as "I want" and "a certain snack street and a certain restaurant" have little impact on the classification result. However, existing models still need to fully encode and calculate these non-key features, leading to increased network layers, a surge in computational load, and consequently, reduced overall performance. Summary of the Invention

[0004] This invention proposes a high-performance semantic recognition method, the purpose of which is to improve the performance of semantic recognition.

[0005] The technical solution of this invention is as follows:

[0006] A high-performance semantic recognition method first converts the sentence into a corresponding word vector sequence according to word order, and then inputs the word vectors into a semantic recognition network to obtain the recognition result;

[0007] The semantic recognition network includes several sequentially arranged semantic recognition units; each semantic recognition unit includes hidden layer neurons, a value prediction network, an output neuron, and a classifier; the hidden layer neurons obtain a state vector based on the first and second input vectors of the current semantic recognition unit, the value prediction network obtains a value prediction value based on the state vector, the output neuron obtains a state transition output value based on the state vector, and the classifier obtains a classification result based on the state transition output value.

[0008] When inputting, for the first One semantic recognition unit, :if Then its first input vector is the first word vector of the word vector sequence, and its second input vector is the second word vector of the word vector sequence; if Then its first input vector is the state vector obtained in the previous semantic recognition unit, and the second input vector is the word vector sequence of the first word. Word vectors;

[0009] The first semantic recognition unit is marked as "retained". Starting from the second semantic recognition unit, after each semantic recognition unit obtains its value prediction value, it is compared with the value prediction value of the previous semantic recognition unit. If the value prediction value of the semantic recognition unit is greater than or equal to the value prediction value of the previous semantic recognition unit, the semantic recognition unit is marked as "retained"; otherwise, it is marked as "discarded". After all word vectors are processed, the classification result output by the classifier in the last semantic recognition unit marked as "retained" is taken as the semantic recognition result of the sentence.

[0010] As a further improvement to the high-performance semantic recognition method: the number of neurons in the hidden layer has the same dimension as the first input vector and the second input vector, both being... .

[0011] As a further improvement to the aforementioned high-performance semantic recognition method, hidden layer neurons obtain state vectors. The process is as follows:

[0012] Step A-1: ​​Each neuron in the hidden layer is calculated as follows: For the _____ The intermediate feature vector is obtained from 1 neuron in the following way. The element values :

[0013] ;

[0014] in, It is the hyperbolic tangent function. The first semantic recognition unit in the current semantic recognition unit The weights of each neuron, The first input vector of the current semantic recognition unit is the first... One element, The first semantic recognition unit in the current semantic recognition unit Bias of each neuron;

[0015] Step A-2: Each neuron in the hidden layer is calculated as follows: For the first... The state vector of each neuron is obtained in the following way. The element values :

[0016] ;

[0017] in, The second input vector of the current semantic recognition unit is the first... One element;

[0018] At this point, the complete state vector is obtained. .

[0019] As a further improvement to the aforementioned high-performance semantic recognition method:

[0020] The value prediction network is a feedforward neural network, whose input is a state vector and whose output is a value prediction value.

[0021] The output neuron transforms the state vector through a linear transformation and activation function. Convert to state transition output value ;

[0022] The classifier is The function whose input is the state transition output value Output It is a probability vector, where the value of each element represents the probability that the current state vector belongs to the corresponding category, and the sum of all elements is 1.

[0023] As a further improvement to the aforementioned high-performance semantic recognition method, the semantic recognition network is trained as follows:

[0024] Step T-1: Prepare the sample statement dataset and label each sample statement with a real label;

[0025] Step T-2: Train the hidden layer neurons of each semantic recognition unit based on the random skipping mechanism;

[0026] Step T-3: Train the value prediction network for each semantic recognition unit based on a dual decision-making mechanism of value judgment and randomness.

[0027] As a further improvement to the high-performance semantic recognition method: In step T-2, the following steps are iterated several times, with different sample sentences randomly selected each time:

[0028] Step T-2-1: Convert the sample sentences into corresponding sample word vector sequences according to word order;

[0029] Step T-2-2: Input the sample word vectors in the sample word vector sequence into the semantic recognition network for calculation; before each semantic recognition unit starts calculation, decide whether to skip the unit based on a random skip mechanism;

[0030] Step T-2-3: Take the classification result output by the classifier in the last unskipped semantic recognition unit as the prediction result for this time. Then, calculate the loss based on the prediction result and the real label corresponding to the sample sentence. Finally, update the network parameters of the hidden layer neurons of the unskipped semantic recognition unit based on the loss.

[0031] As a further improvement to the high-performance semantic recognition method, in step T-2-2, the default input rule is: for the first... One semantic recognition unit, :if Then its first input vector is the first sample word vector of the sample word vector sequence, and its second input vector is the second sample word vector of the sample word vector sequence; if Then its first input vector is the state vector obtained in the previous semantic recognition unit, and the second input vector is the first input vector of the sample word vector sequence. Each sample word vector;

[0032] In step T-2-2, after deciding whether to skip, each semantic recognition unit takes corresponding processing measures based on the decision:

[0033] Case a1: If the current semantic recognition unit decides "not to skip" and the current semantic recognition unit is the first semantic recognition unit, then input is performed according to the default input rules, and then the calculation is performed.

[0034] Case a2: If the current semantic recognition unit decides "not to skip" and is the second or subsequent semantic recognition unit, then its first input vector is set to the state vector obtained from the previous semantic recognition unit that decided "not to skip". If all the preceding semantic recognition units are "skipped", then the state vector of the first semantic recognition unit is used as the first input vector, and the second input vector is input according to the default input rules, and then the calculation is performed.

[0035] Case a3: If the current semantic recognition unit decides to "skip" and the current semantic recognition unit is the first semantic recognition unit, then the first input vector is input according to the default input rule, the second input vector is set to the zero vector, and then the calculation is performed.

[0036] Case a4: If the current semantic recognition unit decides to "skip" and the current semantic recognition unit is the second or subsequent semantic recognition unit, then the calculation of that unit is skipped directly.

[0037] As a further improvement to the aforementioned high-performance semantic recognition method, step T-3 consists of the following steps:

[0038] Step T-3-1: Initialize probability ;

[0039] Step T-3-2: Iterate several times according to the following steps, randomly selecting different sample statements each time:

[0040] Step T-3-2-1: Convert the sample sentences into corresponding sample word vector sequences according to word order;

[0041] Step T-3-2-2: Input the sample word vectors in the sample word vector sequence into the semantic recognition network for calculation, and decide whether to retain each semantic recognition unit;

[0042] Step T-3-2-3: Arrange all semantic recognition units marked "reserved" in sequence to form a valid unit sequence, denoted as... , Indicates the first A set of semantic recognition units are marked as "reserved". Then, the sequence of valid units is traversed, the value error of each semantic recognition unit is calculated, and the network parameters of the value prediction network of the corresponding semantic recognition unit are updated based on the value error.

[0043] Step T-3-3: Determine the probability If the value is 0, then end step T-3; otherwise, decrease the probability. and ensure Then proceed to step T-3-2.

[0044] As a further improvement to the high-performance semantic recognition method: In step T-3-2-2, the default input rule is: for the first... One semantic recognition unit, :if Then its first input vector is the first sample word vector of the sample word vector sequence, and its second input vector is the second sample word vector of the sample word vector sequence; if Then its first input vector is the state vector obtained in the previous semantic recognition unit, and the second input vector is the first input vector of the sample word vector sequence. Each sample word vector;

[0045] In step T-3-2-2, each semantic recognition unit first performs a probability-based calculation before starting the computation. The decision of whether the current semantic recognition unit adopts a value function decision or a random decision is determined by the probability of the value function decision. The probability of random decision is Then, based on the decision, take the corresponding action:

[0046] Case b1: If the current semantic recognition unit decides to use the value function decision and the current semantic recognition unit is the first semantic recognition unit, then the input is performed according to the default input rule, the calculation is performed, and the current semantic recognition unit is marked as "reserved".

[0047] Case b2: If the current semantic recognition unit decides to use the value function decision and is the second or subsequent semantic recognition unit, then its first input vector is set to the state vector obtained from the previous semantic recognition unit marked "retained". The second input vector is input according to the default input rules. Then, the calculation is performed, and the value prediction value obtained by the current semantic recognition unit is compared with the value prediction value obtained from the previous semantic recognition unit marked "retained". If the current value prediction value is greater than or equal to the previous value prediction value, then the current semantic recognition unit is marked as "retained"; otherwise, it is marked as "discarded".

[0048] Case b3: If the current semantic recognition unit decides to use random decision-making, the decision result is "do not skip", and the current semantic recognition unit is the first semantic recognition unit, then input is performed according to the default input rules, then calculation is performed, and the current semantic recognition unit is marked as "reserved".

[0049] Case b4: If the current semantic recognition unit decides to use random decision-making, the decision result is "do not skip", and the current semantic recognition unit is the second or subsequent semantic recognition unit, then its first input vector is set to the state vector obtained from the previous semantic recognition unit marked as "reserved", the second input vector is input according to the default input rules, then the calculation is performed, and the current semantic recognition unit is marked as "reserved".

[0050] Case b5: If the current semantic recognition unit decides to use random decision-making, the decision result is "skip", and the current semantic recognition unit is the first semantic recognition unit, then the first input vector is input according to the default input rule, the second input vector is set to the zero vector, then the calculation is performed, and the current semantic recognition unit is marked as "reserved".

[0051] Case b6: If the current semantic recognition unit decides to use random decision-making, the decision result is "skip", and the current semantic recognition unit is the second or subsequent semantic recognition unit, then no calculation is performed, and the current semantic recognition unit is marked as "discard".

[0052] As a further improvement to the high-performance semantic recognition method: In step T-3-2-3, for the first unit in the effective unit sequence... The value error of each semantic recognition unit is calculated as follows:

[0053] ;

[0054] ;

[0055] in, For the first effective unit in the sequence The value prediction value output by the value prediction network in each semantic recognition unit For the first effective unit in the sequence The training value of each semantic recognition unit. For the first effective unit in the sequence The state transition output values ​​of each semantic recognition unit The value prediction value output by the value prediction network in the last semantic recognition unit of the effective unit sequence. As the discount factor, .

[0056] Compared with the prior art, the present invention has the following advantages:

[0057] 1. This invention adopts a chain-like decision-making mechanism based on value prediction comparison, retaining only semantic units with increasing value, realizing progressive feature compression of sentences and dynamic adjustment of computation paths. During the computation process, the compressed deep semantic features are retained through the state vector transmission of hidden layer neurons. Finally, the results are output through the classifier of the last retained unit. Compared with the traditional TransFormer or LSVM architecture, the amount of computation is reduced while ensuring that semantic features are not lost, thereby significantly improving the performance of semantic recognition.

[0058] 2. The training of this invention is divided into two stages: The first stage focuses on the feature extraction capabilities of hidden layer neurons and ensures breadth through a random skipping mechanism, laying the foundation for the second stage training of the value prediction network. In the second stage, a dual decision-making mechanism of value judgment and randomness is introduced. By adjusting the dynamic decay of probabilities, a smooth transition from exploration to utilization is achieved. In the early stage, exploration is emphasized to avoid getting trapped in local optima. Then, the value prediction network gradually takes over the decision-making power, and finally, value decision-making dominates completely, ensuring the stability of convergence.

[0059] 3. In the second stage of training, the training of the value prediction network is optimized through a temporal difference learning mechanism. The training value of each semantic recognition unit is jointly determined by the current classification confidence and the discounted sum of the future semantic value. This design enables the model to dynamically balance immediate classification accuracy and long-distance semantic dependencies, significantly improving training efficiency and enhancing robustness to truncated samples. Attached Figure Description

[0060] Figure 1 This is an architecture diagram of the semantic recognition unit in this invention. Detailed Implementation

[0061] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0062] A high-performance semantic recognition method comprises the following steps: first, converting the sentence into a corresponding word vector sequence according to word order; then, inputting the word vectors into a semantic recognition network to obtain the recognition result.

[0063] The semantic recognition network comprises several semantic recognition units arranged in sequence.

[0064] like Figure 1 The semantic recognition unit includes hidden neurons, a value prediction network, an output neuron, and a classifier. The hidden neurons obtain a state vector based on the first and second input vectors of the current semantic recognition unit. The value prediction network obtains a value prediction value based on the state vector. The output neuron obtains a state transition output value based on the state vector. The classifier obtains a classification result based on the state transition output value.

[0065] When inputting, for the first One semantic recognition unit, :if Then its first input vector is the first word vector of the word vector sequence, and its second input vector is the second word vector of the word vector sequence; if Then its first input vector is the state vector obtained in the previous semantic recognition unit, and the second input vector is the word vector sequence of the first word. Each word vector.

[0066] The first semantic recognition unit is marked as "retained". Starting from the second semantic recognition unit, each semantic recognition unit, after obtaining its value prediction value, compares it with the value prediction value of the previous semantic recognition unit. If the value prediction value of the current semantic recognition unit is greater than or equal to the value prediction value of the previous semantic recognition unit, the semantic recognition unit is marked as "retained"; otherwise, it is marked as "discarded". After all word vectors have been processed, the classification result output by the classifier in the last semantic recognition unit marked as "retained" is taken as the semantic recognition result of the sentence.

[0067] The specific structure of the semantic recognition unit is as follows:

[0068] 1. Hidden layer neurons

[0069] The number of neurons in the hidden layer has the same dimension as the first and second input vectors. Hidden layer neurons obtain state vectors The process is as follows:

[0070] Step A-1: ​​Each neuron in the hidden layer is calculated as follows: For the _____ The intermediate feature vector is obtained from 1 neuron in the following way. The element values :

[0071] ;

[0072] in, It is the hyperbolic tangent function. The first semantic recognition unit in the current semantic recognition unit The weights of each neuron, The first input vector of the current semantic recognition unit is the first... One element, The first semantic recognition unit in the current semantic recognition unit Bias of each neuron.

[0073] Step A-2: Each neuron in the hidden layer is calculated as follows: For the first... The state vector of each neuron is obtained in the following way. The element values :

[0074] ;

[0075] in, The second input vector of the current semantic recognition unit is the first... Each element.

[0076] At this point, the complete state vector is obtained. .

[0077] 2. Value Prediction Network

[0078] In this embodiment, the value prediction network is a feedforward neural network (FNN), whose input is a state vector and whose output is a value prediction value.

[0079] 3. Output neurons

[0080] The output neuron transforms the state vector through a linear transformation and activation function. Convert to state transition output value .

[0081] 4. Classifier

[0082] The classifier is The function whose input is the state transition output value Output It is a probability vector, where the value of each element represents the probability that the current state vector belongs to the corresponding category, and the sum of all elements is 1.

[0083] Furthermore, the training method for the semantic recognition network is as follows:

[0084] Step T-1: Prepare a sample statement dataset and label each sample statement with a real label.

[0085] Preferably, a continuous segment of a fixed length L can be randomly extracted from the original statement as a sample statement. For example, the original statement "We went to Renmin Road for lunch" can be randomly extracted into sample statements such as "went to Renmin" and "had lunch on Renmin Road" with a fixed length L=5.

[0086] Step T-2: Train the hidden neurons of each semantic recognition unit based on the random skipping mechanism.

[0087] During training, iterate several times (e.g., 1000 times) following these steps, randomly selecting different sample statements each time:

[0088] Step T-2-1: Convert the sample sentences into corresponding sample word vector sequences according to word order;

[0089] Step T-2-2: Input the sample word vectors from the sample word vector sequence into the semantic recognition network for calculation.

[0090] The default input rule is: for the first... One semantic recognition unit, :if Then its first input vector is the first sample word vector of the sample word vector sequence, and its second input vector is the second sample word vector of the sample word vector sequence; if Then its first input vector is the state vector obtained in the previous semantic recognition unit, and the second input vector is the first input vector of the sample word vector sequence. Each sample word vector.

[0091] Before each semantic recognition unit begins computation, a random skipping mechanism is used to determine whether the unit should be skipped, with a 50% probability of skipping.

[0092] Case a1: If the current semantic recognition unit decides "not to skip" and the current semantic recognition unit is the first semantic recognition unit, then input is performed according to the default input rules, and then the calculation is performed.

[0093] Case a2: If the current semantic recognition unit decides "not to skip" and is the second or subsequent semantic recognition unit, then its first input vector is set to the state vector obtained from the previous semantic recognition unit that decided "not to skip". If all the preceding semantic recognition units are "skipped", then the state vector of the first semantic recognition unit is used as the first input vector. The second input vector is input according to the default input rules, and then the calculation is performed.

[0094] Case a3: If the current semantic recognition unit decides to "skip" and the current semantic recognition unit is the first semantic recognition unit, then the first input vector is input according to the default input rule, the second input vector is set to the zero vector, and then the calculation is performed.

[0095] Case a4: If the current semantic recognition unit decides to "skip" and the current semantic recognition unit is the second or subsequent semantic recognition unit, then the calculation of that unit is skipped directly.

[0096] Step T-2-3: Take the classification result output by the classifier in the last unskipped semantic recognition unit as the prediction result for this time. Then, calculate the loss based on the prediction result and the real label corresponding to the sample sentence. Finally, update the network parameters of the hidden layer neurons of the unskipped semantic recognition unit based on the loss.

[0097] The loss can be cross-entropy loss, and the update method can be gradient descent.

[0098] Step T-3: Train the value prediction network for each semantic recognition unit based on a dual decision-making mechanism of value judgment and randomness.

[0099] The value judgment and random dual decision-making mechanism includes value function decision and random decision. Both types of decision are used to determine whether each semantic recognition unit participates in the calculation during training. As training progresses, the proportion of random decision decreases and the proportion of value function decision increases.

[0100] The specific steps are as follows:

[0101] Step T-3-1: Initialize probability In this embodiment, probability The initial value is 98%.

[0102] Step T-3-2: Iterate several times (e.g., 1000 times) following these steps, randomly selecting different sample statements each time:

[0103] Step T-3-2-1: Convert the sample sentences into corresponding sample word vector sequences according to word order;

[0104] Step T-3-2-2: Input the sample word vectors from the sample word vector sequence into the semantic recognition network for calculation.

[0105] The default input rule is: for the first... One semantic recognition unit, :if Then its first input vector is the first sample word vector of the sample word vector sequence, and its second input vector is the second sample word vector of the sample word vector sequence; if Then its first input vector is the state vector obtained in the previous semantic recognition unit, and the second input vector is the first input vector of the sample word vector sequence. Each sample word vector.

[0106] Before each semantic recognition unit begins computation, it is first based on probability. The decision of whether the current semantic recognition unit adopts a value function decision or a random decision is determined by the probability of the value function decision. The probability of random decision is :

[0107] Case b1: If the current semantic recognition unit decides to use the value function decision and the current semantic recognition unit is the first semantic recognition unit, then the input is performed according to the default input rules, the calculation is performed, and the current semantic recognition unit is marked as "reserved".

[0108] Case b2: If the current semantic recognition unit decides to use the value function decision and is the second or subsequent semantic recognition unit, then its first input vector is set to the state vector obtained from the previous semantic recognition unit marked "retained". The second input vector is input according to the default input rules. Then, the calculation is performed, and the value prediction value obtained by the current semantic recognition unit is compared with the value prediction value obtained from the previous semantic recognition unit marked "retained". If the current value prediction value is greater than or equal to the previous value prediction value, then the current semantic recognition unit is marked "retained"; otherwise, it is marked "discarded".

[0109] Case b3: If the current semantic recognition unit decides to use random decision-making, the decision result is "do not skip", and the current semantic recognition unit is the first semantic recognition unit, then the input is performed according to the default input rules, the calculation is executed, and the current semantic recognition unit is marked as "reserved".

[0110] Case b4: If the current semantic recognition unit decides to use random decision-making, the decision result is "do not skip", and the current semantic recognition unit is the second or subsequent semantic recognition unit, then its first input vector is set to the state vector obtained from the previous semantic recognition unit marked as "reserved", the second input vector is input according to the default input rules, then the calculation is performed, and the current semantic recognition unit is marked as "reserved".

[0111] Case b5: If the current semantic recognition unit decides to use random decision-making, the decision result is "skip", and the current semantic recognition unit is the first semantic recognition unit, then the first input vector is input according to the default input rule, the second input vector is set to the zero vector, then the calculation is performed, and the current semantic recognition unit is marked as "reserved".

[0112] Case b6: If the current semantic recognition unit decides to use random decision-making, the decision result is "skip", and the current semantic recognition unit is the second or subsequent semantic recognition unit, then no calculation is performed, and the current semantic recognition unit is marked as "discard".

[0113] When using random decision-making, the probability of "skipping" and "not skipping" is both 50%.

[0114] Step T-3-2-3: Arrange all semantic recognition units marked "reserved" in sequence to form a valid unit sequence, denoted as... , Indicates the first A set of semantic recognition units are marked as "reserved". Then, the sequence of valid units is traversed, the value error of each semantic recognition unit is calculated, and the network parameters of the value prediction network for the corresponding semantic recognition unit are updated based on the value error.

[0115] Specifically, for the first unit in the effective unit sequence The value error of each semantic recognition unit is calculated as follows:

[0116] ;

[0117] ;

[0118] in, For the first effective unit in the sequence The value prediction value output by the value prediction network in each semantic recognition unit For the first effective unit in the sequence The training value of each semantic recognition unit. For the first effective unit in the sequence The state transition output values ​​of each semantic recognition unit The value prediction value output by the value prediction network in the last semantic recognition unit of the effective unit sequence. As the discount factor, .

[0119] Step T-3-3: Determine the probability If the value is 0, then end step T-3; otherwise, decrease the probability. and ensure Then proceed to step T-3-2.

[0120] During network computation, it can be decided whether to complete all computations of the value prediction network, output neurons, and classifier as needed. For example, during normal inference, only the last remaining semantic recognition unit needs to have its output neuron and classifier computed.

[0121] It should be noted that, as will be apparent to those skilled in the art, the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics thereof. The scope of the present invention is defined by the claims rather than the foregoing description.

Claims

1. A high-performance semantic recognition method, which first converts a sentence into a corresponding word vector sequence according to word order, and then inputs the word vectors into a semantic recognition network to obtain the recognition result, characterized in that: The semantic recognition network includes several sequentially arranged semantic recognition units; each semantic recognition unit includes hidden layer neurons, a value prediction network, an output neuron, and a classifier; the hidden layer neurons obtain a state vector based on the first and second input vectors of the current semantic recognition unit, the value prediction network obtains a value prediction value based on the state vector, the output neuron obtains a state transition output value based on the state vector, and the classifier obtains a classification result based on the state transition output value. When inputting, for the first One semantic recognition unit, :if Then its first input vector is the first word vector of the word vector sequence, and its second input vector is the second word vector of the word vector sequence; if Then its first input vector is the state vector obtained in the previous semantic recognition unit, and the second input vector is the word vector sequence of the first word. Word vectors; The first semantic recognition unit is marked as "retained". Starting from the second semantic recognition unit, each semantic recognition unit obtains its value prediction value and compares it with the value prediction value of the previous semantic recognition unit. If the value prediction value of the semantic recognition unit is greater than or equal to the value prediction value of the previous semantic recognition unit, the semantic recognition unit is marked as "retained"; otherwise, it is marked as "discarded". After all word vectors are processed, the classification result output by the classifier in the last semantic recognition unit marked as "retained" is taken as the semantic recognition result of the sentence. The training method for the semantic recognition network is as follows: Step T-1: Prepare the sample statement dataset and label each sample statement with a real label; Step T-2: Train the hidden layer neurons of each semantic recognition unit based on the random skipping mechanism; Step T-3: Train the value prediction network for each semantic recognition unit based on a dual decision-making mechanism of value judgment and randomness; The specific steps of step T-3 are as follows: Step T-3-1: Initialize probability ; Step T-3-2: Iterate several times according to the following steps, randomly selecting different sample statements each time: Step T-3-2-1: Convert the sample sentences into corresponding sample word vector sequences according to word order; Step T-3-2-2: Input the sample word vectors in the sample word vector sequence into the semantic recognition network for calculation, and decide whether to retain each semantic recognition unit; In step T-3-2-2, the default input rule is: for the first... One semantic recognition unit, :if Then its first input vector is the first sample word vector of the sample word vector sequence, and its second input vector is the second sample word vector of the sample word vector sequence; if Then its first input vector is the state vector obtained in the previous semantic recognition unit, and the second input vector is the first input vector of the sample word vector sequence. Each sample word vector; In step T-3-2-2, each semantic recognition unit first performs a probability-based calculation before starting the computation. The decision of whether the current semantic recognition unit adopts a value function decision or a random decision is determined by the probability of the value function decision. The probability of random decision is Then, based on the decision, take the corresponding action: Case b1: If the current semantic recognition unit decides to use the value function decision and the current semantic recognition unit is the first semantic recognition unit, then the input is performed according to the default input rules, the calculation is performed, and the current semantic recognition unit is marked as "reserved". Case b2: If the current semantic recognition unit decides to use the value function decision and is the second or subsequent semantic recognition unit, then its first input vector is set to the state vector obtained from the previous semantic recognition unit marked "retain". The second input vector is input according to the default input rule. Then, the calculation is performed, and the value prediction value obtained by the current semantic recognition unit is compared with the value prediction value obtained from the previous semantic recognition unit marked "retain". If the current value prediction value is greater than or equal to the previous value prediction value, then the current semantic recognition unit is marked as "retain"; otherwise, it is marked as "discard". Case b3: If the current semantic recognition unit decides to use random decision-making, the decision result is "do not skip", and the current semantic recognition unit is the first semantic recognition unit, then input is performed according to the default input rules, then calculation is performed, and the current semantic recognition unit is marked as "reserved". Case b4: If the current semantic recognition unit decides to use random decision-making, the decision result is "do not skip", and the current semantic recognition unit is the second or subsequent semantic recognition unit, then its first input vector is set to the state vector obtained from the previous semantic recognition unit marked as "reserved", the second input vector is input according to the default input rules, then the calculation is performed, and the current semantic recognition unit is marked as "reserved". Case b5: If the current semantic recognition unit decides to use random decision-making, the decision result is "skip", and the current semantic recognition unit is the first semantic recognition unit, then the first input vector is input according to the default input rule, the second input vector is set to the zero vector, then the calculation is performed, and the current semantic recognition unit is marked as "reserved". Case b6: If the current semantic recognition unit decides to use random decision-making, the decision result is "skip," and the current semantic recognition unit is the second or subsequent semantic recognition unit, then no calculation is performed, and the current semantic recognition unit is marked as "discard." Step T-3-2-3: Arrange all semantic recognition units marked "reserved" in sequence to form a valid unit sequence, denoted as... , Indicates the first A set of semantic recognition units are marked as "reserved". Then, the sequence of valid units is traversed, the value error of each semantic recognition unit is calculated, and the network parameters of the value prediction network of the corresponding semantic recognition unit are updated based on the value error. In step T-3-2-3, for the first effective unit in the sequence... The value error of each semantic recognition unit is calculated as follows: ； ； in, For the first effective unit in the sequence The value prediction value output by the value prediction network in each semantic recognition unit For the first effective unit in the sequence The training value of each semantic recognition unit. For the first effective unit in the sequence The state transition output values ​​of each semantic recognition unit The value prediction value output by the value prediction network in the last semantic recognition unit of the effective unit sequence. As the discount factor, ; Step T-3-3: Determine the probability If the value is 0, then end step T-3; otherwise, decrease the probability. and ensure Then proceed to step T-3-2.

2. The high-performance semantic recognition method as described in claim 1, characterized in that: The number of neurons in the hidden layer has the same dimension as the first and second input vectors. .

3. The high-performance semantic recognition method as described in claim 2, characterized in that, Hidden layer neurons obtain state vectors The process is as follows: Step A-1: ​​Each neuron in the hidden layer is calculated as follows: For the first... The intermediate feature vector is obtained from 1 neuron in the following way. The element values : ； in, It is the hyperbolic tangent function. The first semantic recognition unit in the current semantic recognition unit The weights of each neuron, The first input vector of the current semantic recognition unit is the first... One element, The first semantic recognition unit in the current semantic recognition unit Bias of each neuron; Step A-2: Each neuron in the hidden layer is calculated as follows: For the first... The state vector of each neuron is obtained in the following way. The element values : ； in, The second input vector of the current semantic recognition unit is the first... One element; At this point, the complete state vector is obtained. .

4. The high-performance semantic recognition method as described in claim 1, characterized in that: The value prediction network is a feedforward neural network, whose input is a state vector and whose output is a value prediction value. The output neuron transforms the state vector through a linear transformation and activation function. Convert to state transition output value ; The classifier is The function whose input is the state transition output value Output It is a probability vector, where the value of each element represents the probability that the current state vector belongs to the corresponding category, and the sum of all elements is 1.

5. The high-performance semantic recognition method as described in claim 1, characterized in that: In step T-2, the following steps are repeated several times, with a different sample statement randomly selected each time: Step T-2-1: Convert the sample sentences into corresponding sample word vector sequences according to word order; Step T-2-2: Input the sample word vectors in the sample word vector sequence into the semantic recognition network for calculation; before each semantic recognition unit starts calculation, decide whether to skip the unit based on a random skip mechanism; Step T-2-3: Take the classification result output by the classifier in the last unskipped semantic recognition unit as the prediction result for this time. Then, calculate the loss based on the prediction result and the real label corresponding to the sample sentence. Finally, update the network parameters of the hidden layer neurons of the unskipped semantic recognition unit based on the loss.

6. The high-performance semantic recognition method as described in claim 5, characterized in that, In step T-2-2, the default input rule is: for the first... One semantic recognition unit, :if Then its first input vector is the first sample word vector of the sample word vector sequence, and its second input vector is the second sample word vector of the sample word vector sequence; if Then its first input vector is the state vector obtained in the previous semantic recognition unit, and the second input vector is the first input vector of the sample word vector sequence. Each sample word vector; In step T-2-2, after deciding whether to skip, each semantic recognition unit takes corresponding processing measures based on the decision: Case a1: If the current semantic recognition unit decides "not to skip" and the current semantic recognition unit is the first semantic recognition unit, then input is performed according to the default input rules, and then the calculation is performed. Case a2: If the current semantic recognition unit decides "not to skip" and is the second or subsequent semantic recognition unit, then its first input vector is set to the state vector obtained from the previous semantic recognition unit that decided "not to skip". If all the preceding semantic recognition units are "skipped", then the state vector of the first semantic recognition unit is used as the first input vector, and the second input vector is input according to the default input rules, and then the calculation is performed. Case a3: If the current semantic recognition unit decides to "skip" and the current semantic recognition unit is the first semantic recognition unit, then the first input vector is input according to the default input rule, the second input vector is set to the zero vector, and then the calculation is performed. Case a4: If the current semantic recognition unit decides to "skip" and the current semantic recognition unit is the second or subsequent semantic recognition unit, then the calculation of that unit is skipped directly.

Images