Vectorization representation model method for realizing high-accuracy semantic matching and realization method

Abstract

The technical scheme of the invention provides a vectorization representation model method for realizing high-accuracy semantic matching. Another technical scheme of the invention is to provide an implementation method of a vectorization representation model for realizing high-accuracy semantic matching. The invention provides a Chinese vectorization model with double innovation of a structure and a training method, the scheme disclosed by the invention can solve the problems of low accuracy and low recall rate when a general vectorization model is applied in a semantic matching task, the expression effect of the vectorization model is optimized, the problem of sentence expression collapse is solved, and the method is suitable for the semantic matching task. And the accuracy and adaptability of the model are improved. According to the method, vectorization is carried out from the font level and the character meaning level, and the performance of the model on Chinese semantic understanding is improved. According to the dynamic training method for reinforcement learning provided by the invention, a training framework for reinforcement learning is adopted, and the model can be self-optimized through a reward mechanism, so that the expressive force and robustness of the model in practical application are enhanced.

Description

Technical Field

[0001] The present invention relates to a vectorized representation model algorithm for realizing high-accuracy semantic matching and an implementation method thereof. Background Art

[0002] In machine learning and natural language processing, vectorization is an essential and important process that maps high-dimensional data to low-dimensional space. A feature vector is usually a vector composed of real numbers that represents the input data as a point in a continuous numerical space. In natural language processing, the significance of feature vectors lies in that they can represent the semantics of words or sentences. Since the development of deep learning, vectorization technology has been widely used in research. Common algorithms include word2vec, GloVe, and FastText. These algorithms can learn the semantic representation of words or sentences by training neural networks or using only matrix decomposition techniques. These algorithms are widely used in various natural language processing tasks, such as text classification, machine translation, sentiment analysis, etc. In semantic matching tasks, vectorization models are crucial. The general approach is to vectorize the text to be matched, calculate the cosine similarity or dot product between vectors as an indicator of whether two vectors are similar in the semantic space, and determine the degree of relevance between texts.

[0003] Since the pre-trained models represented by the Bert model have continuously refreshed the evaluation list in various tasks, the existing vectorization technology solutions are mainly general vectorization models represented by sentence-transformer. The vectorization model proposed by Zhiyuan adopts the Bert model structure and is pre-trained on a large-scale Chinese general corpus. The model has achieved good performance in Chinese evaluation tasks.

[0004] Although the general vectorization model is competent for semantic similarity matching tasks in most scenarios, it still faces many challenges when applied to specific tasks, especially when the length of the text to be matched is significantly different from the text, a better representation of the text's semantics is required to obtain more accurate text similarity. The general vectorization model has a bias towards matching short texts. After the text is vectorized, when calculating the text similarity through cosine similarity or dot product, the score obtained by the short text is usually higher than that of the long text, even if the long text is more relevant semantically. This shows that the general vectorization model is more sensitive to the length of the text after the data distribution changes. Another point is the problem of representation collapse. The general vectorization model tends to encode all sentences into a smaller spatial area, so that most sentence pairs have high similarity scores, even for semantically irrelevant sentences. This is called the collapse phenomenon of sentence representation. These problems will reduce the matching accuracy and recall rate in the scenario of semantic matching, affecting the matching effect.

[0005] In addition, most existing Chinese vectorization models use general natural language processing algorithms, which often ignore the particularity of the Chinese language. For example, traditional word vector models such as Word2Vec or GloVe mainly rely on word co-occurrence information, but do not fully utilize the unique structural features and rich stroke information of Chinese characters. In addition, these Chinese vectorization models use static loss functions during training and lack the ability to dynamically adapt to changes in word meanings in different contexts. These limitations lead to the following disadvantages:

[0006] (1) Ignoring the internal structure of Chinese characters: Existing models fail to fully utilize the internal structure information of Chinese characters, such as radicals and stroke order, which are crucial for understanding the meaning of Chinese characters;

[0007] (2) Lack of dynamic adaptability: The model is usually unable to dynamically adjust word vectors according to different contexts, resulting in limited expressiveness of vector representations in different contexts;

[0008] (3) Limitations of training methods: Traditional training methods rely on predefined objective functions and lack flexibility and adaptability when faced with complex language phenomena. Summary of the invention

[0009] The technical problem to be solved by the present invention is that the accuracy and recall rate of a general vectorization model are low when applied to a semantic matching task.

[0010] In order to solve the above technical problems, the technical solution of the present invention is to provide a vectorized representation model method for realizing high-accuracy semantic matching, characterized in that a Chinese vectorized model is constructed, and the Chinese vectorized model adopts a pseudo twin network structure including an image recognition module, a semantic feature extraction module and a feature fusion module, wherein:

[0011] The image recognition module extracts features from the text images of all Chinese characters in the text based on the structure and strokes of Chinese characters to obtain the visual feature vector of the current text;

[0012] The semantic feature extraction module extracts the semantic feature vector of the same text;

[0013] The feature fusion module uses an attention mechanism and a multi-layer perceptron to fuse the visual feature vector from the image recognition module with the semantic feature vector from the semantic feature extraction module, and outputs a fused Embedding vector of the current text.

[0014] Preferably, the image recognition module uses a vision transformer model.

[0015] Preferably, the semantic feature module uses a pre-trained sentence-transformer model.

[0016] Another technical solution of the present invention is to provide a method for implementing a vectorized representation model for high-accuracy semantic matching, characterized in that an Actor-Critic method is used to dynamically optimize the Embedding vector output by the Chinese vectorized model constructed by the method, and the Chinese vectorized model is an Actor network. The implementation method includes the following steps:

[0017] Step 1: Prepare the dataset:

[0018] After obtaining the original text, data enhancement is performed on the original text to obtain the data-enhanced text. The original text and the corresponding data-enhanced text are used as two samples to form a pair of positive samples in the data set, and the original text and other text are used as two samples to form a pair of negative samples in the data set.

[0019] The similarity between the two samples is used as the label of the positive sample pair and the negative sample pair, where the similarity between the two samples in the negative sample pair is the lowest;

[0020] Step 2: Initialize the Actor network and the Critic network, and define the network gradient optimizer Adam;

[0021] Step 3: Select multiple pairs of positive sample pairs and negative sample pairs from the data set constructed in step 1, input the samples in the positive sample pairs and negative sample pairs into the current Actor network, generate an Embedding vector for each sample through the current Actor network, and obtain the current state of the current Actor network;

[0022] Step 4: Calculate the cosine similarity of the Embedding vectors of all samples, obtain the square loss based on the cosine similarity, and use the network gradient optimizer Adam to perform gradient backpropagation on the current Actor network in combination with the square loss, change the Embedding vector output by the current Actor network, and obtain the next state of the current Actor network.

[0023] Step 5. Calculate rewards:

[0024] Based on the labels of the sample pairs, we obtain: the number m1 of sample pairs that are both positive and consistent before the corresponding Embedding vector is changed, and the number m2 of sample pairs that are both positive and consistent after the corresponding Embedding vector is changed, the number k1 of sample pairs that are not positive and consistent before the corresponding Embedding vector is changed, and the number k2 of sample pairs that are not positive and consistent after the corresponding Embedding vector is changed, where the consistency of the two sample pairs means that the cosine similarity ranking of all samples contained in the two sample pairs is consistent with the similarity ranking of the labels;

[0025] Calculate the reward value reward:

[0026] reward=reward next +(reward next -reward1)

[0027] Among them, reward1=α*m1+β*k1, reward next =α*m2+β*k2, α and β are weight coefficients, α>β;

[0028] Step 6. The Critic network evaluates the value of the current state V(s) and the value of the next state V(s_next):

[0029] embedding now =concat([embedding text_a ,embedding text_b ])

[0030] x = ReLU(Linear(embedding now ))

[0031] V(s)=Linear(x)

[0032] embedding next =concat([embedding ′ text_a ,embedding ′ text_b ])

[0033] x ′ =ReLU(Linear(embedding next ))

[0034] V(s_next)=Linear(x ′ )

[0035] In the formula, embedding ′ text_a The Embedding vector obtained by the changed current Actor network with text text_a as input, embedding ′ taxt_b The Embedding vector obtained by the changed current Actor network with text text_b as input, embedding text_a The embedding vector obtained by the current Actor network before the change with the text text_a as input, embedding text_b The Embedding vector obtained by the current Actor network before the change with the text text_b as input;

[0036] Step 7. Update the Actor network and Critic network:

[0037] According to the value of the current state V(s) and the value of the next state V(s_next), the Critic network is updated using the temporal difference error td_error and the network gradient optimizer Adam, where td_error = reward + γ * V(s_next) - V(s), and the loss used by the network gradient optimizer Adam is critic_loss = td_error 2 ;

[0038] The Actor network is updated according to the temporal difference error td_error and the policy gradient optimization method, where the policy gradient optimization method is shown as follows:

[0039]

[0040] log θ (a|s)=log(cosine_simi(embedding ′ text_a ,embedding ′ text_b ))

[0041] Where: θ is the network parameter of the Actor network; is the gradient; log θ (a|s) refers to the logarithmic probability of selecting dynamic a under the network parameters θ and the current state s;

[0042] The policy gradient method is used to calculate the parameter gradients that need to be updated in the Actor network, and the Actor network is updated.

[0043] Preferably, in step 1, the similarity between the two samples in the positive sample pair is divided into weak similarity, similarity, and very similarity based on the similarity score, wherein the similarity score between the text text_a and the text text_b is expressed as similarity(text_a, text_b), the text text_a is the original text, and the text text_b is the text after data enhancement, then:

[0044] similarity(text_a,text_b)

[0045] =w1*EditDistance(text_a,text_b)+w2

[0046] *Concurrence(text_a,text_b)

[0047] Where: EditDistance(text_a,text_b) is the edit distance between text_a and text_b. len(text_a) is the length of text text_a, len(text_b) is the length of text text_b, opn is the number of data augmentation operations required to transform text text_a into text text_b; Concurrence(text_a, text_b) is the character overlap between text text_a and text text_b, nc is the number of characters that co-occur in text_a and text_b; w1 and w2 are adjustable parameters.

[0048] Preferably, for the text text_a and the text text_b: if similarity(text_a, text_b) is between [0.85, 1], the label of the current positive sample pair is "very similar"; if similarity(text_a, text_b) is between [0.6, 0.85), the label of the current positive sample pair is "similar"; if similarity(text_a, text_b) is between [0.3, 0.6), the label of the current positive sample pair is "weakly similar".

[0049] Preferably, in step 4, the square loss loss is calculated by the following formula:

[0050]

[0051] Where: N is the total number of sample pairs;

[0052] embedding text_aThe embedding vector obtained by the current Actor network with text text_a as input, embedding text_b is the embedding vector obtained by the current Actor network with text text_b as input, cosine_simi(embedding text_a ,embedding text_b ) is the cosine similarity between the Embedding vector of text_a and the Embedding vector of text_b;

[0053]

[0054] The present invention provides a Chinese vectorization model with dual innovations in structure and training method. The solution disclosed in the present invention can solve the problems of low accuracy and recall rate when applying general vectorization models in semantic matching tasks, optimize the performance of the vectorization model, solve the problem of sentence representation collapse, and improve the accuracy and adaptability of the model.

[0055] On the one hand, the present invention combines the structural features of Chinese characters, and by designing a feature extraction module specifically for the structure and strokes of Chinese characters, vectorization is performed from the two levels of character shape and meaning, so as to better capture the semantic information and visual features of Chinese characters, thereby improving the performance of the model in Chinese semantic understanding. On the other hand, the present invention provides a dynamic training method of reinforcement learning, which adopts the training framework of reinforcement learning, so that the constructed model can dynamically adjust the word vector according to different contexts, provide more accurate semantic representation, and through the reward mechanism, the model can self-optimize to adapt to various language usage scenarios, and enhance its expressiveness and robustness in practical applications.

[0056] Compared with the prior art solutions, the present invention has the following beneficial effects:

[0057] 1. Comprehensive feature extraction: The existing technology usually only uses the visual features or semantic features of Chinese characters for vectorization, while the present invention combines Vision Transformer and Sentence-Transformer to comprehensively extract the visual and semantic features of Chinese characters, thereby generating a more comprehensive and accurate feature vector. This multimodal feature fusion strategy significantly improves the performance of the vectorization model.

[0058] 2. Dynamic adjustment capability: The Actor-Critic method in reinforcement learning is used to enable the model to dynamically adjust word vectors according to different contexts, enhancing the flexibility and adaptability of the model when processing complex text contexts. Existing technologies often lack this dynamic adjustment capability, resulting in limited performance when processing diverse texts.

[0059] 3. Reinforcement learning optimization strategy: Dynamic training is performed through the Actor-Critic method, combined with policy gradient and value function estimation, to improve the stability and efficiency of training. Compared with the traditional static optimization method, the optimization strategy of the present invention can better adapt to the ever-changing text matching tasks and enhance the robustness of the model.

[0060] 4. Refined reward mechanism: In the text matching task, the present invention designs a refined reward mechanism, especially a weighted penalty strategy for positive sample similarity judgment. This reward mechanism can more accurately evaluate and optimize the performance of the model, ensuring that the generated word vector is more accurate in the similarity calculation task.

[0061] 5. Efficient feature fusion: The feature vectors from different sources are effectively fused through the attention mechanism and multi-layer perceptron, and the generated comprehensive feature vector has higher expressive power. Compared with simple feature splicing or weighted average methods, the feature fusion strategy of the present invention can better capture the complex relationships in the text.

[0062] 6. Enhanced text processing capabilities: When processing Chinese text, the present invention can deeply explore the intrinsic structure and semantics of Chinese characters, significantly improving the accuracy and adaptability of text vectorization. Existing technologies often perform poorly when dealing with complex Chinese character structures and ambiguity, but the present invention overcomes these challenges through comprehensive feature extraction and dynamic adjustment. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 The structure of the Chinese vectorization model is shown;

[0064] Figure 2 The reinforcement learning implementation process is illustrated. DETAILED DESCRIPTION

[0065] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the appended claims of the application equally.

[0066] One aspect of an embodiment of the present invention proposes a Chinese vectorization model that combines Chinese character structure and stroke feature extraction and reinforcement learning training methods. Through a specially designed Chinese character feature extraction module, the Chinese vectorization model disclosed in the embodiment of the present invention can deeply explore the intrinsic structure and semantics of Chinese characters, and at the same time adopt a dynamic training strategy of reinforcement learning, so that the Chinese vectorization model can dynamically adjust the word vector according to different contexts, thereby significantly improving the accuracy and adaptability of Chinese text vectorization.

[0067] The model structure of the present invention adopts a pseudo twin network structure, such as Figure 1 As shown in the figure, it consists of an image recognition module, a semantic feature extraction module and a feature fusion module. The image recognition module uses the vision transformer to extract features from Chinese character images to extract the basic visual features of Chinese characters, such as the shape, position, and direction of the strokes, and outputs the visual feature vector of the current text. The semantic feature module uses the pre-trained sentence-transformer model to extract the semantic features of Chinese characters in the same text and outputs the semantic feature vector of the text. The feature fusion module uses the attention mechanism and multi-layer perceptron to fuse feature vectors from different sources and outputs the fused feature vector (i.e., the Embedding vector).

[0068] Another aspect of the implementation of the present invention is to use the Actor-Critic method to optimize the above Chinese vectorization model on the text matching task. The Actor-Critic method combines the advantages of the policy gradient and value function methods, improves the stability and efficiency of training, and is suitable for training tasks that require high efficiency and stability. The Actor-Critic method is a reinforcement learning algorithm that combines policy gradient and value function estimation, aiming to balance the flexibility of policy optimization and the stability of value estimation. Specifically, the Actor-Critic method includes two main components:

[0069] Actor (strategy network): Generates a strategy, that is, selects an action based on the current state. In the embodiment of the present invention, the above Chinese vectorization model is optimized.

[0070] Critic (value network): Evaluates the quality of the current strategy, that is, estimates the value of the state.

[0071] How it works

[0072] Actor: Generates an action (a) based on the current state (s), with the goal of maximizing the future cumulative reward.

[0073] Critic: Evaluates the value (V(s)) of the current state(s) and action(a), and gives feedback to help the Actor adjust its strategy.

[0074] The specific implementation process of the Actor-Critic method disclosed in the embodiment of the present invention is as follows:

[0075] Step 1, prepare the data set: The present invention aims to obtain better accuracy and recall in text matching tasks by optimizing the Chinese vectorization model. The basic corpus is the manually labeled text data of posts in the stock bar community. The text data labeled as advertisements are subjected to data enhancement operations such as randomly splicing meaningless words, deleting non-advertising text, modifying some meaningless words, and modifying the word order. The text pairs consisting of the original text and the data-enhanced text are taken as the positive sample set, and the label is similar. At the same time, the edit distance between the data-enhanced text and the original text and the function of the character overlap (as shown in the following formula) are used as the identification of the degree of similarity, which is divided into 3 levels: weak similarity, similarity, and very similarity.

[0076] similarity(text_a,text_b)

[0077] =w1*EditDistance(text_a,text_b)+w2

[0078] *Concurrence(text_a,text_b)

[0079] Where: EditDistance(text_a,text_b) is the edit distance between text_a and text_b. len(text_a) is the length of text_a, len(text_b) is the length of text_b, opn is the number of operations required to convert text_a into text_b through addition, modification, and deletion operations; Concurrence(text_a, text_b) is the character overlap between text_a and text_b, nc is the number of characters that co-occur in text_a and text_b; w1 and w2 are adjustable parameters, which are set to 0.5 in the embodiment of the present invention.

[0080] According to the above formula, the similarity score similarity(text_a, text_b) between the text text_a and the text text_b is calculated to be in the range of [0,1]. In the embodiment of the present invention, on the enhanced data set, after statistics, the average value of the similarity scores of all original texts and their enhanced texts is 0.846, and this average value is taken as the threshold for dividing "very similar" and "similar". Therefore, the sample pairs with similarities between [0.85,1] are labeled as "very similar"; the sample pairs with similarities between [0.6,0.85) are labeled as "similar"; the sample pairs with similarities between [0.3,0.6) are labeled as "weakly similar". The text pairs consisting of the original text and other texts are taken as negative sample sets and labeled as dissimilar.

[0081] Step 2: Initialize the Actor network (the Chinese vectorization model constructed above) and the Critic network (used to evaluate state value, consisting of a two-layer perceptron network, outputting a single floating point number as value).

[0082] Step 3: Define the network gradient optimizer Adam.

[0083] Step 4: Initialize the reward value to 0. Initialize the loss value to 0. Input a batch of data samples consisting of multiple positive samples and multiple negative samples, and generate their embedding vectors through the Actor network.

[0084] embedding text_a =Actor(text_a)

[0085] embedding text_b =Actor(text_b)

[0086] In the formula, embedding text_a The embedding vector obtained by the Actor network with text text_a as input, embedding text_b It is the Embedding vector obtained by the Actor network with the text text_b as input.

[0087] Calculate the cosine similarity of the embedding vectors of all samples respectively. Use square loss to adjust the Actor network, use the network gradient optimizer Adam combined with the loss function to perform gradient backpropagation on the Actor network, and change the embedding vector output by the Actor network.

[0088]

[0089] Actors next =Adam(Actor,loss)

[0090] embedding ′ text_a =Actor next (text_a)

[0091] embedding ′ text_b =Actor next (text_b)

[0092] In the formula, Actor nextis the Actor network adjusted by the network gradient optimizer Adam based on the loss loss, embedding ′ text_a The Embedding vector obtained by the adjusted Actor network with text text_a as input, embedding ′ text_b The Embedding vector obtained by the adjusted Actor network with the text text_b as input.

[0093] Step 6: Obtain the cosine similarity and label calculation reward of the two samples in the same sample pair according to the Actor network before and after adjustment, which specifically includes the following steps:

[0094] The reward is calculated based on whether the labels are similar and how strong the similarity is. If the cosine similarity ranking of any two sample pairs is consistent with the similarity ranking of the labels, the reward increases. For example, if cosine_simi(text_a,text_b)>cosine_simi(text_a,text_c) is consistent with label(text_a,text_b)='similar'>label(text_a,text_c)='weakly similar', the reward increases. Since we focus more on the similarity judgment of positive samples and hope to achieve a more refined similarity distinction, we multiply the number of consistent samples m for both sample pairs as positive samples by the weight coefficient α to add more rewards; multiply the number of consistent samples k for both sample pairs as positive samples by the weight coefficient β, and add the rewards for the two cases.

[0095] reward time =α*m+β*k

[0096] In the formula, α>β.

[0097] Calculate the rewards for the Embedding vectors output by the Actor network before and after the adjustment. The final reward value is the reward value of the next state plus the difference between the rewards of the previous and next states.

[0098] reward1=α*m1+β*k1

[0099] reward next =α*m2+β*k2

[0100] reward=reward next +(reward next -reward1)

[0101] In the formula, reward1 is the reward obtained by adjusting the Embedding vector output by the Actor network before, and reward next The reward obtained by adjusting the Embedding vector output by the Actor network.

[0102] Step 7. The Critic network evaluates the value of the current state and the state at the next moment. The Critic network consists of two layers of perceptrons. It evaluates the value of the current state V(s) and the value of the state at the next moment V(s_next) based on the Embedding vector generated by the Actor network.

[0103] embedding now =concat([embedding text_a ,embedding text_b ])

[0104] x = ReLU(Linear(embedding now ))

[0105] V(s)=Linear(x)

[0106] embedding next =concat([embedding ′ text_a ,embedding ′ text_b ])

[0107] x ′ =ReLU(Linear(embedding next ))

[0108] V(s_next)=Linear(x ′ )

[0109] Step 8. Update the Actor network and Critic network.

[0110] According to the value of the current state and the next state, the Critic network is updated using the temporal difference error td_error and the Adam network gradient optimizer.

[0111] td_error=reward+γ*V(s_next)-V(s)

[0112] critic_loss=td_error 2

[0113] Where critic_loss is the loss of the Critic network.

[0114] The Actor network is updated based on the temporal difference error td_error and the policy gradient optimization method, so that the probability of high-quality Embedding vectors increases and the probability of poor quality decreases. The policy gradient optimization method is shown in the following formula:

[0115]

[0116] log θ (a|s)=log(cosine_simi(embedding ′ text_a ,embedding ′ text_b ))

[0117] Where: θ is the network parameter of the Actor network; is the gradient; log θ (a|s) refers to the logarithmic probability of selecting dynamic a under the network parameters θ and the current state s. In the current scenario, since the cosine similarity between Embedding vectors determines the action we choose, this value is equal to the logarithm of the cosine similarity. The policy gradient method is used to calculate the parameter gradients that need to be updated in the Actor network, and the Actor network is updated.

[0118] Through the above steps, we can use the Actor-Critic method to dynamically optimize the Embedding vector so that it performs better in different similarity calculation tasks. This method combines the advantages of policy gradient and value estimation to generate a more accurate and robust Chinese vectorization model.

Claims

1. A vectorized representation model method for achieving high-accuracy semantic matching, characterized in that: Construct a Chinese vectorization model, which adopts a pseudo twin network structure including an image recognition module, a semantic feature extraction module, and a feature fusion module, wherein: The image recognition module extracts features from the text images of all Chinese characters in the text based on the structure and strokes of Chinese characters to obtain the visual feature vector of the current text; The semantic feature extraction module extracts the semantic feature vector of the same text; The feature fusion module uses an attention mechanism and a multi-layer perceptron to fuse the visual feature vector from the image recognition module with the semantic feature vector from the semantic feature extraction module, and outputs a fused Embedding vector of the current text.

2. A vectorized representation model method for achieving high-accuracy semantic matching as claimed in claim 1, characterized in that: The image recognition module uses the vision transformer model.

3. A vectorized representation model method for achieving high-accuracy semantic matching as claimed in claim 1, characterized in that: The semantic feature module uses a pre-trained sentence-transformer model.

4. A method for implementing a vectorized representation model for achieving high-accuracy semantic matching, characterized in that: Using the Actor-Critic method to dynamically optimize the Embedding vector output by the Chinese vectorization model constructed by the method of claim 1, the Chinese vectorization model is an Actor network, and the implementation method includes the following steps: Step 1: Prepare the dataset: After obtaining the original text, data enhancement is performed on the original text to obtain the data-enhanced text. The original text and the corresponding data-enhanced text are used as two samples to form a pair of positive samples in the data set, and the original text and other text are used as two samples to form a pair of negative samples in the data set. The similarity between the two samples is used as the label of the positive sample pair and the negative sample pair, where the similarity between the two samples in the negative sample pair is the lowest; Step 2: Initialize the Actor network and the Critic network, and define the network gradient optimizer Adam; Step 3: Select multiple pairs of positive sample pairs and negative sample pairs from the data set constructed in step 1, input the samples in the positive sample pairs and negative sample pairs into the current Actor network, generate an Embedding vector for each sample through the current Actor network, and obtain the current state of the current Actor network; Step 4: Calculate the cosine similarity of the Embedding vectors of all samples, obtain the square loss based on the cosine similarity, and use the network gradient optimizer Adam to perform gradient backpropagation on the current Actor network in combination with the square loss, change the Embedding vector output by the current Actor network, and obtain the next state of the current Actor network. Step 5. Calculate rewards: Based on the labels of the sample pairs, we obtain: the number m1 of sample pairs that are both positive and consistent before the corresponding Embedding vector is changed, and the number m2 of sample pairs that are both positive and consistent after the corresponding Embedding vector is changed, the number k1 of sample pairs that are not positive and consistent before the corresponding Embedding vector is changed, and the number k2 of sample pairs that are not positive and consistent after the corresponding Embedding vector is changed, where the consistency of the two sample pairs means that the cosine similarity ranking of all samples contained in the two sample pairs is consistent with the similarity ranking of the labels; Calculate the reward value reward: reward=reward next +(reward next -reward1) Where, reward1=α*m1+β*k1, reward next = α*m2+β*k2, α and β are weight coefficients, α>β; Step 6. The Critic network evaluates the value of the current state V(s) and the value of the next state V(s_next): embedding now =concat([embedding text_a ,embedding text_b ]) x=ReLU(Linear(embedding now )) V(s)=Linear(x) embedding next =concat([embedding ′ text_a ,embedding ′ text_b ]) x ′ =ReLU(Linear(embedding next )) V(s_next)=Linear(x ′ ) In the formula, embedding ′ text_a The Embedding vector obtained by the changed current Actor network with text text_a as input, embedding ′ text_b The Embedding vector obtained by the changed current Actor network with text text_b as input, embedding text_a The embedding vector obtained by the current Actor network before the change with the text text_a as input, embedding text_b The Embedding vector obtained by the current Actor network before the change with the text text_b as input; Step 7. Update the Actor network and Critic network: According to the value of the current state V(s) and the value of the next state V(s_next), the Critic network is updated using the temporal difference error td_error and the network gradient optimizer Adam, where td_error = reward + γ * V(s_next) - V(s), and the loss used by the network gradient optimizer Adam is critic_loss = td_error 2 ; The Actor network is updated according to the temporal difference error td_error and the policy gradient optimization method, where the policy gradient optimization method is shown as follows: log θ (a|s)=log(cosine_simi(embedding ′ text_a ,embedding t text_b )) Where: θ is the network parameter of the Actor network; is the gradient; log θ (a|s) refers to the logarithmic probability of selecting dynamic a under the network parameters θ and the current state s; The policy gradient method is used to calculate the parameter gradients that need to be updated in the Actor network, and the Actor network is updated.

5. The method for implementing a vectorized representation model for achieving high-accuracy semantic matching according to claim 4, characterized in that: In step 1, the similarity between the two samples in the positive sample pair is divided into weak similarity, similarity, and very similarity based on the similarity score. The similarity score between text_a and text_b is expressed as similarity(text_a, text_b), text_a is the original text, and text_b is the text after data enhancement, then: similarity(text_a,text_b) =w1*EditDistance(text_a,text_b)+w2 *Concurrence(text_a,text_b) Where: EditDistance(text_a,text_b) is the edit distance between text_a and text_b. len(text_a) is the length of text text_a, len(text_b) is the length of text text_b, opn is the number of data augmentation operations required to transform text text_a into text text_b; Concurrence(text_a, text_b) is the character overlap between text text_a and text text_b, nc is the number of characters that co-occur in text_a and text_b; w1 and w2 are adjustable parameters.

6. A method for implementing a vectorized representation model for achieving high-accuracy semantic matching as claimed in claim 5, characterized in that: For the text text_a and the text text_b: if similarity(text_a, text_b) is between [0.85, 1], the label of the current positive sample pair is "very similar"; if similarity(text_a, text_b) is between [0.6, 0.85), the label of the current positive sample pair is "similar"; if similarity(text_a, text_b) is between [0.3, 0.6), the label of the current positive sample pair is "weakly similar".

7. The method for implementing a vectorized representation model for achieving high-accuracy semantic matching according to claim 1, characterized in that: In step 4, the square loss loss is calculated by the following formula: Where: N is the total number of sample pairs; embedding text_a The embedding vector obtained by the current Actor network with text text_a as input, embedding text_b is the embedding vector obtained by the current Actor network with text text_b as input, cosine_simi(embedding text_a ,embedding text_b ) is the cosine similarity between the Embedding vector of text_a and the Embedding vector of text_b;

Images