Semantic matching method, semantic matching model training method and related equipment

Abstract

The embodiment of the invention provides a semantic matching method, and relates to the field of artificial intelligence. The method comprises the steps of determining a first similarity threshold value corresponding to a first query text according to the to-be-matched first query text and a first vector, and determining a first similarity based on each first candidate text in at least one first candidate text and the first query text, and determining a matching result of the first query text according to the first similarity corresponding to the at least one first candidate text and a first similarity threshold. According to the method, the similarity threshold value is dynamically determined according to the to-be-matched text, and the similarity threshold value is related to the to-be-matched text and the first vector, so that semantic matching of the to-be-matched text is realized based on the similarity threshold value, more related candidate texts can be screened out, irrelevant candidate texts can be filtered out, and the semantic matching accuracy is improved. The method does not need to manually adjust the similarity threshold, can be adaptive to various to-be-matched texts to determine the similarity threshold, and is also suitable for all text semantic matching scenes.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of artificial intelligence, and in particular, to a semantic matching method, a semantic matching model training method, and related devices. Background Art

[0002] Artificial Intelligence (AI) is to use a digital computer or a machine controlled by a digital computer to simulate, extend, and expand human intelligence, and is a theory, method, technology, and application system that can perceive the environment, acquire knowledge, and use knowledge to obtain the best results. In other words, artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a way similar to human intelligence. Artificial intelligence also studies the design principles and implementation methods of various intelligent machines, enabling the machines to have the functions of perception, reasoning, and decision-making. Research in the field of artificial intelligence includes robots, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, AI basic theory, etc.

[0003] Semantic matching is a major branch of natural language processing. For example, a large amount of information provided by users, such as the location of a parking lot, the location of a company, etc., can be stored in advance. The stored information is retrieved according to the content of a user query (Query), and it is required that the retrieved information has a high similarity with the content of the user query, and irrelevant information is excluded. Therefore, there is a need for semantic similarity matching, and the most matching stored information needs to be screened according to the similarity between the query content and the stored information.

[0004] During the semantic matching process, it is necessary to screen according to the similarity between the candidate item and the query text to obtain a matching result. For example, similarity screening is performed according to a similarity threshold to obtain a matching result. However, when the user's query text can match multiple candidate items, the similarity differences corresponding to the multiple candidate items are relatively large, and at this time, the similarity threshold cannot meet the accuracy requirements of semantic matching.

[0005] Therefore, how to solve the above problems is a hot topic being studied by those skilled in the art. Summary of the Invention

[0006] The present application provides a semantic matching method, a semantic matching model training method, and related devices, which can effectively improve the accuracy of semantic matching.

[0007] In a first aspect, a semantic matching method is provided. This method can be executed by a semantic matching device or by a chip in the semantic matching device.

[0008] The above semantic matching method includes the following steps: Obtain a first query text input by a user, where the first query text is the text to be matched. Obtain at least one first candidate text corresponding to the first query text according to the first query text. Based on a first similarity threshold and based on the first similarity between each first candidate text in the at least one first candidate text and the first query text, determine the matching result between the first query text and the at least one first candidate text. Among them, the first similarity threshold is related to the first query text and a first vector, and the first vector is used to distinguish positive example candidate texts and negative example candidate texts in the at least one first candidate text.

[0009] It can be seen that in this solution, the first similarity threshold corresponding to the first query text is determined according to the first query text to be matched and the first vector. Based on the first similarity between each first candidate text in the at least one first candidate text and the first query text, the first similarity is determined. Then, according to the first similarity corresponding to the at least one first candidate text and the first similarity threshold, the matching result of the first query text is determined. In the embodiment of the present application, the similarity threshold is dynamically determined according to the text to be matched. Since the similarity threshold is related to the text to be matched and the first vector, therefore, based on the above similarity threshold, the semantic matching of the text to be matched is realized, which can not only screen out more relevant candidate texts, filter out irrelevant candidate texts, and improve the accuracy of semantic matching. Moreover, the semantic matching method in the embodiment of the present application does not require manual adjustment of the threshold, can adapt to various texts to be matched to determine the similarity threshold, and can also be applied to all text semantic matching scenarios.

[0010] In a possible implementation manner of the first aspect, when the first similarity of the second candidate text is greater than or equal to the first similarity threshold, the second candidate text is the matching result of the first query text, and the at least one first candidate text includes the second candidate text. Among them, the number of the second candidate texts can be one or more.

[0011] In a possible implementation manner of the first aspect, the above semantic matching method further includes the following steps: Use a first semantic matching model to perform vectorization processing on the first query text to determine a first query vector. Use the first semantic matching model to perform vectorization processing on each first candidate text in the at least one first candidate text to determine at least one first candidate vector. Use the first semantic matching model to perform vectorization processing on the first vector to determine a second vector. Use the first semantic matching model to determine the first similarity based on the first query vector and the first candidate vector. Use the first semantic matching model to determine the first similarity threshold based on the first query vector and the second vector.

[0012] In this solution, the first query text and the first candidate text are first vectorized, and then the first similarity is determined based on the vectorized text. Similarly, the first vector is vectorized to obtain a second vector, and then the semantic similarity between the first query text corresponding to the first query vector and the second vector is determined, and this semantic similarity is used as the first similarity threshold. By converting the first query text, the first candidate text, and the first vector into vector representations, this solution can calculate the first similarity and the first similarity threshold more accurately.

[0013] In a possible implementation manner of the first aspect, the above-mentioned first query text is a text without a previous context. The above semantic matching method further includes the following steps: obtaining a second query text and a first previous text input by the user. The first previous text is the previous text of the second query text; the first previous text includes the first query text. Obtaining at least one third candidate text corresponding to the second query text according to the second query text. Based on the second similarity threshold and based on the second similarity between each third candidate text in the at least one third candidate text and the second query text, determining the matching result between the second query text and the at least one third candidate text.

[0014] Among them, the second similarity of the third candidate text is obtained based on a third vector and a second candidate vector. The third vector is obtained by fusing the first previous vector and the second query vector. The first previous vector is obtained by vectorizing the first previous text. The second query vector is obtained by vectorizing the second query text using the first semantic matching model. The second candidate vector is obtained by vectorizing the third candidate text using the first semantic matching model. The second similarity threshold is obtained based on the third vector and the second vector.

[0015] In this solution, when processing the text to be matched with a previous text, that is, the second query text, the first semantic matching model is used to vectorize the second query text and the third candidate text to obtain a second query vector and a second candidate vector. In addition, the first previous text is vectorized to obtain a first previous vector, and a third vector is obtained by fusing the first previous vector and the first query vector. Then, the second similarity is calculated based on the third vector and the second candidate vector, and the second similarity threshold is calculated based on the third vector and the second vector. Finally, the semantic matching of the second query text is realized based on the second similarity threshold and the second similarity, that is, multi-round semantic matching. Among them, the above-mentioned first semantic matching model is used for semantic matching of query texts without previous texts (such as the first query text), that is, single-round semantic matching; therefore, this solution can realize both single-round semantic matching and multi-round semantic matching, and the single-round semantic matching and the multi-round semantic matching do not affect each other, and the accuracy of both single-round semantic matching and multi-round semantic matching can be ensured at the same time.

[0016] In a possible implementation of the first aspect, when the second similarity of the fourth candidate text is greater than or equal to the second similarity threshold, the fourth candidate text is a matching result of the second query text, and the at least one third candidate text includes the fourth candidate text. The number of fourth candidate texts can be one or more.

[0017] In a second aspect, the present application further provides a semantic matching method, which can be executed by a semantic matching device or by a chip in the semantic matching device.

[0018] The above semantic matching method includes the following steps: obtaining a third query text and a second above-text input by a user, where the second above-text is the above-text of the third query text; obtaining at least one fifth candidate text corresponding to the third query text; and determining a matching result between the third query text and the at least one fifth candidate text based on a third similarity threshold and the third similarity between each fifth candidate text in the at least one fifth candidate text and the third query text.

[0019] Wherein, the third similarity of the fifth candidate text is obtained based on a fourth vector and a third candidate vector. The fourth vector is obtained by fusing a second above-text vector and a third query vector. The second above-text vector is obtained by vectorizing the second above-text. The third query vector is obtained by vectorizing the third query text using a second semantic matching model, and the second semantic matching model is used for semantic matching of query texts without above-text. The third candidate vector is obtained by vectorizing the fifth candidate text using the second semantic matching model. The third similarity threshold can be set or determined according to actual situations.

[0020] In this solution, during multi-round semantic matching, the second semantic matching model is used to vectorize the third query text and the fifth candidate text to obtain a third query vector and a third candidate vector. In addition, the second above-text is vectorized to obtain a second above-text vector, and a fourth vector is obtained by fusing the second above-text vector and the third query vector. Then, the third similarity is calculated based on the fourth vector and the third candidate vector. Finally, semantic matching of the third query text is achieved based on the third similarity threshold and the third similarity. Therefore, this solution can achieve both single-round semantic matching and multi-round semantic matching, and the single-round semantic matching and multi-round semantic matching do not affect each other, while ensuring the accuracy of both single-round semantic matching and multi-round semantic matching.

[0021] In a possible implementation of the second aspect, when the third similarity of the sixth candidate text is greater than or equal to the third similarity threshold, the sixth candidate text is a matching result of the third query text, and the at least one fifth candidate text includes the sixth candidate text. The number of sixth candidate texts can be one or more.

[0022] In a third aspect, the present application further provides a method for training a semantic matching model. This method can be executed by a semantic matching model training device or by a chip in the semantic matching model training device.

[0023] The above-mentioned method for training a semantic matching model includes the following steps: Obtain a first training set. The first training set includes at least two first training texts, as well as the first positive example text and the first negative example text for each first training text. Train a third semantic matching model according to the first training set and a first loss function to obtain a first semantic matching model. The above-mentioned first loss function satisfies: the fourth similarity is greater than the fifth similarity, and the fifth similarity is greater than the sixth similarity. Among them, the fourth similarity is the similarity between the first training text and the first positive example text. The fifth similarity is the similarity between the first training text and the first vector. The sixth similarity is the similarity between the first training text and the first negative example text.

[0024] Among them, the first training text and the first positive example text form a positive example sample pair, and the first training text and the first negative example sample form a negative example sample pair.

[0025] In this solution, the third semantic matching model is trained according to the above-mentioned first loss function to obtain a first semantic matching model. In the first loss function, the fifth similarity is set between the similarity of the positive example sample pair (the fourth similarity) and the similarity of the negative example sample pair (the sixth similarity), so that the first semantic matching model can better distinguish the positive example text and the negative example text, effectively improving the matching accuracy of the first semantic matching model.

[0026] In a possible implementation manner of the third aspect, the above-mentioned first training text is a text without previous context. Correspondingly, the first semantic matching model is used for semantic matching of query texts without previous context, and the first semantic matching model is a single-round semantic matching model.

[0027] The fourth semantic matching model includes the above-mentioned first semantic matching model and the first vectorization module; the above-mentioned method for training a semantic matching model further includes: obtaining a second training set. The second training set includes at least two second training texts, the above-text of each second training text, and the second positive example text and the second negative example text of each second training text. Training the first vectorization module in the fourth semantic matching model according to the second training set and the second loss function to obtain a fifth semantic matching model. Among them, the second loss function satisfies: the seventh similarity is greater than the eighth similarity, and the eighth similarity is greater than the ninth similarity. The seventh similarity is the similarity between the fifth vector and the first positive example vector. The fifth vector is obtained by fusing the second training vector and the third above-text vector. The second training vector is obtained by vectorizing the second training text using the first semantic matching model, and the third above-text vector is obtained by processing the above-text of the second training text using the first vectorization module. The first positive example vector is obtained by vectorizing the second positive example text of the second training text using the first semantic matching model. The eighth similarity is the similarity between the fifth vector and the second vector. The second vector is obtained by vectorizing the first vector using the first semantic matching model. The ninth similarity is the similarity between the fifth vector and the first negative example vector. The first negative example vector is obtained by vectorizing the second negative example text of the second training text using the first semantic matching model.

[0028] In this solution, when training the semantic matching model in multiple rounds, the single-round semantic matching model is frozen, that is, the first semantic matching model is used to vectorize the second training text, the second positive example text, and the second negative example text to obtain the second training vector, the first positive example vector, and the first negative example vector. And the first vectorization module is used to vectorize the above-text of the second training text to obtain the third above-text vector. The fifth vector is obtained by fusing the second training vector and the third above-text vector. Then, the seventh similarity can be determined according to the fifth vector and the first positive example vector, the eighth similarity can be obtained according to the fifth vector and the second vector, and the ninth similarity can be determined according to the fifth vector and the first negative example vector. Adjusting the parameters of the first vectorization module according to the first loss function, when the model training stop condition is met, a fifth semantic matching model, that is, a multi-round semantic matching model, can be obtained. In this solution, by freezing the single-round semantic matching model, the training of the single-round semantic matching and the multi-round semantic matching can be isolated, and thus the single-round semantic matching and the multi-round semantic matching are isolated. The single-round semantic matching and the multi-round semantic matching do not affect each other, and at the same time, the accuracy of the single-round semantic matching and the multi-round semantic matching is ensured.

[0029] Fourthly, the present application further provides a method for training a semantic matching model. This method can be executed by a semantic matching model training device or by a chip in the semantic matching model training device.

[0030] The above semantic matching model training method includes the following steps: obtaining a second training set. The second training set includes at least two second training texts, the above-text of each second training text, and the second positive example text and the second negative example text of each second training text. Training the second vectorization module in the sixth semantic matching model according to the second training set and the third loss function to obtain a seventh semantic matching model. Among them, the sixth semantic matching model further includes a second semantic matching model, and the second semantic matching model is used for semantic matching of query texts without above-text. The above third loss function satisfies: the seventh similarity is greater than the ninth similarity. The above seventh similarity is the similarity between the fifth vector and the first positive example vector. The above fifth vector is obtained by fusing the second training vector and the third above-text vector. The above second training vector is obtained by vectorizing the second training text using the second semantic matching model. The above third above-text vector is obtained by processing the above-text of the second training text using the second vectorization module. The above first positive example vector is obtained by vectorizing the second positive example text of the second training text using the second semantic matching model. The above ninth similarity is the similarity between the fifth vector and the first negative example vector. The above first negative example vector is obtained by vectorizing the second negative example text of the second training text using the second semantic matching model.

[0031] In this solution, when training the semantic matching model in multiple rounds, the single-round semantic matching model is frozen, that is, the second semantic matching model is used to vectorize the second training text, the second positive example text, and the second negative example text to obtain the second training vector, the first positive example vector, and the first negative example vector. And the second vectorization module is used to vectorize the above-text of the second training text to obtain the third above-text vector. The fifth vector is obtained by fusing the second training vector and the third above-text vector. Then, the seventh similarity can be determined according to the fifth vector and the first positive example vector, and the ninth similarity can be determined according to the fifth vector and the first negative example vector. Adjust the parameters of the second vectorization module according to the second loss function. When the model training stop condition is satisfied, the seventh semantic matching model, that is, the multi-round semantic matching model, can be obtained. This solution can isolate the training of single-round semantic matching and multi-round semantic matching, so that single-round semantic matching and multi-round semantic matching are isolated from each other and do not affect each other, while ensuring the accuracy of single-round semantic matching and multi-round semantic matching.

[0032] In a fifth aspect, the present application further provides a semantic matching device, and the device includes units or modules for executing the semantic matching method described in the first aspect or the second aspect.

[0033] In a sixth aspect, the present application further provides a semantic matching model training device, and the device includes units or modules for executing the semantic matching model training method described in the third aspect or the fourth aspect.

[0034] In a seventh aspect, the present application further provides a semantic matching device, including a processor and a memory. The processor is connected to the memory. The memory is used to store program code, and the processor is used to call the program code to execute the semantic matching method described in the first aspect or the second aspect.

[0035] In an eighth aspect, the present application further provides a semantic matching model training device, including a processor and a memory. The processor is connected to the memory. The memory is used to store program code, and the processor is used to call the program code to execute the semantic matching model training method described in the third aspect or the fourth aspect.

[0036] In a ninth aspect, the present application further provides a computer-readable storage medium storing a computer program, and the computer program is executed by a processor to implement the method described in any one of the first aspect to the fourth aspect.

[0037] In a tenth aspect, the present application further provides a computer program product including instructions. When the computer program product runs on a computer, the computer is caused to execute the method described in any one of the first aspect to the fourth aspect.

[0038] In an eleventh aspect, the present application further provides a chip. The chip includes a processor and a data interface. The processor reads instructions stored on a memory through the data interface and executes the method described in any one of the first aspect to the fourth aspect.

[0039] Optionally, as an implementation, the chip may further include a memory storing instructions. The processor is used to execute the instructions stored on the memory. When the instructions are executed, the processor is used to execute the method described in any one of the first aspect to the fourth aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The following introduces the drawings used in the embodiments of the present application.

[0041] Figure 1A is a schematic diagram of a semantic matching scenario provided by an embodiment of the present application;

[0042] Figure 1B is another schematic diagram of a semantic matching scenario provided by an embodiment of the present application;

[0043] Figure 1C is a schematic diagram of a software system architecture of a semantic matching method provided by an embodiment of the present application;

[0044] Figure 1D is a schematic diagram of a system architecture provided by an embodiment of the present application;

[0045] Figure 2 It is a schematic flowchart of a semantic matching model training method provided by an embodiment of the present application;

[0046] Figure 3A It is a schematic diagram of a semantic matching model training method provided by an embodiment of the present application;

[0047] Figure 3B It is a schematic flowchart of another semantic matching model training method provided by an embodiment of the present application;

[0048] Figure 3C It is a schematic diagram of another semantic matching model training method provided by an embodiment of the present application;

[0049] Figure 3D It is a schematic diagram of another semantic matching model training method provided by an embodiment of the present application;

[0050] Figure 4 It is a schematic flowchart of another semantic matching model training method provided by an embodiment of the present application;

[0051] Figure 5 It is a schematic diagram of another semantic matching model training method provided by an embodiment of the present application;

[0052] Figure 6 It is a schematic flowchart of a semantic matching method provided by an embodiment of the present application;

[0053] Figure 7A It is a schematic diagram of a semantic matching method provided by an embodiment of the present application;

[0054] Figure 7B It is a schematic flowchart of another semantic matching method provided by an embodiment of the present application;

[0055] Figure 7C It is a schematic diagram of another semantic matching method provided by an embodiment of the present application;

[0056] Figure 7D It is a schematic diagram of another semantic matching method provided by an embodiment of the present application;

[0057] Figure 8 It is a schematic flowchart of another semantic matching method provided by an embodiment of the present application;

[0058] Figure 9 It is a schematic diagram of another semantic matching method provided by an embodiment of the present application;

[0059] Figure 10 It is a schematic diagram of the structure of a semantic matching device provided by an embodiment of the present application;

[0060] Figure 11 It is a schematic structural diagram of a semantic matching model training device provided by an embodiment of the present application;

[0061] Figure 12 It is a schematic structural diagram of another semantic matching device provided by an embodiment of the present application;

[0062] Figure 13 It is a schematic structural diagram of another semantic matching model training device provided by an embodiment of the present application. Detailed implementation manners

[0063] Next, the technical solutions in the present application will be described with reference to the accompanying drawings.

[0064] In the embodiments of the present application, words such as "exemplary" or "for example" are used to represent examples, illustrations, or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Exactly speaking, using words such as "exemplary" or "for example" aims to present relevant concepts in a specific manner.

[0065] "At least one" mentioned in the embodiments of the present application means one or more, and "a plurality" means two or more. "At least one (item)" or its similar expressions refer to any combination of these items, including any combination of single item (item) or plural items (items). For example, at least one (item) of a, b, or c can represent: a, b, c, (a and b), (a and c), (b and c), or (a and b and c), where a, b, c can be single or multiple. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. Among them, A and B can be singular or plural. The character " / " generally represents an "or" relationship between the front and back associated objects. And the serial numbers of the steps in the embodiments of the present application (such as step S1, step S21, etc.) are only used to distinguish different steps and do not limit the order of execution between steps.

[0066] Moreover, unless otherwise stated, the ordinal numbers such as "first" and "second" used in the embodiments of the present application are used to distinguish multiple objects and are not used to limit the order, time sequence, priority, or importance of multiple objects. For example, the first device and the second device are only for the convenience of description, and do not indicate differences in the structures, importance, etc. of the first device and the second device. In some embodiments, the first device and the second device can also be the same device.

[0067] As used in the above embodiments, depending on the context, the term "when..." can be interpreted to mean "if...", or "after...", or "in response to determining...", or "in response to detecting...". The above are only alternative embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the concept and principles of the present application shall be included within the protection scope of the present application.

[0068] For ease of understanding, relevant terms and related concepts involved in the embodiments of the present application will be introduced first below.

[0069] (1), Neural Network

[0070] A neural network can be composed of neural units. A neural unit can refer to an operation unit that takes x s and the intercept 1 as inputs. The output of this operation unit can be:

[0071]

[0072] where s = 1, 2,..., d, d is a natural number greater than 1, W s is the weight of x s , b is the bias of the neural unit. f is the activation function of the neural unit (Activation Functions), which is used to introduce non-linear characteristics into the neural network to convert the input signal in the neural unit into an output signal. The output signal of this activation function can be used as the input of the next convolutional layer. The activation function can be the Sigmoid function. A neural network is a network formed by connecting many such single neural units together, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the features of the local receptive field, and the local receptive field can be a region composed of several neural units.

[0073] (2), Deep Neural Network

[0074] A deep neural network (Deep Neural Network, DNN), also known as a multi-layer neural network, can be understood as a neural network with many hidden layers. Here, "many" does not have a specific measurement standard. Dividing the DNN according to the positions of different layers, the neural network inside the DNN can be divided into three categories: the input layer, the hidden layer, and the output layer. Generally, the first layer is the input layer, the last layer is the output layer, and the middle layers are all hidden layers. The layers are fully connected, that is, any neuron in the i-th layer must be connected to any neuron in the i + 1-th layer. Although the DNN seems very complex, in terms of the work of each layer, it is actually not complex. Briefly speaking, it is the following linear relationship expression: where, is the input vector, is the output vector, is the offset vector, W is the weight matrix (also known as the coefficient), and α() is the activation function. Each layer simply performs such a simple operation on the input vector to obtain the output vector Since the DNN has many layers, the number of coefficients W and offset vectors is also very large. The definitions of these parameters in the DNN are as follows: Taking the coefficient W as an example: Suppose in a three-layer DNN, the linear coefficient from the 4th neuron in the second layer to the 2nd neuron in the third layer is defined as The superscript 3 represents the layer where the coefficient W is located, and the subscripts correspond to the index 2 of the output third layer and the index 4 of the input second layer.

[0075] In summary: The coefficient from the nth neuron in the (L - 1)th layer to the jth neuron in the Lth layer is defined as It should be noted that there is no W parameter in the input layer. In a deep neural network, more hidden layers enable the network to better depict complex situations in the real world. Theoretically, the more parameters a model has, the higher its complexity and the greater its "capacity", which means it can complete more complex learning tasks. Training a deep neural network is also the process of learning the weight matrix, and its ultimate goal is to obtain the weight matrices of all layers of the trained deep neural network (the weight matrix formed by vectors W of many layers).

[0076] (3) Convolutional Neural Network

[0077] Convolutional Neural Network (CNN) is a deep neural network with a convolutional structure. The convolutional neural network contains a feature extractor composed of convolutional layers and subsampling layers. This feature extractor can be regarded as a filter, and the convolution process can be regarded as convolving a trainable filter with an input image or a convolutional feature plane (Feature Map). The convolutional layer refers to the neuron layer in the convolutional neural network that performs convolution processing on the input signal. In the convolutional layer of the convolutional neural network, a neuron can only be connected to some neighboring layer neurons. In a convolutional layer, there are usually several feature planes, and each feature plane can be composed of some neurons arranged in a rectangle. The neurons in the same feature plane share weights, and the shared weights here are the convolutional kernels. Sharing weights can be understood as a way of extracting image information that is independent of position. The underlying principle here is that the statistical information of a certain part of the image is the same as that of other parts. That is to say, the image information learned in one part can also be used in another part. Therefore, for all positions on the image, the same learned image information can be used. In the same convolutional layer, multiple convolutional kernels can be used to extract different image information. Generally, the more convolutional kernels there are, the richer the image information reflected by the convolution operation.

[0078] The convolutional kernel can be initialized in the form of a matrix of random size, and during the training process of the convolutional neural network, the convolutional kernel can obtain reasonable weights through learning. In addition, the direct benefit brought by sharing weights is to reduce the connections between the layers of the convolutional neural network and at the same time reduce the risk of overfitting.

[0079] (4), Loss function

[0080] During the process of training a deep neural network, since we hope that the output of the deep neural network is as close as possible to the value we really want to predict, we can compare the predicted value of the current network with the real target value, and then update the weight vector of each layer of the neural network according to the difference between the two (of course, there is usually an initialization process before the first update, that is, pre-configuring parameters for each layer in the deep neural network). For example, if the predicted value of the network is too high, we adjust the weight vector to make it predict lower, and keep adjusting until the deep neural network can predict the real target value or a value very close to the real target value. Therefore, it is necessary to pre-define "how to compare the difference between the predicted value and the target value", which is the loss function or objective function. They are important equations used to measure the difference between the predicted value and the target value. Among them, taking the loss function as an example, the higher the output value (Loss) of the loss function, the greater the difference. Then the training of the deep neural network becomes a process of minimizing this Loss as much as possible.

[0081] (5), Ranking Loss

[0082] Ranking Loss is a loss function that is mainly used in metric learning tasks, such as Siamese Nets or Triplet Nets, etc. The goal of Ranking Loss is to predict the relative distance between inputs.

[0083] Ranking Loss usually changes with the training data. It focuses on the scores that measure the similarity between data. This score can be binary (similar or dissimilar). Under the condition of similar samples, the distance between the sample and the input is small; conversely, for dissimilar samples, the distance between the sample and the input is large.

[0084] (6), Transformer model

[0085] The Transformer model mainly consists of two parts: an encoder and a decoder. The encoder maps the input sequence into a fixed-length vector representation, and the decoder combines the output of the encoder with its own input to generate the target sequence. The core idea of the Transformer model is the self-attention mechanism, which allows the model to consider global information when processing sequence data.

[0086] The Transformer model is widely used in various tasks, such as machine translation, text classification, speech recognition, etc.

[0087] (7) Bidirectional Encoder Representations from Transformers (BERT) model

[0088] The goal of the BERT model is to train using a large-scale unlabeled corpus to obtain a description (representation) of the text containing rich semantic information, that is, the semantic representation of the text, and then fine-tune the semantic representation of the text in a specific natural language processing (NLP) task, and finally apply it to this NLP task.

[0089] (8) Semantic matching

[0090] Semantic matching is one of the basic tasks in the NLP field. The direct goal is to determine whether two sentences express the same or similar meanings. It is widely used in scenarios such as search matching, intelligent question answering, and news recommendation.

[0091] In deep learning, text matching models can be divided into two structures: one is the representation-based model, which converts the input text into vector representations and then performs matching by comparing the similarities between these vectors. The other is the interaction-based model, which inputs the input texts into the model together and allows them to exchange information with each other during the encoding process to achieve a better matching effect.

[0092] (9) Positive samples, Negative samples

[0093] In the fields of machine learning and statistics, positive and negative samples are distinguished based on whether the prediction result or prediction category is correct. Positive samples can also be called "positive samples" or "positive examples" or "positive example texts". Negative samples can also be called "negative samples" or "negative examples" or "negative example texts".

[0094] Positive samples refer to samples where the prediction result or prediction category matches the actual category. For example, in a spam classifier, if a mail is predicted to be spam and it is actually spam, then this prediction is correct, and this mail is a positive sample.

[0095] Negative samples refer to samples where the prediction result or prediction category does not match the actual category. For example, in a spam classifier, if a mail is predicted to be non-spam, but it is actually spam, then this prediction is incorrect, and this mail is a negative sample.

[0096] In a semantic matching scenario, the training text used to train the semantic matching model and the positive example sample corresponding to the training text form a positive example sample pair (or positive pair), while the training text and its corresponding negative example sample form a negative example sample pair (or negative pair). For example, the training text is "Who should I invite to dinner on Monday", the positive example sample is "Invite Zhang to dinner on Monday", and the negative example sample is "The keys are in the car". Correspondingly, for candidate items (or candidate texts), they can be divided into positive example candidate items (or positive example candidate texts) and negative example candidate items (or negative example candidate texts).

[0097] During the semantic matching process, it is necessary to screen according to the similarity between the candidate item and the query text to obtain the matching result. For example, similarity screening is performed according to a similarity threshold to obtain the matching result. However, when the user's query text can match multiple candidate items, the similarities corresponding to the multiple candidate items vary greatly, and the similarity threshold at this time cannot meet the accuracy requirements of semantic matching.

[0098] Therefore, the embodiment of the present application provides a semantic matching method, which can determine the similarity threshold corresponding to the text to be matched according to the text to be matched and the fixed vector (the first vector in the embodiment of the present application), and then determine the matching result of the text to be matched according to the similarity threshold. The embodiment of the present application dynamically determines the similarity threshold according to the text to be matched. Since the similarity threshold is related to the text to be matched and the fixed vector, and the fixed vector can be used to distinguish positive example candidate texts and negative example candidate texts in semantic matching, the above similarity threshold is more accurate. Implementing the semantic matching of the text to be matched based on the above similarity threshold can not only screen out more relevant candidate texts, filter out irrelevant candidate texts, and improve the accuracy of semantic matching; moreover, the method of the embodiment of the present application does not require manual threshold adjustment, can adaptively determine the similarity threshold for various texts to be matched, and can also be applied to all text semantic matching scenarios.

[0099] The embodiment of the present application also provides a semantic matching model training method. Taking the similarity of the positive pair being greater than the similarity between the training text and the fixed vector, and the similarity between the training text and the fixed vector being greater than the similarity of the negative pair as the loss function during the training of the semantic matching model can enable the semantic matching model to better distinguish positive example texts and negative example texts, and improve the accuracy of the semantic matching model.

[0100] In addition, during the first-round semantic matching (which can also be understood as a single-round conversation) process, there is no previous context for the query content to be matched, while during the multi-round semantic matching (which can also be understood as a multi-round conversation) process, there is a previous context for the query content to be matched. The above previous context is, for example, the previous-round query content and the previous-round semantic matching result. When training the semantic matching model, it is impossible to ensure the accuracy of both single-round semantic matching and multi-round semantic matching at the same time.

[0101] Therefore, the embodiments of the present application provide a semantic matching method. The above-text of the text to be matched is vectorized separately, while the text to be matched and the candidate text are vectorized respectively through a frozen single-round semantic matching model. Then, a fusion vector is obtained by fusing the vectorized above-text and the vectorized text to be matched. Similarity calculation is performed based on the fusion vector and the vectorized candidate text. Finally, semantic matching is completed based on the calculated similarity and the similarity threshold. Among them, by freezing the single-round semantic matching model, single-round semantic matching and multi-round semantic matching can be isolated. Both single-round semantic matching and multi-round semantic matching can be achieved, ensuring both the accuracy of single-round semantic matching and the accuracy of multi-round semantic matching.

[0102] The embodiments of the present application also provide a method for training a semantic matching model. Taking the similarity of the positive example pair being greater than the similarity of the negative example pair as the loss function during model training. The similarity of the above positive example pair is obtained based on the fusion vector and the vectorized positive example text, and the similarity of the above negative example pair is obtained based on the fusion vector and the vectorized negative example text. The fusion vector is obtained based on the vectorized training text and the above-text of the vectorized training text. A separate vectorization module is set to vectorize the above-text of the training text, while the training text, the positive example text, and the negative example text are vectorized respectively through a frozen single-round semantic matching model. Using the above training method, the training of a multi-round semantic matching model can be achieved, and single-round semantic matching and multi-round semantic matching can be isolated. Single-round semantic matching and multi-round semantic matching do not affect each other, ensuring both the accuracy of single-round semantic matching and the accuracy of multi-round semantic matching.

[0103] The above method for training a semantic matching model can be executed by a semantic matching model training device (refer to the specific description below Figure 1D ), or by a chip in the semantic matching model training device. In the embodiments of the present application, the semantic matching model training device is used as the execution subject for illustration.

[0104] The above semantic matching method can be executed by a semantic matching device, or by a chip in the semantic matching device. In the embodiments of the present application, the semantic matching device is used as the execution subject for illustration.

[0105] The above semantic matching device can be a terminal device such as a mobile phone or a car machine (refer to the specific description below Figure 1D ). The semantic matching method of the embodiments of the present application can be applied to various semantic matching scenarios for the matching of similar semantics.

[0106] The following introduces the semantic matching method of the embodiments of the present application with a specific semantic matching scenario.

[0107] Refer to Figure 1A ,Figure 1A This is a schematic diagram of a semantic matching scenario provided by an embodiment of the present application. Taking a mobile phone as an example, the semantic matching device is exemplarily provided with a memory module and a semantic matching module. The memory module is used to store the memory information provided by the user, such as "Help me remember that A will invite guests to dinner this Sunday", "Help me remember that my car keys are in the first drawer on the left", and "Help me remember that B borrowed 500 yuan on May 23rd", etc. When the user provides a query text, such as "Where are my car keys", the semantic matching module can retrieve and match the memory information in the memory module according to the user's query text, and return the memory information with a high matching degree. For example, the memory information returned at this time is "Help me remember that my car keys are in the first drawer on the left". In the intelligent voice dialogue scenario, the mobile phone can respond to the user's query text and output "The first drawer on the left" to respond to the user's query. The above "output" can be displayed on the display screen of the mobile phone or output as voice on the speaker of the mobile phone.

[0108] Furthermore, Figure 1A In the scenario of, after obtaining the query text, the mobile phone calculates the similarity according to the query text and a fixed vector, takes the calculated similarity as the similarity threshold, and filters the memory information in the memory module according to the similarity threshold. Specifically, first obtain the similarity between each memory information and the query text, and then match according to the similarities corresponding to all memory information and the similarity threshold, and use the memory information with a similarity greater than or equal to the similarity threshold as the matching result.

[0109] Refer to Figure 1B , Figure 1B This is another schematic diagram of a semantic matching scenario provided by an embodiment of the present application. Figure 1B It shows the application of the semantic matching method of the present application to the recall scenario of a voice assistant plugin, and retrieves the plugins that the user may want to call according to the user's query text. Taking a mobile phone as an example, the voice assistant on the mobile phone defines multiple executable plugins (such as the CheckWeather plugin, the Navigate plugin, or the TurnOnBluetooth plugin), and each plugin has corresponding Chinese description information. For example, the Chinese description of the CheckWeather plugin is to view the weather, the Chinese description of the Navigate plugin is navigation, and the Chinese description of the TurnOnBluetooth plugin is to turn on the Bluetooth. When the user wakes up the voice assistant, the voice assistant performs semantic similarity matching according to the user's query text and the description of the plugin, and returns the plugin with a high similarity to the backend module for processing.

[0110] For example, if the user's query text is "Check weather, navigate home", the similarity threshold can be determined based on the query text and the fixed vector, and the plugins can be filtered according to this similarity threshold. Specifically, first obtain the similarity between the Chinese description of each plugin and the query text, and then match according to the similarities of all plugins and the similarity threshold. The plugins with similarities greater than or equal to the similarity threshold are used as the matching results, namely the CheckWeather plugin and the Navigate plugin. The voice assistant can use the CheckWeather plugin to respond to the user's request to check the weather and output the weather conditions today. In addition, the voice assistant can also use the Navigate plugin to respond to the user's request to navigate home and plan the navigation route.

[0111] Furthermore, Figure 1A and Figure 1B the scenario shown is a single-round dialogue scenario. When the next query text provided by the user appears, that is, a multi-round dialogue scenario. At this time Figure 1A or Figure 1B the query text in is the above text of the next query text. In the multi-round semantic matching process, the current query text and the above text need to be utilized. For the relevant descriptions of specific multi-round semantic matching, reference can be made to the relevant descriptions of Figure 7B and Figure 8 in the following text.

[0112] Refer to Figure 1C , Figure 1C which is a schematic diagram of the software system architecture of a semantic matching method provided by an embodiment of the present application;

[0113] The software system architecture of the semantic matching method of the embodiment of the present application includes multiple modules, and the functions of each module are as follows:

[0114] 00: Dialogue control center, responsible for the management of the entire semantic matching dialogue system, including multi-round dialogue management, and carrying the query text request of the user for semantic matching services.

[0115] 01: Semantic matching service module, used to provide complete semantic matching functions.

[0116] 02: Model inference module, used to vectorize the query text of the user.

[0117] 03: Vector engine, used to store the semantic vectors (i.e., candidate vectors) of all candidate texts, and provide the ability to retrieve similarity vectors.

[0118] 04: Data management platform, used to manage the training data of the semantic matching model.

[0119] 05: Model training module, used for data processing, model training, model pushing, and candidate vector set pushing.

[0120] The following introduces a system architecture provided by an embodiment of the present application.

[0121] Refer to the appendix Figure 1D , an embodiment of the present application provides a system architecture 100. As shown in the system architecture 100, the data acquisition device 160 is used to acquire the training data of the semantic matching model. In this embodiment, the training data is a plurality of training texts and a plurality of candidate texts. The data acquisition device 160 stores the training data in the database 130.

[0122] The training device 120 can perform model training based on the training data maintained in the database 130 to obtain the semantic matching model 101. The training device 120 is the semantic matching model training device in the embodiment of the present application. The specific model training process can refer to the following description of the semantic matching model training method (such as Figure 2 and Figure 4 ), which will not be elaborated here. Exemplarily, in the embodiment of the present application, after the model training is completed, the semantic matching model 101 can obtain the ability to perform semantic matching based on the user's query text. The training device 120 can be a server, a cloud service device, etc., and can also be a mobile terminal, a tablet computer, a notebook computer, AR / VR, a vehicle-mounted terminal, a monitoring device, a vehicle-mounted autonomous driving system, a drone and other devices.

[0123] It should be noted that in actual applications, the training data maintained in the database 130 does not necessarily come from the acquisition of the data acquisition device 160, and it may also be received from other devices. Additionally, it should be noted that the training device 120 does not necessarily perform model training completely based on the training data maintained in the database 130, and it may also obtain training data from the cloud or other places for model training. The above description should not be used as a limitation to the embodiment of the present application.

[0124] The semantic matching model 101 obtained after being processed by the training device 120 can be applied and deployed in the system or device of the terminal, such as being applied to the terminal device 110 shown in Figure 1D . The terminal device 110 can process the query text based on the semantic matching model 101 to obtain the corresponding semantic matching result (as shown in Figure 1D ). The terminal device 110 is the semantic matching device in the embodiment of the present application. The terminal device 110 can be a terminal, such as a mobile terminal, a tablet computer, a notebook computer, AR / VR, a vehicle-mounted terminal, a vehicle-mounted computing platform, a vehicle-mounted domain controller, a monitoring device, a vehicle-mounted autonomous driving system, a drone, etc., and can also be a server or a cloud device, etc. The training device 120 and the terminal device 110 can be the same device.

[0125] In the appendix Figure 1DAmong them, the terminal device 110 is configured with an I / O interface 112 for data interaction with external devices. The user can input data to the I / O interface 112 through the client device 140. In this embodiment, the input data is a query text, which can be input by the user or come from the database 130. Among them, the client device 140 can be a text acquisition device, such as a mobile phone, etc.

[0126] Optionally, the preprocessing module 113 is used to preprocess the input data received by the I / O interface 112. In the embodiment of the present application, the preprocessing module 113 is used to preprocess the input data received by the I / O interface 112, and the preprocessed data enters the calculation module 111. In the embodiment of the present application, the input data can be a query text, and the preprocessing module 113 can be used to denoise the query text and remove the noisy text.

[0127] When the terminal device 110 preprocesses the input data, or when the calculation module 111 of the terminal device 110 performs calculations and other related processing processes, the terminal device 110 can call data, codes, etc. in the data storage system 150 for corresponding processing, or can store the data, instructions, etc. obtained from the corresponding processing in the data storage system 150.

[0128] Finally, the I / O interface 112 returns the model processing result of the input data, that is, the semantic matching result, to the client device 140, and thus provides it to the user. At this time, the client device 140 can be a display.

[0129] Optionally, the model processing result can also be used as the input of the calculation module. The calculation module performs other processing operations according to the model processing result. For example, the final output result is obtained according to the model processing result, and the output result is returned to the client device 140 through the I / O interface 112.

[0130] In the case shown in the appendix Figure 1D The user can manually give the input data, and this manual giving can be operated through the interface provided by the I / O interface 112. In another case, the client device 140 can automatically send the input data to the I / O interface 112. If the client device 140 is required to automatically send the input data and user authorization is required, the user can set the corresponding permissions in the client device 140. The user can view the results output by the terminal device 110 in the client device 140, and the specific presentation form can be specific ways such as display, sound, action, etc. The client device 140 can also be used as a data acquisition end to collect, such as Figure 1DThe input data of the input I / O interface 112 and the output result of the output I / O interface 112 are used as new sample data and stored in the database 130. Of course, it is also possible to collect data without going through the client device 140, but directly by the I / O interface 112 such as Figure 1D The input data of the input I / O interface 112 and the output result of the output I / O interface 112 shown are used as new sample data and stored in the database 130.

[0131] It should be noted that Figure 1D is only a schematic diagram of a system architecture provided by an embodiment of the present application, Figure 1D the positional relationship between the devices, components, modules, etc. shown in does not constitute any limitation. For example, in Figure 1D the data storage system 150 is an external memory relative to the terminal device 110. In other cases, the data storage system 150 can also be placed in the terminal device 110.

[0132] Example 1

[0133] The semantic matching model training method will be described below, taking the semantic matching model training device as an example of the execution subject.

[0134] Referring to Figure 2 , Figure 2 is a schematic flowchart of a semantic matching model training method provided by an embodiment of the present application; the semantic matching model training method 200 includes the following steps:

[0135] 201. The semantic matching model training device obtains a first training set.

[0136] Specifically, the first training set includes at least two first training texts and the first positive example text and the first negative example text of each first training text.

[0137] Exemplarily, for each first training text, the corresponding first positive example text and the first negative example text are pre-annotated. The specific number of the first positive example text and the first negative example text can be determined according to the actual situation and is not specifically limited. The first training text and its corresponding first positive example text form a positive example sample pair, and the first training text and its corresponding first negative example text form a negative example sample pair.

[0138] 202. The semantic matching model training device trains a third semantic matching model according to the first training set and a first loss function to obtain a first semantic matching model.

[0139] The above first loss function satisfies that the fourth similarity is greater than the fifth similarity, and the fifth similarity is greater than the sixth similarity.

[0140] Among them, the fourth similarity is the similarity between the first training text and the first positive example text.

[0141] The fifth similarity is the similarity between the first training text and the first vector.

[0142] The sixth similarity is the similarity between the first training text and the first negative example text.

[0143] In the embodiments of the present application, the third semantic matching model is trained according to the above first loss function to obtain the first semantic matching model. In the first loss function, the fifth similarity is set between the similarity of the positive example sample pair (the fourth similarity) and the similarity of the negative example sample pair (the sixth similarity), so that the first semantic matching model can better distinguish positive example texts and negative example texts, and effectively improve the matching accuracy of the first semantic matching model.

[0144] In a possible embodiment, during the training process of the semantic matching model, the training objective of the model is that all the first training texts meet the requirements of the first loss function. However, during actual training, when the model training stop condition is met, the number of first training texts that meet the first loss function at this time is less than or equal to the total number of first training texts.

[0145] In addition, the first loss function is also applicable to different first training texts. For example, if the first training texts that meet the first loss function include the first training text A and the first training text B, then the fourth similarity is: the similarity between the first training text A and the first positive example text A of the first training text A, and the similarity between the first training text B and the first positive example text B of the first training text B. And the fifth similarity is: the similarity between the first training text A and the first vector, and the similarity between the first training text B and the first vector. In addition, the sixth similarity is: the similarity between the first training text A and the first negative example text A of the first training text A, and the similarity between the first training text A and the first negative example text A of the first training text A. Any of the above fourth similarities is greater than any of the fifth similarities, and any of the fifth similarities is greater than any of the sixth similarities.

[0146] For example, the similarity between the first training text A and the first positive example text A of the first training text A is greater than the similarity between the first training text A and the first vector, and the similarity between the first training text A and the first vector is greater than the similarity between the first training text A and the first negative example text A of the first training text A. Or, the similarity between the first training text B and the first positive example text B of the first training text B is greater than the similarity between the first training text B and the first vector, and the similarity between the first training text B and the first vector is greater than the similarity between the first training text B and the first negative example text B of the first training text B.

[0147] For another example, the similarity between the first training text A and the first positive example text A of the first training text A is greater than the similarity between the first training text B and the first vector, and the similarity between the first training text B and the first vector is greater than the similarity between the first training text A and the first negative example text A of the first training text A. Or, the similarity between the first training text A and the first positive example text A of the first training text A is greater than the similarity between the first training text A and the first vector, and the similarity between the first training text A and the first vector is greater than the similarity between the first training text B and the first negative example text B of the first training text B.

[0148] In a possible implementation, the first training text, the first positive example text, the first vector, and the first negative example text are respectively vectorized to obtain a first training vector, a second positive example vector, a sixth vector, and a second negative example vector. The above fourth similarity is calculated based on the first training vector and the second positive example vector of the first training text. The above fifth similarity is calculated based on the first training vector and the sixth vector of the first training text. The above fourth similarity is calculated based on the first training vector and the second negative example vector of the first training text. There are many specific methods for calculating the similarity based on vectors. For example, the cosine distance or the Euclidean distance between vectors can be used as the similarity between vectors. In the embodiments of the present application, the cosine distance is taken as an example of the similarity.

[0149] Exemplarily, the candidate text set used during the training of the semantic matching model is the same as the candidate text set used during the inference. By annotating the training data for the candidate text set, the first positive example text and the first negative example text corresponding to the first training text can be obtained.

[0150] Reference Figure 3A , Figure 3A is a schematic diagram of a method for training a semantic matching model provided by an embodiment of the present application; the third semantic matching model includes a third vectorization module, a fourth vectorization module, and a fifth vectorization module. The first training text is vectorized by using the third vectorization module to obtain a first training vector u1. The first vector is vectorized by using the fourth vectorization module to obtain a sixth vector g1. For candidate texts, whether they are positive example texts or negative example texts, the candidate texts are vectorized by using the fifth vectorization module to obtain a candidate vector d1, and the candidate vector d1 can be a second positive example vector or a second negative example vector.

[0151] The above first vector is a calibration vector. As long as the first vector is a fixed vector, the dimension of the first vector is consistent with that of the first training vector u1 and the candidate vector d1. For example, the first vector can be a vector of all 1s or a text vector obtained from a fixed text, as long as the vector of all 1s or the text vector is consistent with the dimensions of the first training vector u1 and the candidate vector d1. In addition, the dimension of the sixth vector g1 is also consistent with that of the first training vector u1 and the candidate vector d1.

[0152] The specific structures of the above third vectorization module, fourth vectorization module, and fifth vectorization module may be the same or different. For example, the third vectorization module is implemented using the BERT model, while the fourth vectorization module and the fifth vectorization module are implemented using the Transformer model. Another example is that the third vectorization module, the fourth vectorization module, and the fifth vectorization module are all implemented using the BERT model. For details, refer to Figure 3A , taking the third vectorization module as an example, the third vectorization module includes a BERT model and a pooling layer.

[0153] Refer to Figure 3A , and use the first training set and the first loss function to train the third semantic matching model. The first loss function satisfies:

[0154] cos(Query+,Description+)>cos(Query,Calibration)>cos(Query-,Description-);

[0155] Among them, cos(Query+,Description+) represents the fourth similarity, that is, the similarity of the positive example pair; cos(Query,Calibration) represents the fifth similarity, and cos(Query-,Description-) represents the sixth similarity, that is, the similarity of the negative example pair. "Query" represents the first training text, "Description+" represents the first positive example text, and "Description-" represents the first negative example text. Cos represents the cosine distance. The above first loss function is a RankingLoss function.

[0156] Using the first loss function for semantic matching model training ensures that the similarity distance between the first training vector u1 and the second vector is between the similarity of the positive example pair and the similarity of the negative example pair. Adding the first vector enables the first semantic matching model to better distinguish positive and negative samples and improves the accuracy of the first semantic matching model.

[0157] In a possible embodiment, the third semantic matching model is trained according to a first loss function to obtain a first semantic matching model. The first semantic matching model can perform semantic matching based on the similarity between a query text and a candidate text and a similarity threshold, and the similarity threshold can be determined by various methods without limitation. For example, a similarity threshold determined according to the actual situation or a similarity threshold predicted by a neural network.

[0158] In another possible embodiment, the similarity threshold for the first semantic matching model to perform semantic matching is related to the query text and the first vector. An embodiment of the present application proposes a method for solving a dynamic threshold, and the solution of the dynamic similarity threshold is related to the user's query text and the training data distribution of the first training set. Specifically, the solution of the dynamic threshold is added to the training process of the third semantic matching model, and during inference, the dynamic threshold can be automatically calculated according to different query texts and the first vector, the most matching candidate text can be selected, and irrelevant candidate texts can be filtered out, which can effectively improve the accuracy of the first semantic matching model. At this time, the first semantic matching model includes a trained third vectorization module, a trained fourth vectorization module, and a trained fifth vectorization module. The trained third vectorization module is used to vectorize the query text; the trained fourth vectorization module is used to vectorize the first vector; the trained fifth vectorization module is used to vectorize the candidate text.

[0159] The above first training text is text without context. Correspondingly, the first semantic matching model is used for semantic matching of query texts without context, and the first semantic matching model is a single-round semantic matching model. The training process of the multi-round semantic matching model is introduced below.

[0160] In a possible implementation manner, refer to Figure 3B , Figure 3B is a schematic flowchart of another semantic matching model training method provided by an embodiment of the present application; the above semantic matching model training method 200 further includes:

[0161] 203. The semantic matching model training device obtains a second training set.

[0162] Specifically, the second training set includes at least two second training texts, the context text of each second training text, and the second positive example text and the second negative example text of each second training text.

[0163] Exemplarily, for each second training text, the corresponding second positive example text and second negative example text are pre-annotated. The specific numbers of the second positive example text and the second negative example text can be determined according to the actual situation and are not specifically limited. The second training text and its corresponding second positive example text form a positive example sample pair, and the second training text and its corresponding second negative example text form a negative example sample pair.

[0164] Exemplarily, the candidate text set used during the training of the semantic matching model is the same as the candidate text set used during inference. By annotating the training data for the candidate text set, the second positive example text and the second negative example text corresponding to the second training text can be obtained.

[0165] 204. The semantic matching model training device trains the first vectorization module in the fourth semantic matching model according to the second training set and the second loss function to obtain the fifth semantic matching model.

[0166] Among them, the second loss function satisfies: the seventh similarity is greater than the eighth similarity, and the eighth similarity is greater than the ninth similarity.

[0167] The fourth semantic matching model includes the first semantic matching model obtained in step 202 above and the first vectorization module.

[0168] The seventh similarity is the similarity between the fifth vector and the first positive example vector. The fifth vector is obtained by fusing the second training vector and the third above-text vector. The above fusion can be operations such as addition, subtraction (for example, the second training vector minus the third above-text vector), or weighted operation vectors of the second training vector and the third above-text vector, as long as the dimension of the fused fifth vector is the same as the dimensions of the second training vector, the second vector, and the candidate vector. The above weighted operation vector is obtained by adding the first product vector and the second product vector. The first product vector is obtained by multiplying the second training vector by the first weight, and the second product vector is obtained by multiplying the third above-text vector by the second weight. The specific values of the first weight and the second weight can be set according to the actual situation.

[0169] The second training vector is obtained by vectorizing the second training text using the first semantic matching model; the third above-text vector is obtained by processing the above-text of the second training text using the first vectorization module. The first positive example vector is obtained by vectorizing the second positive example text of the second training text using the first semantic matching model.

[0170] The eighth similarity is the similarity between the fifth vector and the second vector. The second vector is obtained by vectorizing the first vector using the first semantic matching model.

[0171] The ninth similarity is the similarity between the fifth vector and the first negative example vector, and the first negative example vector is obtained by vectorizing the second negative example text of the second training text using the first semantic matching model.

[0172] In the embodiments of the present application, when training the semantic matching model for multiple rounds, the single-round semantic matching model is frozen, that is, the first semantic matching model is used to vectorize the second training text, the second positive example text, and the second negative example text to obtain the second training vector, the first positive example vector, and the first negative example vector. And the first vectorization module is used to vectorize the above-text of the second training text to obtain the third above-text vector. The fifth vector is obtained by fusing the second training vector and the third above-text vector. Then, the seventh similarity can be determined according to the fifth vector and the first positive example vector, the eighth similarity can be obtained according to the fifth vector and the second vector, and the ninth similarity can be determined according to the fifth vector and the first negative example vector. The parameters of the first vectorization module are adjusted according to the first loss function. When the model training stop condition is satisfied, the fifth semantic matching model, that is, the multi-round semantic matching model, can be obtained. In the embodiments of the present application, by freezing the single-round semantic matching model, the training of the single-round semantic matching and the multi-round semantic matching can be isolated, and thus the single-round semantic matching and the multi-round semantic matching are isolated. The single-round semantic matching and the multi-round semantic matching do not affect each other, and at the same time, the accuracy of both the single-round semantic matching and the multi-round semantic matching can be ensured.

[0173] Further exemplarily, the second positive example text and the second negative example text may be the same as the above-mentioned first positive example text and first negative example text; that is, the candidate text sets of the single-round semantic matching and the multi-round semantic matching are the same. The single-round semantic matching and the multi-round semantic matching share the candidate vector set of the candidate text, and at the same time ensure the accuracy of both the single-round semantic matching and the multi-round semantic matching.

[0174] The specific structure of the above-mentioned first vectorization module can be set according to the actual situation, as long as the dimension of the third above-text vector is consistent with the dimension of the second training vector. Exemplarily, the first vectorization module includes an encoding layer. Also exemplarily, the first vectorization module includes a trained third vectorization module (or a trained fourth vectorization module or a trained fifth vectorization module) and an encoding layer. The above-mentioned encoding layer can also be called a network transformation layer, which is used to process the vector obtained by the trained third vectorization module (or the trained fourth vectorization module or the trained fifth vectorization module) to obtain the third above-text vector. The encoding layer can be various network layers, which are not limited. For example, the encoding layer is one or more linear (Linear) layers.

[0175] Reference Figure 3C , Figure 3C is a schematic diagram of another semantic matching model training method provided by the embodiments of the present application; for the training of the multi-round semantic matching model, first according to Figure 3AThe method is used to train the third semantic matching model to obtain the first semantic matching model, then freeze the first semantic matching model, and then use the data of the second training set and the second loss function to train the fourth semantic matching model. Similarly, the second loss function is a Ranking Loss function.

[0176] Specifically, the trained third vectorization module in the first semantic matching model is used to vectorize the second training text to obtain the second training vector u2. The trained fourth vectorization module in the first semantic matching model is used to vectorize the first vector to obtain the second vector g2. The trained fifth vectorization module in the first semantic matching model is used to vectorize the candidate text (such as the second positive example text and the second negative example text) to obtain the candidate vector d2, and the candidate vector d2 can be the first positive example vector or the first negative example vector. And the first vectorization module is used to vectorize the above-text of the second training text to obtain the third above-text vector. Figure 3C In, the first vectorization module takes the trained third vectorization module (parameters are frozen) and the encoding layer as an example.

[0177] Further, the fifth vector is obtained by fusing the third above-text vector and the second training vector, and the fifth vector is equated to Figure 3A the first training vector u1 in, and the seventh similarity, the eighth similarity, and the ninth similarity are calculated by a similar method. The first vectorization module is trained according to the second loss function to obtain the fifth semantic matching model, that is, the multi-round semantic matching model. Among them, Figure 3C in, the unfrozen encoding layer is trained to obtain the fifth semantic matching model. When the first vectorization module does not include the frozen module, the first vectorization module is trained to obtain the fifth semantic matching model.

[0178] In the embodiments of the present application, by freezing the single-round semantic matching model, the training of the single-round semantic matching and the multi-round semantic matching is realized in isolation, and the candidate vector sets of the candidate texts are shared by the single-round semantic matching and the multi-round semantic matching, which can ensure the accuracy of both the single-round semantic matching and the multi-round semantic matching at the same time. Without affecting the first-round dialogue scenario, the accuracy of the multi-round dialogue scenario is effectively improved.

[0179] In another possible embodiment, refer to Figure 3D , Figure 3D is a schematic diagram of another semantic matching model training method provided by the embodiments of the present application; Figure 3D Provide another multi-round semantic matching model training method, which is the same as Figure 3CDifferent from the method shown, in the embodiment of the present application, the fifth vector u3 is obtained by vectorizing the text obtained by splicing the second training text and the above-text of the (second training text) using the trained third vectorization module in the first semantic matching model. Equating the fifth vector u3 to Figure 3A the first training vector u1 in, the seventh similarity, the eighth similarity, and the ninth similarity are calculated using a similar method. Specifically, the seventh similarity is the similarity between the fifth vector u3 and the first positive example vector. The eighth similarity is the similarity between the fifth vector u3 and the second vector g2. The ninth similarity is the similarity between the fifth vector u3 and the first negative example vector. The first vectorization module is trained according to the second loss function to obtain the fifth semantic matching model, that is, the multi-round semantic matching model.

[0180] Example 2

[0181] The semantic matching model training method in the multi-round semantic matching scenario will be described below, taking the semantic matching model training device as the execution subject.

[0182] Referring to Figure 4 , Figure 4 is a schematic flowchart of another semantic matching model training method provided by the embodiment of the present application; the semantic matching model training method 400 includes the following steps:

[0183] 401. The semantic matching model training device obtains a second training set.

[0184] Specifically, the second training set includes at least two second training texts, the above-text of each second training text, and the second positive example text and the second negative example text of each second training text.

[0185] 402. The semantic matching model training device trains the second vectorization module in the sixth semantic matching model according to the second training set and the third loss function to obtain the seventh semantic matching model.

[0186] Specifically, the sixth semantic matching model further includes a second semantic matching model, which is used for semantic matching of query texts without above-text. The second semantic matching model can be trained according to various methods.

[0187] The above third loss function satisfies: the seventh similarity is greater than the ninth similarity.

[0188] The above-mentioned seventh similarity is the similarity between the fifth vector and the first positive example vector. The above-mentioned fifth vector is obtained by fusing the second training vector and the third context vector. The above-mentioned second training vector is obtained by vectorizing the second training text using the second semantic matching model. The above-mentioned third context vector is obtained by processing the context text of the second training text using the second vectorization module. The above-mentioned first positive example vector is obtained by vectorizing the second positive example text of the second training text using the second semantic matching model.

[0189] The above-mentioned ninth similarity is the similarity between the fifth vector and the first negative example vector. The above-mentioned first negative example vector is obtained by vectorizing the second negative example text of the second training text using the second semantic matching model.

[0190] In the embodiments of the present application, when training the semantic matching model for multiple rounds, the single-round semantic matching model is frozen. That is, the second semantic matching model is used to vectorize the second training text, the second positive example text, and the second negative example text to obtain the second training vector, the first positive example vector, and the first negative example vector. And the second vectorization module is used to vectorize the context text of the second training text to obtain the third context vector. The fifth vector is obtained by fusing the second training vector and the third context vector. Then, the seventh similarity can be determined according to the fifth vector and the first positive example vector, and the ninth similarity can be determined according to the fifth vector and the first negative example vector. The parameters of the second vectorization module are adjusted according to the second loss function. When the model training stop condition is satisfied, the seventh semantic matching model, that is, the multi-round semantic matching model, can be obtained. By isolating the training of single-round semantic matching and multi-round semantic matching in the embodiments of the present application, single-round semantic matching and multi-round semantic matching can be isolated, and single-round semantic matching and multi-round semantic matching do not affect each other, while ensuring the accuracy of single-round semantic matching and multi-round semantic matching.

[0191] Reference Figure 5 , Figure 5 is a schematic diagram of another semantic matching model training method provided by the embodiments of the present application; the above-mentioned second semantic matching model includes a sixth vectorization module and a seventh vectorization module. The sixth vectorization module is used to vectorize the query text, and the seventh vectorization module is used to vectorize the candidate text. The specific structures of the sixth vectorization module and the seventh vectorization module can be set according to actual situations, and the specific structures of the sixth vectorization module and the seventh vectorization module are the same or different. For example, the sixth vectorization module is implemented using the BERT model, while the seventh vectorization module is implemented using the Transformer model. Another example is that both the sixth vectorization module and the seventh vectorization module are implemented using the BERT model. Specifically, refer to Figure 5 , taking the sixth vectorization module as an example, the sixth vectorization module includes a BERT model and a pooling layer.

[0192] In the embodiments of the present application, the sixth vectorization module in the second semantic matching model is used to vectorize the second training text to obtain the second training vector u2. And the seventh vectorization module in the second semantic matching model is used to vectorize the candidate text (such as the second positive example text or the second negative example text) to obtain the candidate vector d2, and the candidate vector d2 can be the first positive example vector or the first negative example vector.

[0193] The specific structure of the above-mentioned second vectorization module can be set according to the actual situation, as long as the dimension of the third above-text vector is the same as that of the second training vector. Exemplarily, the second vectorization module includes an encoding layer. And again exemplarily, referring to Figure 5 , the second vectorization module includes the sixth vectorization module (or the seventh vectorization module) and an encoding layer. The above-mentioned encoding layer can also be called a network transformation layer, which is used to process the vector obtained by the sixth vectorization module (or the seventh vectorization module) to obtain the third above-text vector. The encoding layer can be various network layers without limitation. For example, the encoding layer is one or more linear layers.

[0194] The fusion in the above-mentioned fifth vector obtained by fusing the second training vector and the third above-text vector can be operations such as addition, subtraction (such as subtracting the third above-text vector from the second training vector), or weighted operation vectors of the second training vector and the third above-text vector, as long as the dimension of the fused fifth vector is the same as the dimension of the second training vector, the dimension of the second vector, and the dimension of the candidate vector. The above-mentioned weighted operation vector is obtained by adding the first product vector and the second product vector. The first product vector is obtained by multiplying the second training vector by the first weight, and the second product vector is obtained by multiplying the third above-text vector by the second weight. The specific values of the first weight and the second weight can be set according to the actual situation.

[0195] Referring to Figure 5 , in the embodiments of the present application, the third above-text vector and the second training vector are added to obtain the fifth vector. The seventh similarity is calculated based on the fifth vector and the first positive example vector, and the ninth similarity is calculated based on the fifth vector and the first negative example vector. Then, the second vectorization module is trained based on the third loss function to obtain the seventh semantic matching model. The third loss function is a Ranking Loss function. Exemplarily, Figure 5 only the encoding layer is trained to obtain the seventh semantic matching model. When the second vectorization module does not include a frozen module, the second vectorization module is trained to obtain the seventh semantic matching model.

[0196] There can be many specific methods for calculating the similarity based on vectors. For example, the cosine distance or Euclidean distance between vectors can be used as the similarity between vectors. In the embodiments of the present application, the cosine distance is taken as an example of the similarity.

[0197] Example 3

[0198] Corresponding to Embodiment 1, the present application provides a semantic matching method, and the execution subject is taken as a semantic matching device as an example.

[0199] Refer to Figure 6 , Figure 6 is a schematic flowchart of a semantic matching method provided by the embodiments of the present application; the semantic matching method 600 includes the following steps:

[0200] 601. The semantic matching device obtains a first query text input by a user.

[0201] Specifically, the first query text is the text to be matched, and there can be various ways to obtain the first query text. For example, it can be obtained by voice input, input on a display screen, or input by button selection, etc.; there is no special limitation.

[0202] 602. The semantic matching device obtains at least one first candidate text corresponding to the first query text.

[0203] Exemplarily, the semantic matching device can obtain at least one first candidate text corresponding to the first query text according to the first query text. For example, no matter what kind of query text, it corresponds to a candidate text set (including at least one candidate text), and then the candidate texts in the candidate text set are used as the above at least one first candidate text. Another example is to determine at least one first candidate text corresponding to the first query text according to the query type of the first query text. Different candidate text sets (each candidate text set includes at least one candidate text) are set for different query types. For example, a candidate text set for work, a candidate text set for life, and a candidate text set for plugins. Determine the query type according to the first query text, determine the corresponding candidate text set according to the query type, and use the candidate texts in the candidate text set as the above at least one first candidate text.

[0204] 603. The semantic matching device determines the matching result between the first query text and at least one first candidate text based on a first similarity threshold and based on the first similarity between each first candidate text in at least one first candidate text and the first query text.

[0205] Specifically, the first similarity threshold is related to the first query text and the first vector, and the first vector is used to distinguish positive example candidate texts and negative example candidate texts in at least one first candidate text. For the specific determination method of the first vector, reference can be made to the relevant description in Embodiment 1, which will not be elaborated here.

[0206] In the embodiments of the present application, the first similarity threshold corresponding to the first query text is determined according to the first query text to be matched and the first vector. The first similarity between each first candidate text in at least one first candidate text and the first query text is determined, and then the matching result of the first query text is determined according to the first similarity corresponding to at least one first candidate text and the first similarity threshold. In the embodiments of the present application, the similarity threshold is dynamically determined according to the text to be matched. Since the similarity threshold is related to the text to be matched and the first vector, semantic matching of the text to be matched is realized based on the above similarity threshold, which can not only screen out more relevant candidate texts, filter out irrelevant candidate texts, and improve the accuracy of semantic matching. Moreover, the semantic matching method in the embodiments of the present application does not require manual adjustment of the threshold, can adapt to various texts to be matched to determine the similarity threshold, and can also be applied to all text semantic matching scenarios.

[0207] In a possible implementation manner, in step 603, when determining the matching result of the first query text and at least one first candidate text, when the first similarity of the second candidate text is greater than or equal to the first similarity threshold, the second candidate text is the matching result of the first query text, and at least one first candidate text includes the second candidate text. The number of the second candidate texts can be one or more.

[0208] In another possible implementation manner, when determining the matching result based on the first similarity threshold, assuming that the number of matching results determined based on the first similarity threshold is X, the number of matching results of the first query text can be extended to, that is, the first candidate texts ranked in the top X*a + b in terms of the first similarity in at least one first candidate text are output as the matching results of the first query text, and the specific values of a and b can be set according to the actual situation.

[0209] For example, assume that X is 1, a is 1, and b is 2. The original matching result of the first query text is 1 first candidate text. By expanding the number of matching results, the output matching result is 3, and the matching results of the first query text are the first candidate texts ranked in the top 3 in terms of the first similarity.

[0210] In a possible implementation manner, refer to Figure 7A , Figure 7A is a schematic diagram of a semantic matching method provided by the embodiments of the present application; the above semantic matching method 600 further includes the following steps:

[0211] The semantic matching device uses the first semantic matching model 701 to vectorize the first query text to determine the first query vector.

[0212] The semantic matching device uses the first semantic matching model 701 to vectorize each of the at least one first candidate text to determine at least one first candidate vector.

[0213] The semantic matching device uses the first semantic matching model 701 to vectorize the first vector to determine the second vector.

[0214] The semantic matching device uses the first semantic matching model 701 to determine the first similarity based on the first query vector and the first candidate vector.

[0215] The semantic matching device uses the first semantic matching model 701 to determine the first similarity threshold based on the first query vector and the second vector.

[0216] In the embodiments of the present application, the semantic matching device first vectorizes the first query text and the first candidate text, and then determines the first similarity based on the vectorized text. Similarly, the first vector is vectorized to obtain the second vector, and then the semantic similarity between the first query text corresponding to the first query vector and the second vector is determined, and this semantic similarity is used as the first similarity threshold. In the embodiments of the present application, by converting the first query text, the first candidate text, and the first vector into vector representations, the first similarity and the first similarity threshold can be calculated more accurately.

[0217] Specifically, the specific obtaining method of the first semantic matching model 701 can refer to the relevant description in Embodiment 1 and will not be elaborated here.

[0218] Exemplarily, the first candidate vector corresponding to each candidate text can be calculated and stored offline in advance. During online inference, the semantic matching device calls the first semantic matching model to calculate the first query vector of the first query text and the second vector of the first vector. Then, the first similarity between the first query vector and all the first query vectors calculated offline is calculated. Further optionally, the at least one first candidate text can be sorted in descending order of the first similarity. The semantic matching device determines the first similarity threshold according to the first query vector and the second vector. Then, candidate text screening is performed according to the first similarity threshold and all the first similarities. For example, the first candidate text with the first similarity greater than or equal to the first similarity threshold is used as the matching result. Exemplarily, compared with the scheme of using a fixed threshold to screen the matching result, the accuracy rate of the semantic matching method in the embodiments of the present application is increased from 30% to 90%.

[0219] In a possible implementation manner, the above first query text is text without context. Refer to Figure 7B , Figure 7B which is a schematic flowchart of another semantic matching method provided by an embodiment of the present application; the above semantic matching method 600 further includes the following steps:

[0220] 604. The semantic matching device obtains a second query text to be matched and a first context text input by the user.

[0221] Specifically, the first context text is the context text of the second query text; the first context text includes the first query text. Exemplarily, the first context text further includes the matching result of the first query text, and the matching result of the first query text may be a second candidate text.

[0222] 605. The semantic matching device obtains at least one third candidate text corresponding to the second query text according to the second query text.

[0223] Specifically, refer to the description of step 602, which will not be elaborated here.

[0224] 606. The semantic matching device determines the matching result between the second query text and at least one third candidate text based on the second similarity threshold and the second similarity between each third candidate text in the at least one third candidate text and the second query text.

[0225] Wherein, the second similarity of the third candidate text is obtained based on a third vector and a second candidate vector. The third vector is obtained by fusing a first context vector and a second query vector. The first context vector is obtained by vectorizing the first context text. The second query vector is obtained by vectorizing the second query text using a first semantic matching model. The second candidate vector is obtained by vectorizing the third candidate text using the first semantic matching model. The second similarity threshold is obtained based on the third vector and a second vector.

[0226] In the embodiments of the present application, when processing the text to be matched with the above text, that is, the second query text, the first semantic matching model is used to perform vectorization processing on the second query text and the third candidate text to obtain a second query vector and a second candidate vector. In addition, the first above text is vectorized to obtain a first above vector, and a third vector is obtained by fusing the first above vector and the first query vector. Then, a second similarity is calculated based on the third vector and the second candidate vector, and a second similarity threshold is calculated based on the third vector and the second vector. Finally, semantic matching of the second query text is achieved based on the second similarity threshold and the second similarity, that is, multi-round semantic matching. Among them, the above first semantic matching model is used for semantic matching of query texts without above text (such as the first query text), that is, single-round semantic matching. Therefore, this solution can implement both single-round semantic matching and multi-round semantic matching, and the single-round semantic matching and multi-round semantic matching do not affect each other, and the accuracy of both single-round semantic matching and multi-round semantic matching can be ensured at the same time.

[0227] In a possible implementation manner, when the second similarity of the fourth candidate text is greater than or equal to the second similarity threshold, the fourth candidate text is the matching result of the second query text, and the at least one third candidate text includes the fourth candidate text. The number of the fourth candidate texts can be one or more.

[0228] Exemplarily, in the embodiments of the present application, a fifth semantic matching model is used to implement multi-round semantic matching. The fifth semantic matching model includes a trained first vectorization module and a first semantic matching model. The specific acquisition method of the fifth semantic matching model can refer to the relevant description of Embodiment 1 and will not be elaborated here.

[0229] Refer to Figure 7C , Figure 7C is a schematic diagram of another semantic matching method provided by the embodiments of the present application. Exemplarily, during model inference, the semantic matching device obtains the second query text and its corresponding first above text and at least one third candidate text. The semantic matching device uses the trained first vectorization module in the fifth semantic matching model 702 to obtain the first above vector corresponding to the first above text. The semantic matching device uses the first semantic matching model to obtain the second vector of the first vector, the second query vector of the second query text, and the second candidate vector of the third candidate text respectively. The semantic matching device fuses the first above vector and the second query vector to obtain a third vector, determines the second similarity threshold of the second query text according to the third vector and the second vector, and can obtain the second similarity corresponding to each third candidate text according to the third vector and each second candidate vector. Based on the second similarity threshold and all the second similarities, the final matching result can be screened out.

[0230] Refer to Figure 7D ,Figure 7D It is a schematic diagram of another semantic matching method provided by an embodiment of the present application; the semantic matching device uses the trained first vectorization module in the fifth semantic matching model 702 to perform vectorization processing on the text after splicing the second query text and the first above-text to obtain the above third vector. The semantic matching device determines the second similarity threshold of the second query text based on the third vector and the second vector, and based on the third vector and each second candidate vector, the second similarity corresponding to each third candidate text can be obtained. Based on the second similarity threshold and all second similarities, the final matching result can be screened out.

[0231] In the embodiment of the present application, when there is no above-text for the query text, the method shown in Figure 7A is used for semantic matching, and when there is above-text for the query text, the method shown in Figure 7C or Figure 7D is used for semantic matching. It can not only ensure the accuracy of single-round semantic matching but also guarantee the accuracy of multi-round semantic matching.

[0232] Example 4

[0233] Corresponding to Embodiment 2, the embodiment of the present application further provides a semantic matching method, and the execution subject of this method takes a semantic matching device as an example.

[0234] Referring to Figure 8 , Figure 8 is a schematic flowchart of another semantic matching method provided by an embodiment of the present application; the semantic matching method 800 includes the following steps:

[0235] 801. The semantic matching device obtains a third query text and a second above-text input by the user.

[0236] Among them, the second above-text is the above-text of the third query text.

[0237] 802. The semantic matching device obtains at least one fifth candidate text corresponding to the third query text.

[0238] 803. The semantic matching device determines the matching result between the third query text and at least one fifth candidate text based on the third similarity threshold and the third similarity between each fifth candidate text in at least one fifth candidate text and the third query text.

[0239] Among them, the third similarity of the fifth candidate text is obtained based on the fourth vector and the third candidate vector. The fourth vector is obtained by fusing the second context vector and the third query vector. The second context vector is obtained by vectorizing the second context text. The third query vector is obtained by vectorizing the third query text using the second semantic matching model, and the second semantic matching model is used for semantic matching of query texts without context texts. The third candidate vector is obtained by vectorizing the fifth candidate text using the second semantic matching model. The third similarity threshold can be set or determined according to the actual situation.

[0240] In the embodiments of the present application, during multi-round semantic matching, the second semantic matching model is used to vectorize the third query text and the fifth candidate text to obtain the third query vector and the third candidate vector. In addition, the second context text is vectorized to obtain the second context vector, and the fourth vector is obtained by fusing the second context vector and the third query vector. Then, the third similarity is calculated based on the fourth vector and the third candidate vector. Finally, the semantic matching of the third query text is achieved based on the third similarity threshold and the third similarity. Therefore, this solution can implement both single-round semantic matching and multi-round semantic matching, and the single-round semantic matching and multi-round semantic matching do not affect each other, while ensuring the accuracy of both single-round semantic matching and multi-round semantic matching.

[0241] In a possible implementation manner, when the third similarity of the sixth candidate text is greater than or equal to the third similarity threshold, the sixth candidate text is the matching result of the third query text, and the at least one fifth candidate text includes the sixth candidate text. The number of sixth candidate texts can be one or more.

[0242] In the embodiments of the present application, the seventh semantic matching model is used to implement multi-round semantic matching. The seventh semantic matching model includes a trained second vectorization module and a second semantic matching model. The specific acquisition method of the seventh semantic matching model can refer to the relevant description in Embodiment 2 and will not be elaborated here.

[0243] Refer to Figure 9 , Figure 9It is a schematic diagram of another semantic matching method provided by an embodiment of the present application; Exemplarily, during model inference, the semantic matching device obtains a third query text, its corresponding second above-text, and at least one fifth candidate text. The semantic matching device uses the trained second vectorization module in the seventh semantic matching model 901 to obtain the second above-text vector corresponding to the second above-text. The semantic matching device uses the second semantic matching model to obtain the third query vector of the third query text and the third candidate vectors of the fifth candidate texts respectively. The semantic matching device fuses the second above-text vector and the third query vector to obtain a fourth vector, and based on the fourth vector and each third candidate vector, the third similarity corresponding to each fifth candidate text can be obtained. Based on the third similarity threshold and all the third similarities, the final matching result can be screened out.

[0244] The method of the embodiment of the present application is elaborated in detail above. Next, the device provided by the embodiment of the present application will be introduced.

[0245] Figure 10 、 Figure 11 、 Figure 12 and Figure 13 are schematic structural diagrams of possible devices provided by the embodiments of the present application. Among them, Figure 10 The semantic matching device shown can be used to implement the functions of the above-mentioned semantic matching method embodiment, and thus can also achieve the beneficial effects possessed by the above-mentioned semantic matching method embodiment. In the embodiments of the present application, the semantic matching device can be an electronic device or a module (such as a chip) applied to an electronic device. And Figure 11 The semantic matching model training device shown can be used to implement the functions of the above-mentioned semantic matching model training method embodiment, and thus can also achieve the beneficial effects possessed by the above-mentioned semantic matching model training method embodiment. In the embodiments of the present application, the semantic matching model training device can be an electronic device or a module (such as a chip) applied to an electronic device.

[0246] As Figure 10 shown, the semantic matching device 1000 includes an acquisition module 1001 and a determination module 1002. The semantic matching device 1000 is used to implement the functions of the above-mentioned Figure 6 semantic matching method embodiment shown. Or, the semantic matching device 1000 can include modules for implementing any function or operation in the above-mentioned Figure 8 semantic matching method embodiment, and this module can be implemented in whole or in part by software, hardware, firmware, or any combination thereof.

[0247] When the semantic matching device 1000 is used to implement Figure 6When implementing the functions in the method embodiments shown, the obtaining module 1001 is configured to obtain a first query text input by a user. The obtaining module 1001 is further configured to obtain at least one first candidate text corresponding to the first query text. The determining module 1002 is configured to determine a matching result between the first query text and the at least one first candidate text based on a first similarity threshold and a first similarity between each first candidate text in the at least one first candidate text and the first query text.

[0248] Wherein, the first similarity threshold is related to the first query text and a first vector, and the first vector is used to distinguish positive example candidate texts and negative example candidate texts in the at least one first candidate text.

[0249] In a possible implementation manner, when the first similarity of a second candidate text is greater than or equal to the first similarity threshold, the second candidate text is a matching result of the first query text, and the at least one first candidate text includes the second candidate text.

[0250] In a possible implementation manner, the above-mentioned determining module 1002 is further configured to perform vectorization processing on the first query text by using a first semantic matching model to determine a first query vector. The above-mentioned determining module 1002 is further configured to perform vectorization processing on each first candidate text in the at least one first candidate text by using the first semantic matching model to determine at least one first candidate vector. The above-mentioned determining module 1002 is further configured to perform vectorization processing on the first vector by using the first semantic matching model to determine a second vector. The above-mentioned determining module 1002 is further configured to use the first semantic matching model to determine a first similarity based on the first query vector and the first candidate vector. The above-mentioned determining module 1002 is further configured to use the first semantic matching model to determine the first similarity threshold based on the first query vector and the second vector.

[0251] In a possible implementation manner, the above-mentioned first query text is a text without previous context. The obtaining module 1001 is further configured to obtain a second query text to be matched and a first previous text input by the user. The first previous text is the previous text of the second query text; the first previous text includes the first query text. The obtaining module 1001 is further configured to obtain at least one third candidate text corresponding to the second query text. The determining module 1002 is further configured to determine a matching result between the second query text and the at least one third candidate text based on a second similarity threshold and a second similarity between each third candidate text in the at least one third candidate text and the second query text.

[0252] Among them, the second similarity of the third candidate text is obtained based on the third vector and the second candidate vector. The third vector is obtained by fusing the first context vector and the second query vector. The first context vector is obtained by vectorizing the first context text. The second query vector is obtained by vectorizing the second query text using the first semantic matching model. The second candidate vector is obtained by vectorizing the third candidate text using the first semantic matching model. The second similarity threshold is obtained based on the third vector and the second vector.

[0253] In a possible implementation, when the second similarity of the fourth candidate text is greater than or equal to the second similarity threshold, the fourth candidate text is the matching result of the second query text, and the at least one third candidate text includes the fourth candidate text. The number of fourth candidate texts can be one or more.

[0254] For the introduction of each of the above modules, reference can be made to the description in the foregoing embodiments, which will not be elaborated here.

[0255] When the semantic matching device 1000 is used to implement Figure 8 the functions in the method embodiment shown, the obtaining module 1001 is used to obtain the third query text and the second context text input by the user, and the second context text is the context text of the third query text. The obtaining module 1001 is further used to obtain at least one fifth candidate text corresponding to the third query text. The determining module 1002 is used to determine the matching result between the third query text and the at least one fifth candidate text based on the third similarity threshold and the third similarity between each fifth candidate text in the at least one fifth candidate text and the third query text.

[0256] Among them, the third similarity of the fifth candidate text is obtained based on the fourth vector and the third candidate vector. The fourth vector is obtained by fusing the second context vector and the third query vector. The second context vector is obtained by vectorizing the second context text. The third query vector is obtained by vectorizing the third query text using the second semantic matching model, and the second semantic matching model is used for semantic matching of query texts without context texts. The third candidate vector is obtained by vectorizing the fifth candidate text using the second semantic matching model. The third similarity threshold can be set or determined according to the actual situation.

[0257] In a possible implementation, when the third similarity of the sixth candidate text is greater than or equal to the third similarity threshold, the sixth candidate text is the matching result of the third query text, and the at least one fifth candidate text includes the sixth candidate text. The number of sixth candidate texts can be one or more.

[0258] For the introduction of the above modules, please refer to the description of the above embodiments, which will not be repeated here.

[0259] like Figure 11 As shown, the semantic matching model training device 1100 includes an acquisition module 1101 and a training module 1102. The semantic matching model training device 1100 is used to implement the above Figure 2 Alternatively, the semantic matching model training device 1100 may include a method for implementing the above Figure 4 A module for any function or operation in the semantic matching model training method embodiment shown in , which module can be implemented in whole or in part by software, hardware, firmware or any combination thereof.

[0260] When the semantic matching model training device 1100 is used to implement Figure 2 When the function in the method embodiment shown is used, the acquisition module 1101 is used to obtain the first training set. The first training set includes at least two first training texts and the first positive example text and the first negative example text of each first training text. The training module 1102 is used to train the third semantic matching model according to the first training set and the first loss function to obtain the first semantic matching model. The above-mentioned first loss function satisfies: the fourth similarity is greater than the fifth similarity, and the fifth similarity is greater than the sixth similarity. Among them, the fourth similarity is the similarity between the first training text and the first positive example text. The fifth similarity is the similarity between the first training text and the first vector. The sixth similarity is the similarity between the first training text and the first negative example text.

[0261] In a possible implementation, the first training text is a text without a preceding text. Accordingly, the first semantic matching model is used for semantic matching of the query text without a preceding text, and the first semantic matching model is a single-round semantic matching model.

[0262] The fourth semantic matching model includes the first semantic matching model and the first vectorization module; the acquisition module 1101 is also used to acquire a second training set. The second training set includes at least two second training texts, the previous text of each second training text, and the second positive example text and the second negative example text of each second training text. The training module 1102 is also used to train the first vectorization module in the fourth semantic matching model according to the second training set and the second loss function to obtain a fifth semantic matching model.

[0263] Among them, the second loss function satisfies that the seventh similarity is greater than the eighth similarity, and the eighth similarity is greater than the ninth similarity. The seventh similarity is the similarity between the fifth vector and the first positive example vector. The fifth vector is obtained by fusing the second training vector and the third context vector. The second training vector is obtained by vectorizing the second training text using the first semantic matching model, and the third context vector is obtained by processing the context text of the second training text using the first vectorization module. The first positive example vector is obtained by vectorizing the second positive example text of the second training text using the first semantic matching model. The eighth similarity is the similarity between the fifth vector and the second vector, and the second vector is obtained by vectorizing the first vector using the first semantic matching model. The ninth similarity is the similarity between the fifth vector and the first negative example vector, and the first negative example vector is obtained by vectorizing the second negative example text of the second training text using the first semantic matching model.

[0264] For the introduction of each of the above modules, reference can be made to the description in the foregoing embodiments, which will not be elaborated herein.

[0265] When the semantic matching model training device 1100 is used to implement Figure 4 the functions in the method embodiment shown, the obtaining module 1101 is used to obtain a second training set. The second training set includes at least two second training texts, the context text of each second training text, and the second positive example text and the second negative example text of each second training text. The training module 1102 is used to train the second vectorization module in the sixth semantic matching model according to the second training set and the third loss function to obtain a seventh semantic matching model.

[0266] Among them, the sixth semantic matching model further includes a second semantic matching model, which is used for semantic matching of query texts without context text. The above third loss function satisfies that the seventh similarity is greater than the ninth similarity. The above seventh similarity is the similarity between the fifth vector and the first positive example vector. The above fifth vector is obtained by fusing the second training vector and the third context vector. The above second training vector is obtained by vectorizing the second training text using the second semantic matching model, and the above third context vector is obtained by processing the context text of the second training text using the second vectorization module. The above first positive example vector is obtained by vectorizing the second positive example text of the second training text using the second semantic matching model. The above ninth similarity is the similarity between the fifth vector and the first negative example vector, and the above first negative example vector is obtained by vectorizing the second negative example text of the second training text using the second semantic matching model.

[0267] For the introduction of each of the above modules, reference can be made to the description in the foregoing embodiments, which will not be elaborated herein.

[0268] Refer toFigure 12 , Figure 12 is a schematic structural diagram of a semantic matching device provided by an embodiment of the present application; the semantic matching device 1200 includes a memory 1201, a processor 1202, a communication interface 1204, and a bus 1203. Among them, the memory 1201, the processor 1202, and the communication interface 1204 are communicatively connected to each other through the bus 1203.

[0269] Optionally, the above-mentioned semantic matching device 1200 further includes a display screen (not shown), and the display screen is communicatively connected to the memory 1201, the processor 1202, and the communication interface 1204 through the bus 1203. The display screen is used to output information for interaction with the user. Optionally, the above-mentioned semantic matching device 1200 further includes an output module (not shown), and the output module is communicatively connected to the memory 1201, the processor 1202, and the communication interface 1204 through the bus 1203. The output module is used to output information, such as voice output or display output.

[0270] The memory 1201 can be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1201 can store a program. When the program stored in the memory 1201 is executed by the processor 1202, the processor 1202 and the communication interface 1204 are used to execute the steps of the semantic matching method of any embodiment of the present application.

[0271] The processor 1202 can be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), a graphics processing unit (GPU), or one or more integrated circuits, and is used to execute relevant programs to implement the functions required to be executed by the units in the semantic matching device of any embodiment of the present application, or to execute the semantic matching method of any embodiment of the present application.

[0272] The processor 1202 can also be an integrated circuit chip with the ability to process signals. In the implementation process, each step of the semantic matching method according to any embodiment of the present application can be completed by the integrated logic circuit of the hardware in the processor 1202 or the instructions in the form of software. The above-mentioned processor 1202 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the semantic matching method, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the semantic matching method according to any embodiment of the present application can be directly embodied as being executed by the hardware processor, or executed by the combination of the hardware and software modules in the processor. The software module can be located in a mature storage medium in the art such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers, etc. This storage medium is located in the memory 1201, and the processor 1202 reads the information in the memory 1201 and combines its hardware to complete the functions required to be executed by the units included in the semantic matching device according to any embodiment of the present application, or executes the semantic matching method according to any embodiment of the present application.

[0273] The communication interface 1204 uses a transceiver device such as, but not limited to, a transceiver to implement the communication between the semantic matching device 1200 and other devices or communication networks. For example, the first query text can be obtained through the communication interface 1204.

[0274] The bus 1203 can include a path for transmitting information between various components of the semantic matching device 1200 (for example, the memory 1201, the processor 1202, the communication interface 1204). In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be in an electrical, mechanical, or other form.

[0275] Reference Figure 13 , Figure 13It is a schematic structural diagram of a semantic matching model training device provided by an embodiment of the present application; the semantic matching model training device 1300 includes a memory 1301, a processor 1302, a communication interface 1304, and a bus 1303. Among them, the memory 1301, the processor 1302, and the communication interface 1304 are communicatively connected to each other through the bus 1303.

[0276] Optionally, the above-mentioned semantic matching model training device 1300 further includes a display screen (not shown), and the display screen is communicatively connected to the memory 1301, the processor 1302, and the communication interface 1304 through the bus 1303. The display screen is used to output information for interaction with the user. Optionally, the above-mentioned semantic matching model training device 1300 further includes an output module (not shown), and the output module is communicatively connected to the memory 1301, the processor 1302, and the communication interface 1304 through the bus 1303. The output module is used for information output, such as voice output or display output.

[0277] The memory 1301 can be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1301 can store a program. When the program stored in the memory 1301 is executed by the processor 1302, the processor 1302 and the communication interface 1304 are used to execute the steps of the semantic matching model training method of any embodiment of the present application.

[0278] The processor 1302 can be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), a graphics processing unit (GPU), or one or more integrated circuits, and is used to execute relevant programs to implement the functions required to be executed by the units in the semantic matching model training device of any embodiment of the present application, or to execute the semantic matching model training method of any embodiment of the present application.

[0279] The processor 1302 can also be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the semantic matching model training method according to any embodiment of the present application can be completed by the integrated logic circuit of the hardware in the processor 1302 or the instructions in the form of software. The above-mentioned processor 1302 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the semantic matching model training method, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. Combining the steps of the semantic matching model training method according to any embodiment of the present application can be directly embodied as being executed by the hardware processor, or completed by the combination of the hardware and software modules in the processor. The software module can be located in a mature storage medium in the art such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers, etc. This storage medium is located in the memory 1301, and the processor 1302 reads the information in the memory 1301 and combines its hardware to complete the functions required to be executed by the units included in the semantic matching model training device according to any embodiment of the present application, or execute the semantic matching model training method according to any embodiment of the present application.

[0280] The communication interface 1304 uses a transceiver device such as, but not limited to, a transceiver to implement the communication between the semantic matching model training device 1300 and other devices or communication networks. For example, the first training text and the like can be obtained through the communication interface 1304.

[0281] The bus 1303 can include a path for transmitting information between various components of the semantic matching model training device 1300 (for example, the memory 1301, the processor 1302, and the communication interface 1304). In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be in electrical, mechanical, or other forms.

[0282] The unit described as a separation component may or may not be physically separated. The component displayed as a unit may or may not be a physical unit, that is, it may be located in one place or distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0283] In addition, in each embodiment of the present application, each functional unit may be integrated in a processing unit, may exist separately as individual physical units, or two or more units may be integrated in one unit.

[0284] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that the computer can access or a data storage device such as a server or data center that includes one or more integrated available media. The available medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic medium, such as a floppy disk, a hard disk, a magnetic tape, a magnetic disk, or an optical medium, such as a digital versatile disc (DVD), or a semiconductor medium, such as a solid state disk (SSD), etc.

[0285] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, and all of them should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A semantic matching method, characterized in that, Comprising: Obtaining a first query text input by a user; Obtaining at least one first candidate text corresponding to the first query text; Based on a first similarity threshold and based on the first similarity between each first candidate text in the at least one first candidate text and the first query text, determining a matching result between the first query text and the at least one first candidate text; Wherein, the first similarity threshold is related to the first query text and a first vector, and the first vector is used to distinguish a positive example candidate text and a negative example candidate text in the at least one first candidate text.

2. The method according to claim 1, characterized in that, When the first similarity of a second candidate text is greater than or equal to the first similarity threshold, the second candidate text is a matching result of the first query text, and the at least one first candidate text includes the second candidate text.

3. The method according to claim 1 or 2, characterized in that, The method further comprises: Performing vectorization processing on the first query text by using a first semantic matching model to determine a first query vector; Performing vectorization processing on each first candidate text in the at least one first candidate text by using the first semantic matching model to determine at least one first candidate vector; Performing vectorization processing on the first vector by using the first semantic matching model to determine a second vector; Using the first semantic matching model to determine the first similarity based on the first query vector and the first candidate vector; Using the first semantic matching model to determine the first similarity threshold based on the first query vector and the second vector.

4. The method according to claim 3, characterized in that, When the first query text is a text without previous context, the method further comprises: Obtaining a second query text input by a user and a first previous context text, where the first previous context text is the previous context text of the second query text; the first previous context text includes the first query text; Obtaining at least one third candidate text corresponding to the second query text; Based on a second similarity threshold and based on the second similarity between each third candidate text in the at least one third candidate text and the second query text, determining a matching result between the second query text and the at least one third candidate text; Wherein, the second similarity of the third candidate text is obtained based on a third vector and a second candidate vector; The third vector is obtained by fusing a first previous context vector and a second query vector; The first previous context vector is obtained by performing vectorization processing on the first previous context text; The second query vector is obtained by performing vectorization processing on the second query text by using the first semantic matching model; The second candidate vector is obtained by performing vectorization processing on the third candidate text by using the first semantic matching model; The second similarity threshold is obtained based on the third vector and the second vector.

5. The method according to claim 4, characterized in that, When the second similarity of a fourth candidate text is greater than or equal to the second similarity threshold, the fourth candidate text is a matching result of the second query text, and the above at least one third candidate text includes the fourth candidate text.

6. A semantic matching method, characterized in that, Comprising: Obtaining a third query text input by a user and a second previous context text, where the second previous context text is the previous context text of the third query text; Obtain at least one fifth candidate text corresponding to the third query text; Based on a third similarity threshold and based on the third similarity between each fifth candidate text in the at least one fifth candidate text and the third query text, determine the matching result between the third query text and the at least one fifth candidate text; Wherein, the third similarity of the fifth candidate text is obtained based on a fourth vector and a third candidate vector; The fourth vector is obtained by fusing a second context vector and a third query vector; The second context vector is obtained by vectorizing the second context text; The third query vector is obtained by vectorizing the third query text using a second semantic matching model, and the second semantic matching model is used for semantic matching of query texts without context texts; The third candidate vector is obtained by vectorizing the fifth candidate text using the second semantic matching model.

7. The method according to claim 6, characterized in that, When the third similarity of the sixth candidate text is greater than or equal to the third similarity threshold, the sixth candidate text is the matching result of the third query text, and the at least one fifth candidate text includes the sixth candidate text.

8. A method for training a semantic matching model, characterized in that, The method includes: Obtain a first training set, where the first training set includes at least two first training texts and the first positive example text and the first negative example text of each first training text; Train a third semantic matching model according to the first training set and a first loss function to obtain a first semantic matching model; the first loss function satisfies: The fourth similarity is greater than the fifth similarity, and the fifth similarity is greater than the sixth similarity; Wherein, the fourth similarity is the similarity between the first training text and the first positive example text, The fifth similarity is the similarity between the first training text and a first vector, The sixth similarity is the similarity between the first training text and the first negative example text.

9. The method according to claim 8, characterized in that, The first training text is a text without context; the method further includes: Obtain a second training set, where the second training set includes at least two second training texts, the context text of each second training text, and the second positive example text and the second negative example text of each second training text; Train a first vectorization module in a fourth semantic matching model according to the second training set and the second loss function to obtain a fifth semantic matching model; Wherein, the fourth semantic matching model further includes the first semantic matching model; the second loss function satisfies: The seventh similarity is greater than the eighth similarity, and the eighth similarity is greater than the ninth similarity; The seventh similarity is the similarity between the fifth vector and the first positive example vector. The fifth vector is obtained by fusing the second training vector and the third context vector. The second training vector is obtained by vectorizing the second training text using the first semantic matching model. The third context vector is obtained by processing the context text of the second training text using the first vectorization module. The first positive example vector is obtained by vectorizing the second positive example text of the second training text using the first semantic matching model; The eighth similarity is the similarity between the fifth vector and the second vector. The second vector is obtained by vectorizing the first vector using the first semantic matching model; The ninth similarity is the similarity between the fifth vector and the first negative example vector. The first negative example vector is obtained by vectorizing the second negative example text of the second training text using the first semantic matching model.

10. A method for training a semantic matching model, characterized in that, The method includes: Obtaining a second training set, where the second training set includes at least two second training texts, the context text of each second training text, and the second positive example text and the second negative example text of each second training text; Training the second vectorization module in the sixth semantic matching model according to the second training set and the third loss function to obtain a seventh semantic matching model; Among them, the sixth semantic matching model further includes a second semantic matching model, and the second semantic matching model is used for semantic matching of query texts without context text; The third loss function satisfies: the seventh similarity is greater than the ninth similarity; The seventh similarity is the similarity between the fifth vector and the first positive example vector. The fifth vector is obtained by fusing the second training vector and the third context vector. The second training vector is obtained by vectorizing the second training text using the second semantic matching model. The third context vector is obtained by processing the context text of the second training text using the second vectorization module. The first positive example vector is obtained by vectorizing the second positive example text of the second training text using the second semantic matching model; The ninth similarity is the similarity between the fifth vector and the first negative example vector. The first negative example vector is obtained by vectorizing the second negative example text of the second training text using the second semantic matching model.

11. A semantic matching device, characterized in that, The device includes units or modules for executing the semantic matching method according to any one of claims 1 to 7.

12. A semantic matching model training device, characterized in that, The device includes units or modules for executing the semantic matching model training method according to any one of claims 8 to 10.

13. A semantic matching device, characterized in that, Including a processor and a memory. Among them, the processor and the memory are connected. The memory is used for storing program code, and the processor is used for calling the program code to execute the semantic matching method according to any one of claims 1 to 7.

14. A semantic matching model training device, characterized in that, It includes a processor and a memory. Among them, the processor is connected to the memory. The memory is used to store program codes, and the processor is used to call the program codes to execute the semantic matching model training method according to any one of claims 8 to 10.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 10.

Images