Training method, device, equipment and storage medium for similar text matching model

Abstract

The embodiments of the present application disclose a training, device, equipment and storage medium for a similar text matching model. The relevant embodiments can be applied to various scenarios such as cloud technology, artificial intelligence and intelligent transportation, and are used to improve the recall rate of similar texts. The method includes: obtaining a first batch of sample sets corresponding to the target scenario, and inputting them into the original similar text matching model for vector conversion operation to obtain a first batch of positive example sentence vectors and a first batch of negative example sentence vectors, performing a triple construction operation on the first batch of positive example sentence vectors to obtain a plurality of first batch triples, and performing a loss calculation operation to obtain a first batch loss function, and performing a parameter adjustment operation on the original similar text matching model to obtain an intermediate similar text matching model, repeatedly obtaining a second batch of sample sets corresponding to the target scenario, and performing a vector conversion operation, a triple construction operation, a loss calculation operation and a parameter adjustment operation to obtain a target similar text matching model.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing technology, and in particular to a training method, apparatus, device and storage medium for a similar text matching model. Background Art

[0002] With the development of science and technology, the application of retrieving similar texts is becoming more and more widespread. When users use search engines to search for information, there is often a discrepancy between the concepts of the same word entered and the search results obtained. In order to better retrieve the content required by users from massive data quickly and accurately, and provide or push it to users.

[0003] Among them, the retrieval of similar texts is usually implemented based on a deep learning text similarity model. However, the deep learning text similarity model is a supervised model, which requires a large number of manually annotated training samples as supervision signals to make the model better fit and generalize. However, the text is semantically rich and the standards of text similarity among different annotators vary greatly and are difficult to unify. Therefore, it will bring great difficulties to the manual data annotation training samples. The training, optimization, and iteration of the model require huge costs, resulting in poor fitting and generalization effects of the model, thereby reducing the recall rate of the model. Summary of the invention

[0004] The embodiments of the present application provide a training method, apparatus, device and storage medium for a similar text matching model, which is used to shorten the distance between positive example samples and similar samples and increase the distance between positive example samples and heterogeneous samples through a triplet loss function, so that similar text vectors can form clusters in the feature space, improve the learning ability of the similar text matching model for the similarity between text vectors, enable the similar text matching model to fit better, and thus improve the recall rate of the target similar text matching model for similar texts.

[0005] On the one hand, an embodiment of the present application provides a training method for a similar text matching model, including:

[0006] Acquire a first batch of sample sets corresponding to the target scene, wherein the first batch of sample sets includes a first batch of positive samples and a first batch of negative samples;

[0007] The first batch of positive sample and the first batch of negative sample are respectively input into the original similar text matching model for vector conversion operation to obtain the first batch of positive sentence vector and the first batch of negative sentence vector;

[0008] Performing a triplet construction operation on the first batch of positive example sentence vectors to obtain a plurality of first batch triplets, wherein each first batch triplet includes a first batch of positive example sentence vectors, a first batch of similar sentence vectors, and a first batch of different sentence vectors, and the first batch of similar sentence vectors and the first batch of different sentence vectors are derived from the first batch of negative example sentence vectors;

[0009] Perform loss calculation operations on several first batch triples to obtain the first batch loss function corresponding to the first batch sample set;

[0010] According to the first batch loss function, the parameters of the original similar text matching model are adjusted to obtain an intermediate similar text matching model;

[0011] Based on the intermediate similar text matching model, the second batch of sample sets corresponding to the target scene are repeatedly obtained, and vector conversion operations, triple construction operations, loss calculation operations and parameter adjustment operations are performed to obtain the target similar text matching model.

[0012] On the other hand, the present application provides a training device for a similar text matching model, comprising:

[0013] An acquisition unit, used for acquiring a first batch of sample sets corresponding to the target scene, wherein the first batch of sample sets includes a first batch of positive samples and a first batch of negative samples;

[0014] A processing unit, used to input the first batch of positive example samples and the first batch of negative example samples into the original similar text matching model for vector conversion operation, so as to obtain the first batch of positive example sentence vectors and the first batch of negative example sentence vectors;

[0015] The processing unit is further used to perform a triple construction operation on the first batch of positive example sentence vectors to obtain a plurality of first batch triples, wherein each first batch triple includes a first batch of positive example sentence vectors, a first batch of similar sentence vectors, and a first batch of different sentence vectors, and the first batch of similar sentence vectors and the first batch of different sentence vectors are derived from the first batch of negative example sentence vectors;

[0016] The processing unit is further used to perform loss calculation operations on a plurality of first batch triples to obtain a first batch loss function corresponding to the first batch sample set;

[0017] The processing unit is further used to perform parameter adjustment operations on the original similar text matching model according to the first batch loss function to obtain an intermediate similar text matching model;

[0018] The processing unit is also used to repeatedly obtain a second batch of sample sets corresponding to the target scene based on the intermediate similar text matching model, and perform vector conversion operations, triple construction operations, loss calculation operations and parameter adjustment operations to obtain a target similar text matching model.

[0019] In a possible design, in an implementation of another aspect of the embodiment of the present application, the acquisition unit may be specifically used to:

[0020] Acquire a target text data set corresponding to the target scene, wherein the target text data set includes at least a first batch of positive example samples and source text data corresponding to the target scene;

[0021] Retrieving N first matching texts corresponding to the first batch of positive example samples from the target text dataset as N first batch of negative example samples, where N is an integer greater than 1;

[0022] Calculate the matching scores between the first batch of positive samples and each of the first batch of negative samples to obtain N first matching scores;

[0023] Normalizing the N first matching scores respectively to obtain N sample matching scores;

[0024] The first batch of sample sets is constructed according to the first batch of positive samples, the first batch of negative samples and the sample matching scores.

[0025] In a possible design, in an implementation of another aspect of the embodiment of the present application, the processing unit may be specifically used for:

[0026] According to the sample matching scores, the first batch of negative example sentence vectors are divided into a similar sentence vector set and a heterogeneous sentence vector set.

[0027] Extract any similar sentence vector from the similar sentence vector set to obtain the first batch of similar sentence vectors;

[0028] Extract any heterogeneous sentence vector from the heterogeneous sentence vector set to obtain the first batch of heterogeneous sentence vectors.

[0029] In a possible design, in an implementation of another aspect of the embodiment of the present application, the processing unit may be specifically used for:

[0030] Perform loss calculation operations on the first batch of positive example sentence vectors, the first batch of similar sentence vectors, and the first batch of heterogeneous sentence vectors, respectively, to obtain loss functions corresponding to several first batch triples;

[0031] The loss functions corresponding to several first batch triplets are weightedly calculated to obtain the first batch loss function.

[0032] In a possible design, in an implementation of another aspect of the embodiment of the present application, the processing unit may be specifically used for:

[0033] Obtain a second batch of sample sets corresponding to the target scene, and perform a vector conversion operation, a triplet construction operation, and a loss calculation operation according to the second batch of sample sets to obtain a second loss function;

[0034] If the second loss function is less than the first threshold, the current intermediate similar text matching model is used as the target similar text matching model.

[0035] In a possible design, in an implementation of another aspect of the embodiment of the present application, the processing unit may be specifically used for:

[0036] Obtaining intermediate model parameters of the intermediate similar text matching model;

[0037] After obtaining a second batch of sample sets corresponding to the target scene and performing a vector conversion operation, a triplet construction operation, and the above parameter adjustment operation, a current similar text matching model is obtained, and the current similar text matching model includes current model parameters;

[0038] If the difference between the intermediate model parameters and the current model parameters meets the second threshold, the current intermediate similar text matching model is used as the target similar text matching model.

[0039] In a possible design, in an implementation of another aspect of the embodiment of the present application,

[0040] The acquisition unit is also used to receive the text to be matched;

[0041] The processing unit is also used to pass the to-be-matched text and the target text data set through the target similar text matching model to obtain the to-be-matched sentence vector and a plurality of original sentence vectors;

[0042] The processing unit is also used to calculate the similarity between the sentence vector to be matched and each original sentence vector to obtain a number of similarity scores;

[0043] The determination unit is used to determine the target similar text according to a plurality of similarity scores, and push the target similar text to the target terminal device.

[0044] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0045] By obtaining the first batch of sample sets corresponding to the target scene, the first batch of positive samples and the first batch of negative samples in the first batch of sample sets are respectively input into the original similar text matching model for vector conversion operation to obtain the first batch of positive sentence vectors and the first batch of negative sentence vectors, and the first batch of positive sentence vectors are subjected to triple construction operation to obtain several first batch triplets, and then the loss calculation operation can be performed on the several first batch triplets to obtain the first batch loss function corresponding to the first batch of sample sets, and according to the first batch loss function, the original similar text matching model is subjected to parameter adjustment operation to obtain the intermediate similar text matching model, and then, based on the intermediate similar text matching model, the second batch of sample sets corresponding to the target scene can be repeatedly obtained, and the vector conversion operation, the triple construction operation, the loss calculation operation and the parameter adjustment operation can be performed to obtain the target similar text matching model. Through the above method, triplets can be constructed through the first batch of positive example sentence vectors and the first batch of negative example sentence vectors to obtain the triplet loss function. The triplet loss function can be used to shorten the distance between the positive example samples and the same type of samples, and to increase the distance between the positive example samples and the different types of samples, so that similar text vectors can form clusters in the feature space, thereby improving the learning ability of the similar text matching model for the similarity between text vectors, and enabling the similar text matching model to fit better, thereby improving the recall rate of the target similar text matching model for similar texts. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a schematic diagram of the structure of the text data control system in the embodiment of the present application;

[0047] Figure 2 is a flow chart of an embodiment of a training method for a similar text matching model in an embodiment of the present application;

[0048] Figure 3 It is a schematic diagram of a retrieval principle of a training method for a similar text matching model in an embodiment of the present application;

[0049] Figure 4 It is another schematic diagram of the retrieval principle of the training method of the similar text matching model in the embodiment of the present application;

[0050] Figure 5 It is a schematic diagram of constructing a sample set of a training method for a similar text matching model in an embodiment of the present application;

[0051] Figure 6 It is a schematic diagram of a model training process of the training method of the similar text matching model in the embodiment of the present application;

[0052] Figure 7 It is a schematic diagram of an embodiment of a training device for a similar text matching model in an embodiment of the present application;

[0053] Figure 8 It is a schematic diagram of an embodiment of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0054] The embodiments of the present application provide a training method, apparatus, device and storage medium for a similar text matching model, which is used to shorten the distance between positive example samples and similar samples and increase the distance between positive example samples and heterogeneous samples through a triplet loss function, so that similar text vectors can form clusters in the feature space, improve the learning ability of the similar text matching model for the similarity between text vectors, enable the similar text matching model to fit better, and thus improve the recall rate of the target similar text matching model for similar texts.

[0055] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "corresponding to" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0056] With the rapid development of information, cloud technology has gradually entered all aspects of people's lives. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool, which is used on demand and is flexible and convenient. Cloud computing technology will become an important support. The background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the rapid development and application of the Internet industry, each item may have its own identification mark in the future, and all need to be transmitted to the background system for logical processing. Data of different levels will be processed separately. All kinds of industry data require strong system backing support, which can only be achieved through cloud computing.

[0057] Among them, cloud security refers to the general term for security software, hardware, users, organizations, and security cloud platforms based on cloud computing business model applications. Cloud security integrates emerging technologies and concepts such as parallel processing, grid computing, and unknown virus behavior judgment. Through a large number of meshed clients, it monitors the abnormal behavior of software in the network, obtains the latest information of Trojans and malicious programs on the Internet, and sends it to the server for automatic analysis and processing, and then distributes virus and Trojan solutions to each client. The training and testing method of the similar text matching model provided in the embodiment of the present application can be implemented through cloud computing technology and cloud security technology.

[0058] It should be understood that the training and testing method of the similar text matching model provided in the present application can be applied to cloud technology, artificial intelligence, smart transportation and other fields, and can be used in scenarios where similar texts are pushed or delivered to target objects through matching. As an example, by matching similar texts for advertising texts, more matching advertisements can be recommended to target objects; as another example, by matching similar texts for product texts, more matching products can be recommended to target objects; for example, by matching similar texts for book texts, more matching books or documents can be recommended to target objects. In the above scenarios, in order to achieve matching of similar texts, a text similarity model is usually implemented based on deep learning, but a large number of manually annotated training samples are required to make the model fit and generalize better. However, the text is semantically rich and the standards of text similarity for different annotators vary greatly and are difficult to unify. Therefore, it will bring great difficulties to the manual data annotation training samples, and the training, optimization, and iteration of the model will cost huge costs, resulting in poor fitting and generalization effects of the model, thereby reducing the recall rate of the model.

[0059] In order to solve the above problems, this application proposes a training and testing method for a similar text matching model. Figure 1 Text data control system shown, see Figure 1 , Figure 1 Schematic diagram of a text data control system in an embodiment of the present application. Figure 1As shown, the server obtains the first batch of sample sets corresponding to the target scene, and inputs the first batch of positive samples and the first batch of negative samples in the first batch of sample sets into the original similar text matching model for vector conversion operation to obtain the first batch of positive sentence vectors and the first batch of negative sentence vectors, and performs a triple construction operation on the first batch of positive sentence vectors to obtain several first batch triplets, and then performs a loss calculation operation on the several first batch triplets to obtain the first batch loss function corresponding to the first batch sample sets, and according to the first batch loss function, performs a parameter adjustment operation on the original similar text matching model to obtain an intermediate similar text matching model, and then, based on the intermediate similar text matching model, repeatedly obtains the second batch of sample sets corresponding to the target scene, and performs vector conversion operation, triple construction operation, loss calculation operation and parameter adjustment operation to obtain the target similar text matching model. Through the above method, triplets can be constructed through the first batch of positive example sentence vectors and the first batch of negative example sentence vectors to obtain the triplet loss function. The triplet loss function can be used to shorten the distance between the positive example samples and the same type of samples, and to increase the distance between the positive example samples and the different types of samples, so that similar text vectors can form clusters in the feature space, thereby improving the learning ability of the similar text matching model for the similarity between text vectors, and enabling the similar text matching model to fit better, thereby improving the recall rate of the target similar text matching model for similar texts.

[0060] Understandably, Figure 1 Only one type of terminal device is shown in the figure. In actual scenarios, more types of terminal devices may participate in the data processing process. Terminal devices include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, car terminals, etc. The specific number and type depend on the actual scenario and are not limited here. Figure 1 One server is shown in the figure, but in actual scenarios, multiple servers may also be involved, especially in the scenario of multi-model training interaction. The number of servers depends on the actual scenario and is not limited here.

[0061] It should be noted that in this embodiment, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The terminal device and the server can be directly or indirectly connected through wired or wireless communication, and the terminal device and the server can be connected to form a blockchain network, which is not limited in this application.

[0062] In order to solve the above problems, the present application proposes a training method for a similar text matching model, which is generally executed by a server or a terminal device. Accordingly, a training device applied to the similar text matching model is generally set in the server or the terminal device.

[0063] It is understandable that, in the training method, device, equipment and storage medium of the similar text matching model disclosed in this application, multiple servers or terminal devices can form a blockchain, and the server or terminal device is a node on the blockchain. In practical applications, data sharing between nodes may be required in the blockchain, and each node may store text data, etc.

[0064] The following is an introduction to the training method of the similar text matching model in this application. Figure 2 as well as Figure 6 In one embodiment of the present application, a training method for a similar text matching model includes:

[0065] In step S101, a first batch of sample sets corresponding to the target scene is obtained, wherein the first batch of sample sets includes a first batch of positive samples and a first batch of negative samples;

[0066] In this embodiment, before obtaining a matching request or a retrieval request sent by a target terminal device, such as before obtaining a text to be matched or a text to be retrieved, this embodiment can obtain a first batch of sample sets corresponding to the target scene, so that the original similar text matching model can be subsequently trained through the first batch of positive samples and the first batch of negative samples in the first batch of sample sets to optimize the original similar text matching model.

[0067] Specifically, before obtaining the text to be matched or the text to be retrieved, based on the target scene stored in the database, the target scene can be specifically expressed as an advertising scene, a news information scene, a book management scene, etc., or other target scenes, which are not specifically limited here. Then, a first batch of sample sets corresponding to the target scene can be obtained, wherein the first batch of sample sets can be a batch of sample data randomly selected from the sample set, including a positive sample set and a negative sample set, the first positive sample set is an advertising text in a known advertising scene, and the first negative sample set is text data matched with the first positive sample through ES, wherein, as shown in Table 1, one positive sample can correspond to one or more negative samples.

[0068] Table 1

[0069]

[0070] For example, a first batch sample set may be 128-dimensional sample data, that is, it includes 12 sample data, including 10 positive samples, namely, the first batch of positive samples, and 118 text data matched with the 10 positive samples by ES, namely, the first batch of negative samples.

[0071] In step S102, the first batch of positive sample and the first batch of negative sample are respectively input into the original similar text matching model for vector conversion operation to obtain the first batch of positive sentence vector and the first batch of negative sentence vector;

[0072] In this embodiment, after obtaining the first batch of positive samples and the first batch of negative samples, the first batch of positive samples and the first batch of negative samples can be respectively input into several original similar text matching models for vector conversion to obtain the first batch of positive sentence vectors and the first batch of negative sentence vectors, so that the distance between each positive sample and the negative sample can be better calculated through the first batch of positive sentence vectors and the first batch of negative sentence vectors, so that the similarity between the positive sample and the negative sample can be better characterized through the distance between the positive sample and the negative sample.

[0073] Specifically, Figure 6 As shown in , the original similar text matching model can be specifically represented by a combination of the Bert model and several fully connected layers and pooling layers. Other text processing models can also be used, which are not specifically limited here. After obtaining the first batch of positive samples and the first batch of negative samples, as shown in Figure 6 As shown, the first batch of positive samples and the first batch of negative samples can be respectively input into several original similar text matching models for vector conversion. For example, the first batch of positive samples and the first batch of negative samples are first compiled through the Bert model to obtain at least two word vectors corresponding to the first batch of positive samples and at least two word vectors corresponding to the first batch of negative samples. Then, the at least two word vectors corresponding to the first batch of positive samples and the at least two word vectors corresponding to the first batch of negative samples are respectively passed through several fully connected layers and pooling layers to obtain the first batch of positive sentence vectors and the first batch of negative sentence vectors.

[0074] In step S103, a triple construction operation is performed on the first batch of positive example sentence vectors to obtain a plurality of first batch triples, wherein each first batch triple includes a first batch of positive example sentence vectors, a first batch of similar sentence vectors, and a first batch of different sentence vectors, and the first batch of similar sentence vectors and the first batch of different sentence vectors are derived from the first batch of negative example sentence vectors;

[0075] In this embodiment, after obtaining the first batch of positive example sentence vectors and the first batch of negative example sentence vectors, for each first batch of positive example sentence vectors, a first batch of positive example sentence vectors can be randomly combined with a first batch of similar sentence vectors and a first batch of heterogeneous sentence vectors to form a triple, wherein a first batch of positive example sentence vectors can correspond to one or more triplets, so several first batch triplets can be obtained.

[0076] Among them, the first batch of similar sentence vectors can be understood as negative example samples with high similarity to the first batch of positive example sentence vectors, which can be specifically expressed as sentence vectors corresponding to negative example samples with matching scores greater than 0.5; the first batch of heterogeneous sentence vectors can be understood as negative example samples with low similarity to the first batch of positive example sentence vectors, which can be specifically expressed as sentence vectors corresponding to negative example samples with matching scores less than 0.5.

[0077] Specifically, after obtaining the first batch of positive example sentence vectors and the first batch of negative example sentence vectors, for example, 10 first batch positive example samples and 118 first batch negative example samples are obtained. Assuming that for a first batch positive example sample, there are 3 first batch negative example samples matched with the first batch positive example sample through ES, among which, the 3 first batch negative example samples correspond to 2 first batch similar sentence vectors and 1 first batch heterogeneous sentence vector. Randomly extract a first batch positive example sentence vector and a first batch similar sentence vector and a first batch heterogeneous sentence vector to form a triple, and 2 triplets corresponding to the first batch positive example sample can be obtained.

[0078] In step S104, a loss calculation operation is performed on a number of first batch triples to obtain a first batch loss function corresponding to the first batch sample set;

[0079] In this embodiment, after obtaining several first batch triplets, the loss function can be calculated for each triplet, and then the several obtained loss functions can be integrated into one loss function, that is, the first batch loss function corresponding to the first batch sample set. The triplet loss function can be constructed based on the triplet, which can shorten the distance between the positive example sample and the same type of sample, and increase the distance between the positive example sample and the different type of sample, so that similar text vectors can form clusters in the feature space, thereby achieving the purpose of text matching.

[0080] Furthermore, the similarity between text sentences can be regressed through the triplet loss function, so that the embedding vector (Embedding) obtained after learning the original similar text model, that is, the similarity between each sentence vector, can be as close as possible to the normalized matching score.

[0081] Among them, the first batch loss function can be expressed as a triple loss function, which can be shown as follows:

[0082]

[0083] Among them, L is the first batch loss function, ES(a,p) represents the normalized matching score between the positive sample a corresponding to the positive sentence vector and the negative sample p corresponding to the same type of sentence vector, and d(p,n) represents the cosine similarity between the same type of negative sentence vector corresponding to the negative sample p and the different type of sentence vector corresponding to the negative sample n, that is, d(a,p)=1-cosine(a,p).

[0084] In step S105, according to the first batch loss function, the original similar text matching model is adjusted in parameters to obtain an intermediate similar text matching model;

[0085] Specifically, after obtaining the first batch of loss functions, the parameters of the original similar text matching model can be adjusted. Specifically, the reverse gradient descent algorithm can be used to update the model parameters in BERT until convergence, and an intermediate similar text matching model can be obtained.

[0086] In step S106, based on the intermediate similar text matching model, a second batch of sample sets corresponding to the target scene are repeatedly obtained, and vector conversion operations, triple construction operations, loss calculation operations, and parameter adjustment operations are performed to obtain a target similar text matching model.

[0087] In this embodiment, after obtaining the intermediate similar text matching model, the second batch of sample sets corresponding to the target scene can be repeatedly obtained, and based on the obtained second batch of sample sets, the vector conversion operation, triple construction operation, loss calculation operation and parameter adjustment operation similar to steps S102 to S105 are repeatedly performed until the model parameters of the intermediate similar text matching model tend to be stable. The intermediate similar text matching model can be used as the target similar text matching model. It can be understood that the second batch of sample sets corresponding to the target scene can be a batch of sample data randomly selected from the sample set, including a positive sample set and a negative sample set.

[0088] In an embodiment of the present application, a training method for a similar text matching model is provided. Through the above method, a triplet loss function can be obtained by constructing a triplet through the first batch of positive example sentence vectors and the first batch of negative example sentence vectors. The triplet loss function can be used to shorten the distance between the positive example sample and the same type of sample, and to increase the distance between the positive example sample and the different type of sample, so that similar text vectors can form clusters in the feature space, thereby improving the learning ability of the similar text matching model for the similarity between text vectors, so that the similar text matching model can be better fitted, thereby improving the recall rate of the target similar text matching model for similar texts.

[0089] Optionally, in the above Figure 2 On the basis of the corresponding embodiment, in another optional embodiment of the training method of the similar text matching model provided in the embodiment of the present application, obtaining a first batch of sample sets corresponding to the target scene includes:

[0090] Acquire a target text data set corresponding to the target scene, wherein the target text data set includes at least a first batch of positive example samples and source text data corresponding to the target scene;

[0091] Retrieving N first matching texts corresponding to the first batch of positive example samples from the target text dataset as N first batch of negative example samples, where N is an integer greater than 1;

[0092] Calculate the matching scores between the first batch of positive samples and each of the first batch of negative samples to obtain N first matching scores;

[0093] Normalizing the N first matching scores respectively to obtain N sample matching scores;

[0094] The first batch of sample sets are constructed according to the first batch of positive samples, the first batch of negative samples and the sample matching scores.

[0095] In this embodiment, if Figure 5 As shown, before model training, the corresponding target text data set can be obtained according to the target scenario, and N first matching texts corresponding to the first batch of positive samples can be retrieved from the target text data set to obtain the first batch of negative samples. Then, the N first matching scores can be normalized respectively to obtain N sample matching scores to obtain the first batch of sample sets corresponding to the target scenario. The target text data set can be constructed through a search engine (ElasticSearch, ES), wherein ES is a text retrieval engine that supports efficient and multiple scoring strategies, and self-supervised training samples are obtained through the target text data set, such as the first batch of sample sets. There is no need to spend a lot of time formulating text similarity standards or performing tedious manual annotations. The target text data set can be replaced and adjusted according to different target scenarios and needs, so that a better and more accurate sample set suitable for the target scenario can be obtained, making the construction of the sample set more flexible and more extensible.

[0096] Specifically, before obtaining the target text data set corresponding to the target scene, a retrieval library corresponding to the target scene can be established through a search engine, that is, the source text data corresponding to the target scene can be obtained, wherein the source text data can be specifically expressed as advertising copy or description or product copy in an advertising scene, etc., and can also be expressed as other text data, which is not specifically limited here. Obtaining the source text data can specifically be obtaining the initial text data corresponding to the target scene through a search engine. Furthermore, since the initial text data such as advertising copy may have many similar texts, for example, only punctuation marks are different, and only individual characters are different, therefore, in order to increase the diversity of text data as much as possible, the obtained initial text can be The data is deduplicated, for example, the edit distance between each two texts is calculated or the length of the longest text between each two texts is counted. If the edit distance or length difference between each two texts is less than the preset distance threshold, the text with shorter length can be filtered as a surprise sample to obtain the source text data. Then, when a source text data is stored by ES, ES can use a tokenizer to extract word units (tokens) from the source text data to support index storage and retrieval. Among them, the word units can be extracted using many built-in tokenizers in ES, or the selected (IKAnalyzer, IK) tokenizer can be used as a token segmentation tool for each source text data.

[0097] Furthermore, if Figure 5 As shown, after obtaining the source text data, you can obtain positive example samples (query) collected according to the target scenario, and form the target text data with the obtained source text data. For example, assuming that six thousand positive example samples are collected in an advertising scenario, they can be combined with two million source text data in the advertising scenario to form two million six hundred target text data. Then, when a target text data is stored by ES, ES can use a word segmenter to extract word units from the target text data to establish an index for the target text data.

[0098] Further, if Figure 5 As shown, each positive example can be searched in the target text data. For example, 6,000 positive examples are searched in 2,600,000 target text data. The search can be performed according to the ES matching, which can be based on the bag-of-words model. Keyword matching, or sentence matching that generates sentence vectors from word vectors, etc., one or more matching methods, for example, Figure 3As shown in the figure, assuming that a positive example is "Beijing Tiananmen", a group of word units such as "Beijing" and "Tiananmen" can be obtained after passing through the word segmenter. If there is a source text data "I love Beijing Tiananmen", after word segmentation, "I", "love", "Beijing" and "Tiananmen" are obtained. Based on the keyword matching of the bag-of-words model, that is, "Beijing" and "Tiananmen", the correlation score between the positive example sample and the source text data can be calculated according to the hit word units, that is, the matching score. Similarly, the topN text data similar to each positive example in the target text data, that is, the N first matching texts, and the matching score corresponding to each first matching text can be returned, and the matching scores are arranged from high to low.

[0099] For example, Figure 4 As shown in the figure, suppose there is a positive sample "'Electric drill' screwdriver is popular in the countryside, 250 revolutions per minute, easy to drill holes! 47 screwdriver bits, suitable for 500 kinds of screws! 10% off!", which can be matched with the target text data containing the positive sample in ES. If the text matched by the positive sample is also the positive sample, the maximum matching score such as "133.11464" can be obtained. Alternatively, the positive sample can be matched with a source text data such as "'Electric drill' screwdriver is popular in the countryside! 2500 revolutions per minute, drilling in as fast as 3 seconds! 47 screwdriver bits, suitable for 500 kinds of screws! 10% off!", then To get a matching score such as "105.32658", or, the positive example sample can be matched to another source text data such as "This screwdriver is popular! 250 revolutions per minute, easy to drill holes! 47 screwdriver bits, can screw 500 kinds of screws! Good to use, not expensive, only 10% off!", then a matching score such as "102.89305" can be obtained, or, the positive example sample can be matched to another source text data such as "'This electric screwdriver is so powerful, 250 revolutions per minute, easy to drill holes, 47 screwdriver bits, can screw 500 kinds of screws! Now 10% off", then a matching score such as "99.31" can be obtained.

[0100] Furthermore, after obtaining N first matching scores, since there is no clear range for the matching scores, for example, the highest matching score can vary from dozens to hundreds depending on different positive samples. If the matching score is directly fitted with the deep learning model, it will make the model training process difficult to converge. Therefore, Figure 5 As shown, each matching score can be normalized to obtain a sample matching score, which can standardize the matching scores and solve the comparability between matching scores. The matching scores are made at the same order of magnitude, so that they can better fit the deep learning model and make the model training process converge better and easier. Then, the first batch of sample sets can be better constructed according to the first batch of positive samples, the first batch of negative samples and the sample matching scores. The normalization of each matching score can be obtained by the following formula (1):

[0101]

[0102] Where q is a positive sample, d is the source text data, k is the total number of target text data, es_normed(q, d) is the sample matching score, score(q, d) is the matching score, is the maximum matching score, where, because the positive sample is the best match with itself, the maximum matching score is the score obtained by matching the positive sample with itself.

[0103] Among them, score(q, d) can be obtained by the following formula (2):

[0104] score(q,d)=coord(q,d)*queryNorm(q)*∑ t∈d (tf(t∈d)*idf(t) 2 *

[0105] boost(t)*norm(t,d))(2)

[0106] Among them, coord(q, d) is the coordination factor, queryNorm(q) is the query norm, tf(t∈d) is the term frequency, idf(t) 2 is the inverse document frequency, boost(t) is the term weight, and norm(t, d) is the length norm.

[0107] Optionally, in the above Figure 2 On the basis of the corresponding embodiment, in another optional embodiment of the training method of the similar text matching model provided in the embodiment of the present application, a triple construction operation is performed on the first batch of positive example sentence vectors to obtain a plurality of first batch triples, including:

[0108] According to the sample matching scores, the first batch of negative example sentence vectors are divided into a similar sentence vector set and a heterogeneous sentence vector set.

[0109] Extract any similar sentence vector from the similar sentence vector set to obtain the first batch of similar sentence vectors;

[0110] Extract any heterogeneous sentence vector from the heterogeneous sentence vector set to obtain the first batch of heterogeneous sentence vectors.

[0111] In this embodiment, after obtaining the first batch of positive example sentence vectors and the first batch of negative example sentence vectors, the first batch of negative example sentence vectors can be divided into a similar sentence vector set and a heterogeneous sentence vector set according to the sample matching scores of the first batch of sample sets. Then, any similar sentence vector can be extracted from the similar sentence vector set to obtain the first batch of similar sentence vectors. Similarly, any heterogeneous sentence vector can be extracted from the heterogeneous sentence vector set to obtain the first batch of heterogeneous sentence vectors. Therefore, triples can be constructed for the first batch of positive example sentence vectors based on the first batch of similar sentence vectors and the first batch of heterogeneous sentence vectors, so that the distance between similar text and dissimilar text can be better represented by triples.

[0112] Specifically, according to the sample matching score, the first batch of negative example sentence vectors are divided. Specifically, the sentence vectors with sample matching scores greater than 0.5 can be regarded as a similar sentence vector set. Similarly, the sentence vectors with sample matching scores less than 0.5 can be regarded as a heterogeneous sentence vector set. Then, any similar sentence vector can be randomly extracted from the similar sentence vector set to obtain the first batch of similar sentence vectors. Similarly, any heterogeneous sentence vector can be randomly extracted from the heterogeneous sentence vector set to obtain the first batch of heterogeneous sentence vectors.

[0113] Table 2

[0114]

[0115] For example, as shown in Table 2, the matching scores corresponding to the negative samples can be used, and the negative sample 1 and the negative sample 2 corresponding to the positive sample "This electric screwdriver is amazing! 250 revolutions per minute, easy to drill holes! 47 bits, can screw 500 kinds of screws! Good to use and not expensive, only 10% off!" can be converted into negative sentence vectors to obtain negative sentence vector 1 and negative sentence vector 2. Then, according to the sample matching score 0.89 corresponding to the negative sentence vector 1 and the sample matching score 0.18 corresponding to the negative sentence vector 2, the negative sentence vector 1 can be used as the same sentence vector, and the negative sentence vector 2 can be used as the different sentence vector.

[0116] Optionally, in the above Figure 2 On the basis of the corresponding embodiment, in another optional embodiment of the training method of the similar text matching model provided in the embodiment of the present application, a loss calculation operation is performed on a plurality of first batch triples to obtain a first batch loss function corresponding to the first batch sample set, including:

[0117] Perform loss calculation operations on the first batch of positive example sentence vectors, the first batch of similar sentence vectors, and the first batch of heterogeneous sentence vectors, respectively, to obtain loss functions corresponding to several first batch triples;

[0118] The loss functions corresponding to several first batch triplets are weightedly calculated to obtain the first batch loss function.

[0119] In this embodiment, after obtaining several first batch triplets constructed based on the first batch positive example sentence vectors, loss calculation can be performed on each triplet, that is, loss calculation is performed on the first batch positive example sentence vectors, the first batch similar sentence vectors and the first batch heterogeneous sentence vectors respectively to obtain the loss function corresponding to each first batch triplet, and then, weighted calculation can be performed on the loss functions corresponding to several first batch triplets according to preset weights to obtain the first batch loss function, and a triplet loss function can be constructed based on triplets, which can shorten the distance between positive example samples and similar samples and increase the distance between positive example samples and heterogeneous samples, so that similar text vectors can form clusters in the feature space to achieve the purpose of text matching.

[0120] Specifically, after obtaining several first batch triplets, the loss function can be calculated for each triplet first. Specifically, the first batch positive example sentence vector, the first batch similar sentence vector and the first batch heterogeneous sentence vector can be substituted into the function expression of the triple loss function such as step S104 for calculation, and a loss function corresponding to a triplet can be obtained. Then, the several obtained loss functions can be integrated into one loss function. Specifically, the loss functions corresponding to several first batch triplets can be weightedly calculated by preset weights to obtain the first batch loss function corresponding to the first batch sample set, wherein the preset weights are set according to actual application requirements and are not specifically limited here.

[0121] Optionally, in the above Figure 2 On the basis of the corresponding embodiment, in another optional embodiment of the training method of the similar text matching model provided in the embodiment of the present application, based on the intermediate similar text matching model, repeatedly obtaining a second batch of sample sets corresponding to the target scene, and performing a vector conversion operation, a triple construction operation, a loss calculation operation, and a parameter adjustment operation to obtain a target similar text matching model, including:

[0122] Obtain a second batch of sample sets corresponding to the target scene, and perform a vector conversion operation, a triplet construction operation, and a loss calculation operation according to the second batch of sample sets to obtain a second loss function;

[0123] If the second loss function is less than the first threshold, the current intermediate similar text matching model is used as the target similar text matching model.

[0124] In this embodiment, after obtaining the intermediate similar text matching model, the second batch of sample sets corresponding to the target scene can be further obtained to perform vector conversion operations, triple construction operations, and loss calculation operations to obtain a second loss function, and the intermediate similar text matching model can be reversely iterated trained through the second loss function. When the second loss function is less than the first threshold, the current intermediate similar text matching model can be used as the target similar text matching model, so that the model can fully learn the similarity between text vectors, so that the target similar text matching model can be better fitted, thereby improving the recall rate of the target similar text matching model for similar texts to a certain extent.

[0125] Specifically, after obtaining the second batch of sample sets corresponding to the target scene, wherein the second batch of sample sets is used to generally refer to other batches of sample sets extracted from the sample set that are different from the first batch of sample sets, which can be specifically expressed as three batches, four batches or N batches of sample sets, then, operations similar to the vector conversion operation, triple construction operation, and loss calculation operation in steps S102 to S104 can be performed, which will not be repeated here, to obtain a second loss function, wherein the second loss function can be used to generally refer to the loss function corresponding to each batch of sample sets.

[0126] Furthermore, after obtaining the second loss function, it can be understood that the smaller the second loss function is, the better the model fit is. Therefore, the second loss function can be compared with the first threshold, wherein the first threshold can be specifically expressed as a smaller value, such as 0.18. The first threshold is set according to actual application requirements and is not specifically limited here. Then, when the second loss function is less than the first threshold, it can be understood that when the second loss function is small enough, the current intermediate similar text matching model tends to be stable, and the current intermediate similar text matching model has converged, then the current intermediate similar text matching model can be used as the target similar text matching model.

[0127] Optionally, in the above Figure 2 On the basis of the corresponding embodiment, in another optional embodiment of the training method of the similar text matching model provided in the embodiment of the present application, based on the intermediate similar text matching model, repeatedly obtaining a second batch of sample sets corresponding to the target scene, and performing a vector conversion operation, a triple construction operation, a loss calculation operation, and a parameter adjustment operation to obtain a target similar text matching model, including:

[0128] Obtaining intermediate model parameters of the intermediate similar text matching model;

[0129] A second batch of sample sets corresponding to the target scene is obtained, and after performing a vector conversion operation, a triplet construction operation, and the above-mentioned parameter adjustment operation, a current similar text matching model is obtained, and the current similar text matching model includes current model parameters;

[0130] If the difference between the intermediate model parameters and the current model parameters meets the second threshold, the current intermediate similar text matching model is used as the target similar text matching model.

[0131] In this embodiment, after obtaining the intermediate similar text matching model, the second batch of sample sets corresponding to the target scene can be further obtained to perform vector conversion operations, triple construction operations, and loss calculation operations to obtain a second loss function, and the intermediate similar text matching model can be reversely iterated trained through the second loss function. When the difference between the intermediate model parameters and the current model parameters meets the second threshold, the current intermediate similar text matching model can be used as the target similar text matching model, so that the model can fully learn the similarity between text vectors, so that the target similar text matching model can be better fitted, thereby improving the recall rate of the target similar text matching model for similar texts to a certain extent, wherein the second threshold can be specifically expressed as a smaller value, and the second threshold is set according to actual application requirements, and no specific restrictions are made here.

[0132] Specifically, after obtaining the intermediate similar text matching model, the intermediate model parameters can be extracted. Further, the next batch of sample sets corresponding to the target scene can be obtained, and operations similar to the vector conversion operation, triple construction operation and the above-mentioned parameter adjustment operation in steps S102 to S105 can be repeatedly performed, which will not be repeated here. The current similar text matching model can be obtained, and the model parameters of the current similar text matching model can be extracted to obtain the current model parameters.

[0133] Furthermore, since the intermediate similar text matching model tends to be stable, the convergence of the intermediate similar text matching model can be reflected by the stability of the model parameters. Therefore, the difference between the intermediate model parameters and the current model parameters can be calculated. If the difference between the intermediate model parameters and the current model parameters is less than the second threshold, it can be understood that the difference between the intermediate model parameters and the current model parameters is small enough, that is, the model parameters tend to be stable, that is, the intermediate similar text matching model tends to be stable, and the current intermediate similar text matching model can be used as the target similar text matching model.

[0134] Optionally, in the above Figure 2On the basis of the corresponding embodiment, in another optional embodiment of the training method of the similar text matching model provided in the embodiment of the present application, based on the intermediate similar text matching model, repeatedly obtaining a second batch of sample sets corresponding to the target scene, and performing a vector conversion operation, a triple construction operation, a loss calculation operation, and a parameter adjustment operation, after obtaining the target similar text matching model, the method further includes:

[0135] Receive the text to be matched;

[0136] The text to be matched and the target text dataset are respectively passed through the target similar text matching model to obtain the sentence vector to be matched and several original sentence vectors;

[0137] Calculate the similarity between the sentence vector to be matched and each original sentence vector to obtain several similarity scores;

[0138] The target similar text is determined according to a number of similarity scores, and the target similar text is pushed to the target terminal device.

[0139] In this embodiment, after obtaining the target similar text matching model, the target similar text matching model can be applied by receiving the text to be matched and obtaining the target text data set corresponding to the text to be matched. Then, the text to be matched and the target text data set are respectively input into the target similar text matching model. The sentence vector to be matched and several original sentence vectors are obtained through the target similar text model. The similarity between the sentence vector to be matched and each original sentence vector can be calculated to obtain several similarity scores. The target similar text can be determined based on the several similarity scores and pushed to the target terminal device, so as to better and more accurately recommend matching target similar texts for the target objects.

[0140] Specifically, the text to be matched can be expressed as advertising copy, product copy, product keywords, etc., or other texts, which are not specifically limited here. After the target similar text matching model is obtained, if the text to be matched, such as advertising copy A, is received from the target object through the target terminal device, the target scene to which the text to be matched belongs, such as the advertising retrieval scene, can be determined first, and the corresponding target similar text matching model and target text data set, such as the advertising copy retrieval library, can be determined according to the target scene. Then, the obtained text to be matched and the target text data set can be input into the target similar text matching model respectively, and the sentence vector to be matched and several original sentence vectors can be obtained through the target similar text matching model, and then the sentence vector to be matched can be matched with each original sentence vector in pairs, and the similarity between the sentence vector to be matched and each original sentence vector can be calculated to obtain the similarity score between the sentence vector to be matched and each original sentence vector, wherein the similarity can be calculated specifically by Euclidean distance or cosine similarity, or other similarity calculation methods, which are not specifically limited here.

[0141] Furthermore, after obtaining the similarity score between the sentence vector to be matched and each original sentence vector, the similarity scores can be sorted from high to low according to the principle that the higher the similarity score, the higher the similarity, and the original sentence vectors corresponding to the top ten or one hundred similarity scores can be selected according to the needs of the target scenario. Then, the text corresponding to the selected ten or one hundred original sentence vectors can be determined as the target similar text and pushed to the target terminal device.

[0142] Among them, when the target similar text is pushed to the target terminal device, it can be pushed specifically according to the type of the target similar text. For example, assuming that the target similar text is expressed as a product image or a product link, etc., the target similar text can be directly pushed to the target terminal device; assuming that the target similar text is expressed as a video advertisement or hot information, etc., based on the historical click-through rate and conversion rate of the target object, the text that better matches the target object can be further determined from the target similar text and then pushed to the target terminal device.

[0143] It is understandable that the target similarity text matching model can also be used in multiple links such as product retrieval, advertising material retrieval, advertising estimation model features, advertising analysis and diagnosis, etc., which can improve the overall advertising delivery effect of the entire link. For example, products or advertising materials usually have corresponding copywriting. By combining text keyword retrieval and target similarity text matching model retrieval, the matching accuracy and recall rate can be greatly improved, thereby recommending more matching products or advertisements to the target object. Alternatively, it can also be applied to multiple links involved in the advertising delivery process, such as recall, rough or fine sorting model estimation and strategy adjustment, and delivery effect analysis and diagnosis. For example, in the rough sorting stage, the target similarity text matching model is used to obtain similar texts and pictures and videos in the advertisement to filter similar advertisements and increase the diversity of advertisements. Alternatively, by combining the rough or fine sorting estimation model with the target similarity text matching model, the advertisement itself can be better understood and the model generalization performance can be increased.

[0144] It is understandable that the target similarity text matching model can also be applied to other text retrieval scenarios, such as public common sense, virtual or physical item retrieval, fine-grained book retrieval, legal document retrieval, etc., without specific limitation here.

[0145] The following is a detailed description of the training device for the similar text matching model in this application. Figure 7 , Figure 7 This is a schematic diagram of an embodiment of a training device for a similar text matching model in an embodiment of the present application. The training device 20 for a similar text matching model includes:

[0146] An acquisition unit 201 is used to acquire a first batch of sample sets corresponding to a target scene, wherein the first batch of sample sets includes a first batch of positive samples and a first batch of negative samples;

[0147] A processing unit 202 is used to input the first batch of positive sample and the first batch of negative sample into the original similar text matching model for vector conversion operation to obtain the first batch of positive sentence vector and the first batch of negative sentence vector;

[0148] The processing unit 202 is further used to perform a triple construction operation on the first batch of positive example sentence vectors to obtain a plurality of first batch triples, wherein each first batch triple includes a first batch of positive example sentence vectors, a first batch of similar sentence vectors, and a first batch of different sentence vectors, and the first batch of similar sentence vectors and the first batch of different sentence vectors are derived from the first batch of negative example sentence vectors;

[0149] The processing unit 202 is further used to perform a loss calculation operation on a plurality of first batch triples to obtain a first batch loss function corresponding to the first batch sample set;

[0150] The processing unit 202 is further used to perform parameter adjustment operations on the original similar text matching model according to the first batch loss function to obtain an intermediate similar text matching model;

[0151] The processing unit 202 is also used to repeatedly obtain a second batch of sample sets corresponding to the target scene based on the intermediate similar text matching model, and perform vector conversion operations, triple construction operations, loss calculation operations and parameter adjustment operations to obtain a target similar text matching model.

[0152] Optionally, in the above Figure 7 On the basis of the corresponding embodiment, in another embodiment of the training device for the similar text matching model provided by the embodiment of the present application, the acquisition unit 201 can be specifically used for:

[0153] Acquire a target text data set corresponding to the target scene, wherein the target text data set includes at least a first batch of positive example samples and source text data corresponding to the target scene;

[0154] Retrieve N first matching texts corresponding to the first batch of positive samples from the target text dataset as N first batch of negative samples;

[0155] Calculate the matching scores between the first batch of positive samples and each of the first batch of negative samples to obtain N first matching scores;

[0156] Normalizing the N first matching scores respectively to obtain N sample matching scores;

[0157] The first batch of sample sets is constructed according to the first batch of positive samples, the first batch of negative samples and the sample matching scores.

[0158] Optionally, in the above Figure 7 On the basis of the corresponding embodiment, in another embodiment of the training device for the similar text matching model provided by the embodiment of the present application, the processing unit 202 can be specifically used for:

[0159] According to the sample matching scores, the first batch of negative example sentence vectors are divided into a similar sentence vector set and a heterogeneous sentence vector set.

[0160] Extract any similar sentence vector from the similar sentence vector set to obtain the first batch of similar sentence vectors;

[0161] Extract any heterogeneous sentence vector from the heterogeneous sentence vector set to obtain the first batch of heterogeneous sentence vectors.

[0162] Optionally, in the above Figure 7 On the basis of the corresponding embodiment, in another embodiment of the training device for the similar text matching model provided by the embodiment of the present application, the processing unit 202 can be specifically used for:

[0163] Perform loss calculation operations on the first batch of positive example sentence vectors, the first batch of similar sentence vectors, and the first batch of heterogeneous sentence vectors, respectively, to obtain loss functions corresponding to several first batch triples;

[0164] The loss functions corresponding to several first batch triplets are weightedly calculated to obtain the first batch loss function.

[0165] Optionally, in the above Figure 7 On the basis of the corresponding embodiment, in another embodiment of the training device for the similar text matching model provided by the embodiment of the present application, the processing unit 202 can be specifically used for:

[0166] Obtain a second batch of sample sets corresponding to the target scene, and perform a vector conversion operation, a triplet construction operation, and a loss calculation operation according to the second batch of sample sets to obtain a second loss function;

[0167] If the second loss function is less than the first threshold, the current intermediate similar text matching model is used as the target similar text matching model.

[0168] Optionally, in the above Figure 7 On the basis of the corresponding embodiment, in another embodiment of the training device for the similar text matching model provided by the embodiment of the present application, the processing unit 202 can be specifically used for:

[0169] Obtaining intermediate model parameters of the intermediate similar text matching model;

[0170] After obtaining a second batch of sample sets corresponding to the target scene and performing a vector conversion operation, a triplet construction operation, and the above parameter adjustment operation, a current similar text matching model is obtained, and the current similar text matching model includes current model parameters;

[0171] If the difference between the intermediate model parameters and the current model parameters meets the second threshold, the current intermediate similar text matching model is used as the target similar text matching model.

[0172] Optionally, in the above Figure 7 Based on the corresponding embodiment, in another embodiment of the training device for the similar text matching model provided in the embodiment of the present application,

[0173] The acquisition unit 201 is also used to receive the text to be matched;

[0174] The processing unit 202 is further used to respectively pass the to-be-matched text and the target text dataset through the target similar text matching model to obtain a to-be-matched sentence vector and a plurality of original sentence vectors;

[0175] The processing unit 202 is further used to calculate the similarity between the sentence vector to be matched and each original sentence vector to obtain a plurality of similarity scores;

[0176] The determination unit 203 is used to determine the target similar text according to a plurality of similarity scores, and push the target similar text to the target terminal device.

[0177] On the other hand, the present application provides another schematic diagram of a computer device, such as Figure 8 As shown, Figure 8 3 is a schematic diagram of a computer device structure provided in an embodiment of the present application. The computer device 300 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 310 (for example, one or more processors) and a memory 320, and one or more storage media 330 (for example, one or more mass storage devices) storing application programs 331 or data 332. Among them, the memory 320 and the storage medium 330 may be temporary storage or permanent storage. The program stored in the storage medium 330 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the computer device 300. Furthermore, the central processing unit 310 may be configured to communicate with the storage medium 330 to execute a series of instruction operations in the storage medium 330 on the computer device 300.

[0178] The computer device 300 may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input and output interfaces 360, and / or one or more operating systems 333, such as Windows Server 2000, Windows Server 2003, Windows Server 2003E, Windows Server 2003R ... TM , Mac OS X TM , Unix TM ,Linux TM , FreeBSD TM etc.

[0179] The computer device 300 is also used to execute the following steps: Figure 2 The steps in the corresponding embodiments.

[0180] Another aspect of the present application provides a computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium, and when the instructions are executed on a computer, the computer executes the following Figure 2 The illustrated embodiments describe steps in a method.

[0181] Another aspect of the present application provides a computer program product comprising instructions, which when executed on a computer or a processor, causes the computer or the processor to execute the following Figure 2 The illustrated embodiments describe steps in a method.

[0182] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0183] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0184] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0185] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0186] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

Claims

1. A training method for a similar text matching model, characterized in that: include: Acquire a first batch of sample sets corresponding to the target scene, wherein the first batch of sample sets is a self-supervised training sample set, and the first batch of sample sets includes a first batch of positive samples and a first batch of negative samples; Inputting the first batch of positive example samples and the first batch of negative example samples into the original similar text matching model for vector conversion operation respectively, to obtain the first batch of positive example sentence vectors and the first batch of negative example sentence vectors; Performing a triplet construction operation on the first batch of positive example sentence vectors to obtain a plurality of first batch triples, wherein each of the first batch triples includes the first batch of positive example sentence vectors, the first batch of similar sentence vectors, and the first batch of heterogeneous sentence vectors, and the first batch of similar sentence vectors and the first batch of heterogeneous sentence vectors are derived from the first batch of negative example sentence vectors; Performing a loss calculation operation on the plurality of first batch triples to obtain a first batch loss function corresponding to the first batch sample set; According to the first batch loss function, adjusting the parameters of the original similar text matching model to obtain an intermediate similar text matching model; Based on the intermediate similar text matching model, repeatedly obtaining a second batch of sample sets corresponding to the target scene, and performing the vector conversion operation, the triple construction operation, the loss calculation operation, and the parameter adjustment operation to obtain a target similar text matching model; The step of obtaining a first batch of sample sets corresponding to the target scene includes: Acquire a target text data set corresponding to the target scenario, wherein the target text data set at least includes the first batch of positive example samples and source text data corresponding to the target scenario; Retrieving N first matching texts corresponding to the first batch of positive example samples from the target text data set as N first batch of negative example samples, where N is an integer greater than 1; Calculate the matching scores between the first batch of positive example samples and each of the first batch of negative example samples to obtain N first matching scores; Normalizing the N first matching scores respectively to obtain N sample matching scores; The first batch of sample sets is constructed according to the first batch of positive samples, the first batch of negative samples, and the sample matching scores.

2. The method according to claim 1, characterized in that The triple construction operation is performed on the first batch of positive example sentence vectors to obtain a plurality of first batch triples, including: According to the sample matching scores, the first batch of negative example sentence vectors are divided to obtain a similar sentence vector set and a heterogeneous sentence vector set; Extract any similar sentence vector from the similar sentence vector set to obtain the first batch of similar sentence vectors; Any heterogeneous sentence vector is extracted from the heterogeneous sentence vector set to obtain the first batch of heterogeneous sentence vectors.

3. The method according to claim 1, characterized in that The performing loss calculation operation on the plurality of first batch triples to obtain the first batch loss function corresponding to the first batch sample set includes: Performing loss calculation operations on the first batch of positive example sentence vectors, the first batch of similar sentence vectors, and the first batch of heterogeneous sentence vectors respectively to obtain loss functions corresponding to the plurality of first batch triples; A weighted calculation operation is performed on the loss functions corresponding to the plurality of first batch triplets to obtain the first batch loss functions.

4. The method according to claim 1, characterized in that: The method of repeatedly acquiring a second batch of sample sets corresponding to the target scene based on the intermediate similar text matching model, and performing the vector conversion operation, the triple construction operation, the loss calculation operation, and the parameter adjustment operation to obtain a target similar text matching model includes: Acquire a second batch of sample sets corresponding to the target scene, and perform the vector conversion operation, the triplet construction operation, and the loss calculation operation according to the second batch of sample sets to obtain a second loss function; If the second loss function is less than the first threshold, the current intermediate similar text matching model is used as the target similar text matching model.

5. The method according to claim 1, characterized in that The method of repeatedly acquiring a second batch of sample sets corresponding to the target scene based on the intermediate similar text matching model, and performing the vector conversion operation, the triple construction operation, the loss calculation operation, and the parameter adjustment operation to obtain a target similar text matching model includes: Obtaining intermediate model parameters of the intermediate similar text matching model; After obtaining a second batch of sample sets corresponding to the target scene and performing the vector conversion operation, the triplet construction operation and the parameter adjustment operation, a current similar text matching model is obtained, wherein the current similar text matching model includes current model parameters; If the difference between the intermediate model parameters and the current model parameters meets the second threshold, the current intermediate similar text matching model is used as the target similar text matching model.

6. The method according to claim 1, characterized in that After repeatedly acquiring a second batch of sample sets corresponding to the target scene based on the intermediate similar text matching model, and performing the vector conversion operation, the triple construction operation, the loss calculation operation, and the parameter adjustment operation to obtain the target similar text matching model, the method further includes: Receive the text to be matched; Passing the to-be-matched text and the target text dataset through the target similar text matching model respectively to obtain a to-be-matched sentence vector and a plurality of original sentence vectors; Calculate the similarity between the sentence vector to be matched and each original sentence vector to obtain a number of similarity scores; A target similar text is determined according to the plurality of similarity scores, and the target similar text is pushed to a target terminal device.

7. A training device for a similar text matching model, characterized in that: include: An acquisition unit, configured to acquire a first batch of sample sets corresponding to a target scene, wherein the first batch of sample sets is a self-supervised training sample set, and the first batch of sample sets includes a first batch of positive samples and a first batch of negative samples; A processing unit, configured to input the first batch of positive example samples and the first batch of negative example samples into an original similar text matching model for vector conversion operation to obtain a first batch of positive example sentence vectors and a first batch of negative example sentence vectors; The processing unit is further used to perform a triple construction operation on the first batch of positive example sentence vectors to obtain a plurality of first batch triples, wherein each of the first batch triples includes the first batch of positive example sentence vectors, the first batch of similar sentence vectors, and the first batch of heterogeneous sentence vectors, and the first batch of similar sentence vectors and the first batch of heterogeneous sentence vectors are derived from the first batch of negative example sentence vectors; The processing unit is further used to perform a loss calculation operation on the plurality of first batch triples to obtain a first batch loss function corresponding to the first batch sample set; The processing unit is further used to perform a parameter adjustment operation on the original similar text matching model according to the first batch loss function to obtain an intermediate similar text matching model; The processing unit is further used to repeatedly obtain a second batch of sample sets corresponding to the target scene based on the intermediate similar text matching model, and perform the vector conversion operation, the triple construction operation, the loss calculation operation and the parameter adjustment operation to obtain a target similar text matching model; The acquisition unit can be used specifically for: Acquire a target text data set corresponding to the target scenario, wherein the target text data set at least includes the first batch of positive example samples and source text data corresponding to the target scenario; Retrieving N first matching texts corresponding to the first batch of positive example samples from the target text data set as N first batch of negative example samples, where N is an integer greater than 1; Calculate the matching scores between the first batch of positive example samples and each of the first batch of negative example samples to obtain N first matching scores; Normalizing the N first matching scores respectively to obtain N sample matching scores; The first batch of sample sets is constructed according to the first batch of positive samples, the first batch of negative samples, and the sample matching scores.

8. The device according to claim 7, characterized in that The processing unit can be used for: According to the sample matching scores, the first batch of negative example sentence vectors are divided to obtain a similar sentence vector set and a heterogeneous sentence vector set; Extract any similar sentence vector from the similar sentence vector set to obtain the first batch of similar sentence vectors; Any heterogeneous sentence vector is extracted from the heterogeneous sentence vector set to obtain the first batch of heterogeneous sentence vectors.

9. The device according to claim 7, characterized in that The processing unit can be used for: Performing loss calculation operations on the first batch of positive example sentence vectors, the first batch of similar sentence vectors, and the first batch of heterogeneous sentence vectors respectively to obtain loss functions corresponding to the plurality of first batch triples; A weighted calculation operation is performed on the loss functions corresponding to the plurality of first batch triplets to obtain the first batch loss functions.

10. The device according to claim 7, characterized in that The processing unit can be used for: Acquire a second batch of sample sets corresponding to the target scene, and perform the vector conversion operation, the triplet construction operation, and the loss calculation operation according to the second batch of sample sets to obtain a second loss function; If the second loss function is less than the first threshold, the current intermediate similar text matching model is used as the target similar text matching model.

11. The device according to claim 7, characterized in that The processing unit can be used for: Obtaining intermediate model parameters of the intermediate similar text matching model; After obtaining a second batch of sample sets corresponding to the target scene and performing the vector conversion operation, the triplet construction operation and the parameter adjustment operation, a current similar text matching model is obtained, wherein the current similar text matching model includes current model parameters; If the difference between the intermediate model parameters and the current model parameters meets the second threshold, the current intermediate similar text matching model is used as the target similar text matching model.

12. The device according to claim 7, characterized in that The acquisition unit is also used to receive the text to be matched; The processing unit is further used to respectively pass the to-be-matched text and the target text data set through the target similar text matching model to obtain a to-be-matched sentence vector and a plurality of original sentence vectors; The processing unit is further used to calculate the similarity between the sentence vector to be matched and each original sentence vector to obtain a plurality of similarity scores; A determination unit is used to determine a target similar text according to the plurality of similarity scores, and push the target similar text to a target terminal device.

13. A computer device, characterized in that: include: memories, transceivers, processors, and bus systems; Wherein, the memory is used to store programs; The processor is used to implement the method according to any one of claims 1 to 6 when executing the program in the memory; The bus system is used to connect the memory and the processor so that the memory and the processor can communicate with each other.

14. A computer-readable storage medium comprising instructions, which, when executed on a computer, causes the computer to perform the method according to any one of claims 1 to 6.

15. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images