Model training method, text matching method, corresponding device, equipment and medium

By obtaining and using sentences, keywords and intent information in the training text pair, the text matching model is trained, and the problem of low sentence-level text matching accuracy in the prior art is solved, and higher text matching accuracy and User experience.

CN119067122APending Publication Date: 2024-12-03BEIJING XIAOMI MOBILE SOFTWARE CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310629033.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-30
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

During the training process, the existing sentence-level text matching model cannot fully understand the semantics of the sentence due to sample number and scene limitations, resulting in low matching accuracy, affecting user experience and product quality.

Method used

By obtaining multiple training text pairs, each text pair includes sentence, keyword information and intent information, and using pre-trained language model and classifier, the target training text feature vector and probability distribution are generated, and the model is trained based on the matching results. Introduce keyword and intent information to improve matching accuracy.

Benefits of technology

A more accurate text matching model is realized, avoiding incorrect matching of keyword granularity and intention granularity, improving the accuracy of text matching, and enhancing user experience and product quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119067122A_ABST
    Figure CN119067122A_ABST
Patent Text Reader

Abstract

The invention relates to a model training method, a text matching method, corresponding devices, equipment and a medium. The training method comprises the steps that multiple training text pairs are obtained, each training text pair comprises a first training text, a second training text and a matching result, the first training text comprises a first sentence, first keyword information and first intention information, and the second training text comprises a second sentence, second keyword information and second intention information; the matching result is used for representing whether the first sentence is matched with the second sentence; and training the initial model according to the training text pair to obtain a text matching model. Thus, the training text marked with the keyword information and the intention information is used for training to obtain the text matching model, the accurate text matching model can be obtained, whether sentences in the text pair are matched or not can be accurately recognized on the basis of the text matching model, and the text matching accuracy is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a text matching model training method, a text matching method, a training device, a text matching device, an electronic device and a medium. Background Art

[0002] Text matching is a basic work in the field of natural language processing. It is a technology that determines whether text pairs are semantically similar. It is widely used in search engine systems, question-answering systems, and recommendation systems. Among them, sentence-level text matching is the core technology of current industrial-level question-answering systems. Its application scope includes but is not limited to open domain knowledge question-answering, intelligent customer service, and professional field question-answering such as finance and medical care. Its matching accuracy affects user experience and product quality.

[0003] In recent years, the common method for sentence-level text matching tasks is based on deep learning. Among them, deep learning models mostly use submodules such as convolutional neural networks and recurrent neural networks to construct models. However, due to the limitations of the number of training samples or the scenarios to which the training samples belong, the trained deep learning models are still unable to fully understand the semantics of the sentences, resulting in low accuracy of text matching, which affects user experience and product quality. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides a training method for a text matching model, a text matching method, a training device, a text matching device, an electronic device and a medium.

[0005] According to a first aspect of an embodiment of the present disclosure, a method for training a text matching model is provided, the training method comprising:

[0006] Acquire multiple training text pairs, wherein each of the training text pairs includes a first training text, a second training text, and a matching result, the first training text includes a first sentence, first keyword information, and first intent information, the second training text includes a second sentence, second keyword information, and second intent information, and the matching result is used to indicate whether the first sentence and the second sentence match;

[0007] The initial model is trained according to the training text pairs to obtain a text matching model.

[0008] Optionally, the initial model includes a pre-trained language model and a first classifier;

[0009] The initial model is trained according to the training text to obtain a text matching model, including:

[0010] For each of the training text pairs, format conversion is performed on the first training text and the second training text included in the training text pair to obtain a target training text that conforms to the text format corresponding to the pre-trained language model;

[0011] Inputting the target training text into the pre-trained language model to obtain a target training text feature vector output by the pre-trained language model;

[0012] Inputting the target training text feature vector into the first classifier to obtain a target training text probability distribution output by the first classifier, wherein the target training text probability distribution can indicate whether the first sentence matches the second sentence;

[0013] The initial model is trained according to the target training text probability distribution and the matching results included in the training text pairs to obtain a text matching model.

[0014] Optionally, the training of the initial model according to the training text pair to obtain a text matching model further includes:

[0015] generating a keyword probability distribution according to the first keyword information and the second keyword information included in the target training text, wherein the keyword probability distribution can indicate whether the first keyword information matches the second keyword information; and

[0016] generating an intent probability distribution according to the first intent information and the second intent information included in the target training text, wherein the intent probability distribution can indicate whether the first intent information matches the second intent information;

[0017] The step of training the initial model according to the target training text probability distribution and the matching results of the training text pairs to obtain a text matching model comprises:

[0018] The initial model is trained according to the target training text probability distribution, the matching results included in the training text pairs, the keyword probability distribution and the intention probability distribution to obtain a text matching model.

[0019] Optionally, the initial model further includes a second classifier and a third classifier; and generating a keyword probability distribution according to the first keyword information and the second keyword information included in the target training text includes:

[0020] Generate a keyword feature vector of the target training text according to the first keyword information, the second keyword information and the pre-trained language model included in the target training text;

[0021] Inputting the keyword feature vector into the second classifier to obtain a keyword probability distribution generated by the second classifier;

[0022] The generating the intent probability distribution according to the first intent information and the second intent information included in the target training text includes:

[0023] Generate an intent feature vector of the target training text according to the first intent information, the second intent information and the pre-trained language model included in the target training text;

[0024] The intention feature vector is input into the third classifier to obtain the intention probability distribution generated by the third classifier.

[0025] Optionally, the training of the initial model according to the target training text probability distribution, the matching results included in the training text pairs, the keyword probability distribution and the intention probability distribution to obtain a text matching model includes:

[0026] Determining a target training text loss according to the target training text probability distribution and the matching results included in the training text pair;

[0027] Determine a keyword information classification loss according to the keyword probability distribution and the target training text probability distribution, and determine an intent information classification loss according to the intent probability distribution and the target training text probability distribution;

[0028] The initial model is trained according to the target training text loss, the keyword information classification loss and the intent information classification loss to obtain a text matching model.

[0029] Optionally, determining the keyword information classification loss according to the keyword probability distribution and the target training text probability distribution, and determining the intent information classification loss according to the intent probability distribution and the target training text probability distribution, includes:

[0030] The divergence distance between the keyword probability distribution and the target training text probability distribution is determined as the keyword information classification loss, and the divergence distance between the intent probability distribution and the target training text probability distribution is determined as the intent information classification loss.

[0031] Optionally, generating a keyword feature vector of the target training text according to the first keyword information, the second keyword information and the pre-trained language model included in the target training text includes:

[0032] Generating a keyword mask structure in the pre-trained language model according to the first sentence, the second sentence, the first keyword information, and the second keyword information included in the target training text;

[0033] Generating a keyword feature vector of the target training text according to the target training text and the keyword mask structure;

[0034] The step of generating the intent feature vector of the target training text according to the first intent information, the second intent information and the pre-trained language model included in the target training text includes:

[0035] Generate an intent mask structure in the pre-trained language model according to the first sentence, the second sentence, the first intent information, and the second intent information included in the target training text;

[0036] An intention feature vector of the target training text is generated according to the target training text and the intention mask structure.

[0037] Optionally, the obtaining of multiple training text pairs includes:

[0038] Acquire a plurality of sentence pairs, wherein the sentence pairs include a first sentence and a second sentence;

[0039] For each sentence pair, first keyword information and first intention information are identified from the first sentence, second keyword information and second intention information are identified from the second sentence, and matching results of the first sentence and the second sentence are obtained, the first sentence, the first keyword information and the first intention information are determined as a first training text, the second sentence, the second keyword and the second intention information are determined as a second training text, and the first training text, the second training text and the matching results are determined as a set of training text pairs.

[0040] According to a second aspect of an embodiment of the present disclosure, a text matching method is provided, the text matching method comprising:

[0041] Acquire a text pair to be matched, wherein the text pair to be matched includes a first sentence to be matched and a second sentence to be matched;

[0042] According to the to-be-matched text pair and a preset text matching model, a matching result of the first to-be-matched sentence and the second to-be-matched sentence is determined; wherein the text matching model is generated according to the text matching model training method described in any one of the first aspects of the present disclosure.

[0043] Optionally, the text matching model includes a pre-trained language model and a first classifier;

[0044] The determining, according to the text to be matched and a preset text matching model, a matching result of the first sentence to be matched and the second sentence to be matched includes:

[0045] Performing format conversion on the to-be-matched text pair to obtain a target text that conforms to the text format corresponding to the pre-trained language model;

[0046] Inputting the target text into the pre-trained language model to obtain a target text feature vector output by the pre-trained language model;

[0047] Inputting the target text feature vector into the first classifier to obtain a target text probability distribution output by the first classifier, wherein the target text probability distribution can indicate whether the first sentence to be matched matches the second sentence to be matched;

[0048] According to the target text probability distribution, a matching result of the first sentence to be matched and the second sentence to be matched is determined.

[0049] According to a third aspect of an embodiment of the present disclosure, a training device for a text matching model is provided, the training device comprising:

[0050] A first acquisition module is configured to acquire a plurality of training text pairs, wherein each of the training text pairs includes a first training text, a second training text, and a matching result, the first training text includes a first sentence, first keyword information, and first intent information, the second training text includes a second sentence, second keyword information, and second intent information, and the matching result is used to indicate whether the first sentence and the second sentence match;

[0051] The training module is configured to train the initial model according to the training text pairs to obtain a text matching model.

[0052] According to a fourth aspect of an embodiment of the present disclosure, a text matching device is provided, the text matching device comprising:

[0053] A second acquisition module is configured to acquire a to-be-matched text pair, wherein the to-be-matched text pair includes a first to-be-matched sentence and a second to-be-matched sentence;

[0054] The first determination module is configured to determine the matching results of the first sentence to be matched and the second sentence to be matched according to the text pair to be matched and a preset text matching model; wherein the text matching model is generated according to the training method of the text matching model described in any one of the first aspects of the present disclosure.

[0055] According to a fifth aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0056] processor;

[0057] a memory for storing processor-executable instructions;

[0058] Wherein, the processor is configured to execute the text matching model training method described in the first aspect of the present disclosure, or to execute the text matching method described in the second aspect of the present disclosure.

[0059] According to the sixth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the training method of the text matching model provided in the first aspect of the present disclosure are implemented, or the steps of the text matching method provided in the second aspect of the present disclosure are implemented.

[0060] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0061] Using the above technical solution, first, multiple training text pairs are obtained, each training text pair includes a first training text, a second training text and a matching result, the first training text includes a first sentence, a first keyword information and a first intention information, the second training text includes a second sentence, a second keyword information and a second intention information, and the matching result is used to characterize whether the first sentence and the second sentence match; then, the initial model is trained according to the training text pairs to obtain a text matching model. In this way, the text matching model is obtained by training the training texts with the keyword information and the intention information marked, and the keyword information and the intention information of the sentence are introduced in the process of training the text matching model, so that the text matching model can avoid the problem of incorrect matching of the keyword granularity and the intention granularity in the text matching, and a more accurate text matching model can be obtained, and then based on the text matching model, it can accurately identify whether the sentences in the text pair match, thereby improving the accuracy of the text matching.

[0062] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0064] Figure 1 The figure is a flowchart of a method for training a text matching model according to an exemplary embodiment.

[0065] Figure 2 The present invention is a flowchart of a method for obtaining a text matching model by training an initial model according to a training text pair according to an exemplary embodiment.

[0066] Figure 3 It is a schematic structural diagram of an initial model according to an exemplary embodiment.

[0067] Figure 4 The figure is a flowchart of a text matching method according to an exemplary embodiment.

[0068] Figure 5 It is a block diagram of a training device for a text matching model according to an exemplary embodiment.

[0069] Figure 6 The invention is a block diagram of a text matching device according to an exemplary embodiment.

[0070] Figure 7 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0071] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0072] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the device is located and with the authorization given by the owner of the corresponding device.

[0073] At present, sentence-level text matching technology has the problem of fine-grained matching errors, which can easily mistakenly identify two sentences that are literally similar but semantically different as matches. According to the different types of matching errors, they are generally divided into key information matching errors and intent information matching errors.

[0074] For example, taking the key information mismatch as an example, suppose sentence Q1: How to adjust the brightness of the mobile phone, sentence Q2: How to adjust the brightness of the watch, one of the key information in sentence Q1 is "mobile phone", one of the key information in sentence Q2 is "watch", the words are similar, and the intentions of sentence Q1 and sentence Q2 are the same, the deep learning model easily recognizes sentence Q1 and sentence Q2 as matching. However, the semantics of these two sentences are different, that is, the recognition result of the deep learning model is inaccurate. For another example, taking the intention information mismatch as an example, suppose sentence Q3: When is the rocket launch generally, sentence Q4: Why do you want to launch a rocket? Since sentence Q3 and sentence Q4 have the same key information "rocket launch", the deep learning model easily recognizes sentence Q3 and sentence Q4 as matching, but the intentions of sentence Q3 and sentence Q4 are different. The intention of sentence Q3 is "when", and the intention of sentence Q4 is "why", therefore, sentence Q3 and sentence Q4 do not match, that is, the recognition result of the deep learning model is inaccurate. Therefore, in the related art, the accuracy of the trained deep learning model is low and it is unable to accurately identify whether the sentences in the text pairs match, which affects the user experience and product quality.

[0075] In view of this, the present disclosure provides a training method for a text matching model, a text matching method, a training device, a text matching device, an electronic device and a medium. By adopting the training method provided by the present disclosure, a more accurate text matching model can be obtained, and then based on the text matching model, it is possible to accurately identify whether the sentences in a text pair match, thereby improving the accuracy of text matching.

[0076] First, the training method of the text matching model provided in the present disclosure is described.

[0077] Figure 1 FIG. 1 is a flowchart of a method for training a text matching model according to an exemplary embodiment. Figure 1 As shown, the training method of the text matching model may include the following steps.

[0078] In step S11, a plurality of training sample pairs are obtained.

[0079] In the present disclosure, each training text pair includes a first training text, a second training text and a matching result. The first training text includes a first sentence, first keyword information and first intention information. The second training text includes a second sentence, second keyword information and second intention information. The matching result is used to characterize whether the first sentence and the second sentence match.

[0080] By way of example, the specific implementation of the above steps is: obtaining multiple sentence pairs, the sentence pairs including a first sentence and a second sentence; for each sentence pair, identifying first keyword information and first intent information from the first sentence, identifying second keyword information and second intent information from the second sentence, and obtaining matching results of the first sentence and the second sentence, determining the first sentence, the first keyword information and the first intent information as a first training text, determining the second sentence, the second keyword and the second intent information as a second training text, and determining the first training text, the second training text and the matching results as a set of training text pairs.

[0081] In one possible implementation, a pre-trained keyword recognition model is used to recognize keyword information from the first sentence as the first keyword information, and then a pre-trained intent recognition model is used to recognize intent information from the first sentence as the first intent information. Similarly, the pre-trained keyword recognition model and intent recognition model are used to respectively recognize second keyword information and second intent information from the second sentence. In this way, for each sentence, different recognition models are required to recognize information of different dimensions, making the method of determining key information and intent information more complicated.

[0082] For example, the first sentence is when a rocket is generally launched, and the second sentence is why a rocket is launched. The matching result is 0, where the flag 0 indicates that the first sentence and the second sentence do not match, and the flag 1 indicates that the first sentence and the second sentence match. Thus, the training text pair determined in the above manner is as follows:

[0083] First training text: When is a rocket usually launched? | When is the rocket launched?

[0084] Second training text: Why launch a rocket | why launch a rocket

[0085] Matching results: 0

[0086] Among them, in the first training text, "rocket|launch" represents the first keyword information, and "when" represents the first intention information; in the second training text, "rocket|launch" represents the second keyword information, and "why" represents the second intention information.

[0087] In another possible implementation, a pre-trained keyword recognition model is used to recognize keyword information from the first sentence as the first keyword information, and other information in the first sentence except the first keyword information is recognized as the first intention information. A pre-trained keyword recognition model is used to recognize keyword information from the second sentence as the second keyword information, and other information in the second sentence except the second keyword information is recognized as the second intention information. Taking the first sentence and the second sentence in the above example as an example, the training text pairs determined in the above manner are as follows:

[0088] First training text: When is a rocket usually launched? Rocket | Launch

[0089] Second training text: Why launch a rocket? Rocket | Launch

[0090] Matching results: 0

[0091] Among them, in the first training text, "rocket|launch" represents the first keyword information, and the other information, "when" represents the first intention information by default; in the second training text, "rocket|launch" represents the second keyword information, and "why" represents the second intention information by default.

[0092] In another embodiment, the first intent information of the first sentence and the second intent information of the second sentence are respectively identified using a pre-trained intent recognition model, and the other information in the first sentence except the first intent information is identified as the first keyword information, and the other information in the second sentence except the second intent information is identified as the second keyword information. Thus, compared with the previous embodiment, this embodiment can simplify the method of identifying key information and intent information.

[0093] In step S12, the initial model is trained according to the training text pairs to obtain a text matching model.

[0094] In one embodiment, a text matching model that can adapt to different scenarios can be trained. In this embodiment, the acquired training text pairs include text pairs belonging to different scenarios, for example, text pairs belonging to a search engine system scenario, text pairs belonging to a question-answering system scenario, text pairs belonging to a recommendation system scenario, etc. In this way, the text matching model trained using the multiple training text pairs can identify whether sentences in text pairs in different scenarios match.

[0095] In another embodiment, different text matching models are trained for different scenarios. For example, a text pair belonging to a search engine system scenario is obtained as a training text pair, and the training text pair is used to train the initial model to obtain a text matching model applied to the search engine system scenario. For another example, a text pair belonging to a question-answering system scenario is obtained as a training text pair, and the training text pair is used to train the initial model to obtain a text matching model applied to the question-answering system scenario.

[0096] By adopting the above technical scheme, a text matching model is obtained by training with training texts that have been marked with keyword information and intent information. The keyword information and intent information of the sentence are introduced in the process of training the text matching model, so that the text matching model can avoid the problem of incorrect matching of keyword granularity and intent granularity in text matching. In this way, a more accurate text matching model can be obtained, and then based on the text matching model, it is possible to accurately identify whether the sentences in the text pair match, thereby improving the accuracy of text matching.

[0097] Optionally, the initial model may include a pre-trained language model and a first classifier; accordingly, Figure 1 The specific implementation of step S12 is:

[0098] First, for each training text pair, the format of the first training text and the second training text included in the training text pair is converted to obtain a target training text that conforms to the text format corresponding to the pre-trained language model.

[0099] For example, the text format corresponding to the pre-trained language model is a format that uses special symbols [CLS] and [SEP] for splicing. The specific format is that the special symbol [CLS] is at the beginning of the sentence, followed by the training text, [SEP] is used as a separator, followed by another training text, and [SEP] is spliced ​​at the end of the sentence. Taking the training sample pair shown above as an example, the target training text after splicing is: [CLS] When is a rocket generally launched? Rocket|Launch [SEP] Why is a rocket launched? Rocket|Launch [SEP].

[0100] Next, the target training text is input into the pre-trained language model to obtain a target training text feature vector output by the pre-trained language model.

[0101] For example, the pre-trained language model encodes the target training text to obtain target training text encoding information, and uses a vector to represent the target training text encoding information to obtain a target training text feature vector.

[0102] It should be understood that the target training text encoding information represents the overall information of the target training text. In addition, the pre-trained language model can perform multi-level encoding on the target training text, wherein the top-level encoding incorporates the most information and can better reflect the information of the target training text. Therefore, the top-level encoding information of the pre-trained language model can be determined as the target training text encoding information, and the target training text feature vector obtained can reflect more information of the target training text.

[0103] Afterwards, the target training text feature vector is input into the first classifier to obtain the target training text probability distribution output by the first classifier, and the target training text probability distribution can indicate whether the first sentence matches the second sentence.

[0104] By way of example, the target training text probability distribution may be a binomial distribution, that is, the target training text probability distribution includes the probability of matching and the probability of not matching the first sentence and the second sentence. By way of another example, the target training text probability distribution may also be other distributions, for example, the target training text probability distribution includes the probability of matching, the probability of partial matching, and the probability of not matching the first sentence and the second sentence. The present disclosure does not specifically limit this.

[0105] Finally, the initial model is trained according to the probability distribution of the target training text and the matching results included in the training text pairs to obtain a text matching model.

[0106] For example, the model parameters of the initial model are adjusted according to the error between the target training text probability distribution and the matching result included in the training text pair. When the matching result represented by the target training text probability distribution is consistent with the matching result included in the training text pair, the training is terminated to obtain the text matching model.

[0107] Considering that keyword information and intent information have been marked in the training samples, in one embodiment, in addition to using the target training text probability distribution and the matching results included in the training text pairs to train the initial model, the keyword probability distribution can also be determined based on the marked keyword information, and the intent probability distribution can be determined based on the marked intent information, and then the initial model can be trained according to the target training text probability distribution, the matching results included in the training text pairs, the keyword probability distribution and the intent probability distribution.

[0108] For example, the initial model is trained according to the training text pairs to obtain a text matching model, and further includes:

[0109] Generate a keyword probability distribution according to the first keyword information and the second keyword information included in the target training text, where the keyword probability distribution can indicate whether the first keyword information matches the second keyword information; and

[0110] generating an intent probability distribution according to the first intent information and the second intent information included in the target training text, where the intent probability distribution can indicate whether the first intent information matches the second intent information;

[0111] According to the probability distribution of the target training text and the matching results of the training text pairs, the initial model is trained to obtain a text matching model, including:

[0112] The initial model is trained according to the target training text probability distribution, the matching results included in the training text pairs, the keyword probability distribution and the intent probability distribution to obtain a text matching model.

[0113] For example, the initial model trained by the present disclosure can be used to obtain the keyword probability distribution and the intent probability distribution, that is, the initial model trained by the present disclosure can be used to obtain the target training text probability distribution, the keyword probability distribution and the intent probability distribution, respectively. For another example, the keyword probability distribution and the intent probability distribution can also be obtained by other methods, for example, manually determining the keyword probability distribution and the intent probability distribution, or using other models different from the initial model trained by the present disclosure to generate the keyword probability distribution and the intent probability distribution. The present disclosure does not make specific limitations on this.

[0114] The following is a complete example in which a specific method of training an initial model according to training texts to obtain a text matching model is described in detail.

[0115] Figure 2 is a flowchart of a method for training an initial model according to a training text pair to obtain a text matching model according to an exemplary embodiment. In this embodiment, the initial model may include a second classifier and a third classifier in addition to a pre-trained language model and a first classifier. Accordingly, Figure 1 Step S12 may include the following steps:

[0116] In step S121, for each training text pair, format conversion is performed on the first training text and the second training text included in the training text pair to obtain a target training text that conforms to the text format corresponding to the pre-trained language model.

[0117] In step S122, the target training text is input into the pre-trained language model to obtain a target training text feature vector output by the pre-trained language model.

[0118] In step S123, the target training text feature vector is input into the first classifier to obtain the target training text probability distribution output by the first classifier.

[0119] The specific implementations of step S121, step S122 and step S123 have been described above and will not be repeated here.

[0120] In this embodiment, the step of generating a keyword probability distribution according to the first keyword information and the second keyword information included in the target training text may specifically include step S124 and step S125.

[0121] In step S124, a keyword feature vector of the target training text is generated according to the first keyword information, the second keyword information and the pre-trained language model included in the target training text.

[0122] Optionally, the specific implementation of step S124 is as follows: first, according to the first sentence, the second sentence, the first keyword information and the second keyword information included in the target training text, a keyword mask structure is generated in the pre-trained language model. For example, in the keyword mask structure, the position corresponding to the keyword information can be marked as 1, and other positions can be marked as 0.

[0123] Afterwards, a keyword feature vector of the target training text is generated according to the target training text and the keyword mask structure. For example, in the pre-trained language model, the top layer of encoding information of the keyword information can be obtained according to the target training text and the keyword mask structure, and the top layer of encoding information of the keyword information is average-pooled to obtain the keyword feature vector.

[0124] In step S125, the keyword feature vector is input into the second classifier to obtain the keyword probability distribution generated by the second classifier.

[0125] The keyword probability distribution can indicate whether the first keyword information matches the second keyword information.

[0126] For example, the second classifier is taken as a Softmax classifier, where the mathematical expression of the Softmax classifier is:

[0127] Among them, softmax(s i ) represents the probability distribution of keywords, S i Characterizes the score value of the Sofmax classifier for the first keyword information and the second keyword information in the i-th category. For example, assuming that the keyword probability distribution is a binomial distribution, N is 2, wherein the first category can represent the category where the first keyword information and the second keyword information match, and the second category can represent the category where the first keyword information and the second keyword information do not match.

[0128] In this embodiment, the step of generating the intent probability distribution according to the first intent information and the second intent information included in the target training text may specifically include the following steps S126 and S127.

[0129] In step S126, an intent feature vector of the target training text is generated according to the first intent information, the second intent information and the pre-trained language model included in the target training text.

[0130] Optionally, the specific implementation of step S126 is: according to the first sentence, the second sentence, the first intent information and the second intent information included in the target training text, an intent mask structure is generated in the pre-trained language model. For example, in the intent mask structure, the position corresponding to the intent information can be marked as 1, and other positions can be marked as 0.

[0131] Afterwards, the intent feature vector of the target training text is generated according to the target training text and the intent mask structure. For example, in the pre-trained language model, the top layer of encoding information of the intent information can be obtained according to the target training text and the intent mask structure, and the top layer of encoding information of the intent information is mean-pooled to obtain the intent feature vector.

[0132] In step S127, the intention feature vector is input into the third classifier to obtain the intention probability distribution generated by the third classifier.

[0133] The intent probability distribution can indicate whether the first intent information matches the second intent information. In addition, the method of generating the intent probability distribution is similar to the method of generating the keyword probability distribution in step S125, and this disclosure will not repeat it.

[0134] In this way, in order to facilitate the distinction between keyword information and intent information in the encoding stage, the keyword mask structure and the intent mask structure can be directly introduced into the pre-trained language model. That is, the pre-trained language model can be modified directly, which is suitable for a variety of pre-trained language models and easy to migrate to other application scenarios, thereby improving the flexibility of training.

[0135] In step S128, the initial model is trained according to the target training text probability distribution, the matching results included in the training text pairs, the keyword probability distribution and the intention probability distribution to obtain a text matching model.

[0136] Figure 3 FIG. 1 is a schematic diagram of the structure of an initial model according to an exemplary embodiment. Figure 3 As shown, the initial model includes a pre-trained language model, a first classifier, a second classifier, and a third classifier. The first classifier, the second classifier, and the third classifier are respectively connected to the pre-trained language model. For example, Figure 3 In the example, the first classifier, the second classifier, and the third classifier are all Softmax classifiers.

[0137] like Figure 3As shown, first, the converted target training text that conforms to the text format corresponding to the pre-trained language model is input into the pre-trained language model, and the pre-trained language model encodes the target training text to obtain target training text encoding information, keyword encoding information, and intent encoding information, respectively. The above encoding information is represented by vectors to obtain target training text feature vectors, keyword feature vectors, and intent feature vectors. Afterwards, the target training text feature vector is input into the first classifier to obtain the target training text probability distribution, the keyword feature vector is input into the second classifier to obtain the keyword probability distribution, and the intent feature vector is input into the third classifier to obtain the intent probability distribution.

[0138] By adopting the above technical solution, the model is trained to obtain a text matching model by comprehensively considering the probability distribution of the target training text, the matching results of the training text pairs, the probability distribution of keywords, and the probability distribution of intent, which further improves the accuracy of the text matching model. In addition, the use of multiple classifiers can obtain the probability distribution of the target training text, the probability distribution of keywords, and the probability distribution of intent in parallel without increasing the training time.

[0139] In one embodiment, step S128 trains the initial model according to the target training text probability distribution, the matching results included in the training text pair, the keyword probability distribution, and the intention probability distribution to obtain a text matching model, which may further include:

[0140] Step (1): Determine the target training text loss according to the target training text probability distribution and the matching results included in the training text pair. For example, a cross entropy loss function is used to calculate the target training text probability distribution and the matching results included in the training text pair to obtain a cross entropy loss, that is, the target training text loss.

[0141] Step (2): Determine the keyword information classification loss based on the keyword probability distribution and the target training text probability distribution, and determine the intent information classification loss based on the intent probability distribution and the target training text probability distribution.

[0142] Optionally, the divergence distance between the keyword probability distribution and the target training text probability distribution may be determined as the keyword information classification loss, and the divergence distance between the intent probability distribution and the target training text probability distribution may be determined as the intent information classification loss.

[0143] For example, KL divergence or bidirectional KL divergence can be used as the divergence distance. Take the bidirectional KL divergence between the keyword probability distribution and the target training text probability distribution as the keyword information classification loss as an example for explanation. The calculation formula of the bidirectional KL divergence is:

[0144]

[0145] in, It represents the bidirectional KL divergence between the keyword probability distribution and the target training text probability distribution, that is, the keyword information classification loss. P represents the keyword probability distribution, and Q represents the target training text probability distribution.

[0146] It should be understood that the method of determining the intent information classification loss is similar to the method of determining the keyword information classification loss, and the present disclosure will not elaborate on this.

[0147] Step (3): Train the initial model according to the target training text loss, keyword information classification loss and intent information classification loss to obtain a text matching model.

[0148] In a possible implementation, corresponding loss thresholds are preset for the target training text loss, keyword information classification loss, and intent information classification loss, respectively. When the target training text loss is less than its corresponding loss threshold, the keyword information classification loss is less than its corresponding loss threshold, and the intent information classification loss is less than its corresponding loss threshold, the training is terminated, and the initial model after the training is determined as the text matching model.

[0149] In another possible embodiment, when the sum of the target training text loss, the keyword information classification loss and the intent information classification loss is less than a preset loss threshold, the training is terminated and the initial model after the training is determined as the text matching model.

[0150] By adopting the above technical solution, during the model training process, the divergence distance is used to shorten the distance between the target training text probability distribution and the keyword probability distribution and the intention probability distribution. In this way, the target training text probability distribution output by the first classifier can accurately represent whether the text matches, that is, the accuracy of the text matching model is further improved.

[0151] It should be understood that during the model training process, the divergence distance is used to shorten the distance between the target training text probability distribution and the keyword probability distribution and the intent probability distribution, so that the target training text probability distribution output by the first classifier can accurately characterize whether the sentences in the text pair match. Therefore, when used online, it is possible to determine whether the two sentences in the text pair match based on the probability distribution output by the first classifier, and then the text matching model can only include the pre-trained language model and the first classifier to simplify the structure of the text matching model.

[0152] Based on the same inventive concept, the present disclosure also provides a text matching method. The text matching method provided by the present disclosure is described below.

[0153] Figure 4FIG. 1 is a flowchart of a text matching method according to an exemplary embodiment. Figure 4 As shown, the text matching method may include the following steps.

[0154] In step S41, a text pair to be matched is obtained, where the text pair to be matched includes a first sentence to be matched and a second sentence to be matched.

[0155] In step 42, it is determined whether the first sentence to be matched and the second sentence to be matched match according to the pair of texts to be matched and a preset text matching model.

[0156] The text matching model is generated according to the training method of the text matching model described above.

[0157] By adopting the above technical solution, whether the first sentence to be matched and the second sentence to be matched are matched is determined based on a more accurate text matching model, thereby improving the accuracy of text matching and enhancing the user experience.

[0158] Optionally, the text matching model includes a pre-trained language model and a first classifier, and the specific implementation of the above step S42 is as follows: convert the format of the text pair to be matched to obtain a target text that conforms to the text format corresponding to the pre-trained language model; input the target text into the pre-trained language model to obtain a target text feature vector output by the pre-trained language model; input the target text feature vector into the first classifier to obtain a target text probability distribution output by the first classifier, and the target text probability distribution can indicate whether the first sentence to be matched matches the second sentence to be matched; determine the matching result of the first sentence to be matched and the second sentence to be matched according to the target text probability distribution. Among them, the text format corresponding to the text matching model is as described above, and this disclosure will not elaborate on this.

[0159] For example, taking the target text probability distribution as a binomial distribution as an example, that is, the target text probability distribution can indicate the matching probability of the first sentence to be matched and the second sentence to be matched and the non-matching probability of the first sentence to be matched and the second sentence to be matched. When the matching probability is greater than the preset probability value, the matching result of the first sentence to be matched and the second sentence to be matched is determined to be a match, otherwise, the matching result of the first sentence to be matched and the second sentence to be matched is determined to be a non-match.

[0160] It should be understood that during the model training process, since the divergence distance is used to shorten the distance between the target training text probability distribution and the keyword probability distribution and the intent probability distribution, at the end of the training, the probability distribution output by the first classifier of the text matching model can accurately characterize whether the text matches. Therefore, in the present disclosure, whether the first sentence to be matched and the second sentence to be matched are matched is determined only based on the target text probability distribution output by the first classifier, without considering the keyword probability distribution and the intent probability distribution. Therefore, there is no need to annotate the keyword information and the intent information in the text pair to be matched, which reduces the noise introduced by the additional information. Moreover, there is no need to add additional computing operations in the text matching model. Only the pre-trained language model and the first classifier can be used to identify whether the two sentences match, thereby improving the running speed of the text model.

[0161] Based on the same inventive concept, the present disclosure also provides a training device for a text matching model. Figure 5 FIG. 1 is a block diagram of a text matching model training device according to an exemplary embodiment. Figure 5 As shown, the text matching model training device 500 may include:

[0162] A first acquisition module 501 is configured to acquire a plurality of training text pairs, wherein each of the training text pairs includes a first training text, a second training text, and a matching result, wherein the first training text includes a first sentence, first keyword information, and first intent information, and the second training text includes a second sentence, second keyword information, and second intent information, and the matching result is used to indicate whether the first sentence and the second sentence match;

[0163] The training module 502 is configured to train the initial model according to the training text pairs to obtain a text matching model.

[0164] Optionally, the initial model includes a pre-trained language model and a first classifier; the training module 502 includes:

[0165] A first conversion submodule is configured to, for each of the training text pairs, perform format conversion on the first training text and the second training text included in the training text pair to obtain a target training text that conforms to the text format corresponding to the pre-trained language model;

[0166] A first input submodule is configured to input the target training text into the pre-trained language model to obtain a target training text feature vector output by the pre-trained language model;

[0167] A second input submodule is configured to input the target training text feature vector into the first classifier to obtain a target training text probability distribution output by the first classifier, wherein the target training text probability distribution can indicate whether the first sentence matches the second sentence;

[0168] The first training submodule is configured to train the initial model according to the target training text probability distribution and the matching results included in the training text pair to obtain a text matching model.

[0169] Optionally, the training module 502 further includes:

[0170] a first generating submodule, configured to generate a keyword probability distribution according to the first keyword information and the second keyword information included in the target training text, wherein the keyword probability distribution can indicate whether the first keyword information matches the second keyword information; and

[0171] A second generating submodule is configured to generate an intent probability distribution according to the first intent information and the second intent information included in the target training text, wherein the intent probability distribution can indicate whether the first intent information matches the second intent information;

[0172] The first training submodule includes:

[0173] The second training submodule is configured to train the initial model according to the target training text probability distribution, the matching results included in the training text pairs, the keyword probability distribution and the intention probability distribution to obtain a text matching model.

[0174] Optionally, the initial model further includes a second classifier and a third classifier; and the first generation submodule includes:

[0175] A third generating submodule is configured to generate a keyword feature vector of the target training text according to the first keyword information, the second keyword information and the pre-trained language model included in the target training text;

[0176] A third input submodule is configured to input the keyword feature vector into the second classifier to obtain a keyword probability distribution generated by the second classifier;

[0177] The second generation submodule includes:

[0178] a fourth generating submodule, configured to generate an intent feature vector of the target training text according to the first intent information, the second intent information and the pre-trained language model included in the target training text;

[0179] The fourth input submodule is configured to input the intention feature vector into the third classifier to obtain the intention probability distribution generated by the third classifier.

[0180] Optionally, the second training submodule includes:

[0181] A first determination submodule is configured to determine a target training text loss according to the target training text probability distribution and the matching result included in the training text pair;

[0182] A second determination submodule is configured to determine a keyword information classification loss according to the keyword probability distribution and the target training text probability distribution, and to determine an intent information classification loss according to the intent probability distribution and the target training text probability distribution;

[0183] The third training submodule is configured to train the initial model according to the target training text loss, the keyword information classification loss and the intent information classification loss to obtain a text matching model.

[0184] Optionally, the second determination submodule is configured to: determine the divergence distance between the keyword probability distribution and the target training text probability distribution as the keyword information classification loss, and determine the divergence distance between the intent probability distribution and the target training text probability distribution as the intent information classification loss.

[0185] Optionally, the third generating submodule is configured to: generate a keyword mask structure in the pre-trained language model according to the first sentence, the second sentence, the first keyword information and the second keyword information included in the target training text; generate a keyword feature vector of the target training text according to the target training text and the keyword mask structure;

[0186] The fourth generation submodule is configured to: generate an intent mask structure in the pre-trained language model according to the first sentence, the second sentence, the first intent information and the second intent information included in the target training text; and generate an intent feature vector of the target training text according to the target training text and the intent mask structure.

[0187] Optionally, the first acquisition module 501 includes:

[0188] A first acquisition submodule is configured to acquire a plurality of sentence pairs, wherein the sentence pairs include a first sentence and a second sentence;

[0189] The third determination submodule is configured to, for each sentence pair, identify first keyword information and first intent information from the first sentence, identify second keyword information and second intent information from the second sentence, and obtain matching results of the first sentence and the second sentence, determine the first sentence, the first keyword information and the first intent information as a first training text, determine the second sentence, the second keyword and the second intent information as a second training text, and determine the first training text, the second training text and the matching results as a set of training text pairs.

[0190] Regarding the training device for the text matching model in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0191] Based on the same inventive concept, the present disclosure also provides a text matching device. Figure 6 FIG. 1 is a block diagram of a text matching device according to an exemplary embodiment. Figure 6 As shown, the text matching device 600 includes:

[0192] The second acquisition module 601 is configured to acquire a text pair to be matched, wherein the text pair to be matched includes a first sentence to be matched and a second sentence to be matched;

[0193] The first determination module 602 is configured to determine the matching result of the first sentence to be matched and the second sentence to be matched according to the text pair to be matched and a preset text matching model; wherein the text matching model is generated according to the training method of the matching model provided in the present disclosure.

[0194] Optionally, the text matching model includes a pre-trained language model and a first classifier; the first determination module 602 includes:

[0195] A second conversion submodule is configured to perform format conversion on the to-be-matched text pair to obtain a target text that conforms to the text format corresponding to the pre-trained language model;

[0196] A fifth input submodule is configured to input the target text into the pre-trained language model to obtain a target text feature vector output by the pre-trained language model;

[0197] a sixth input submodule, configured to input the target text feature vector into the first classifier to obtain a target text probability distribution output by the first classifier, wherein the target text probability distribution can indicate whether the first sentence to be matched matches the second sentence to be matched;

[0198] The fourth determination submodule is configured to determine the matching result of the first sentence to be matched and the second sentence to be matched according to the probability distribution of the target text.

[0199] Regarding the text matching device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0200] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implement the steps of the text matching model training method and the text matching method provided by the present disclosure.

[0201] Figure 7 8 is a block diagram of an electronic device according to an exemplary embodiment. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0202] Reference Figure 7 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output interface 812 , a sensor component 814 , and a communication component 816 .

[0203] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the training method of the text matching model and the text matching method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0204] The memory 804 is configured to store various types of data to support operations on the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0205] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0206] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0207] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0208] The input / output interface 812 provides an interface between the processing component 802 and the peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0209] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the electronic device 800, the relative positioning of the components, such as the display and keypad of the electronic device 800, and the sensor assembly 814 can also detect the position change of the electronic device 800 or a component of the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0210] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0211] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute a text matching model training method and a text matching method.

[0212] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the above instructions can be executed by the processor 820 of the electronic device 800 to complete the training method of the text matching model and the text matching method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0213] In another exemplary embodiment, a computer program product is also provided, which includes a computer program that can be executed by a programmable device, and the computer program has a code portion for executing the above-mentioned text matching model training method and text matching method when executed by the programmable device.

[0214] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the present disclosure. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0215] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for training a text matching model, characterized in that: The training method comprises: Acquire multiple training text pairs, wherein each of the training text pairs includes a first training text, a second training text, and a matching result, the first training text includes a first sentence, first keyword information, and first intent information, the second training text includes a second sentence, second keyword information, and second intent information, and the matching result is used to indicate whether the first sentence and the second sentence match; The initial model is trained according to the training text pairs to obtain a text matching model.

2. The training method according to claim 1, characterized in that: The initial model includes a pre-trained language model and a first classifier; The initial model is trained according to the training text to obtain a text matching model, including: For each of the training text pairs, format conversion is performed on the first training text and the second training text included in the training text pair to obtain a target training text that conforms to the text format corresponding to the pre-trained language model; Inputting the target training text into the pre-trained language model to obtain a target training text feature vector output by the pre-trained language model; Inputting the target training text feature vector into the first classifier to obtain a target training text probability distribution output by the first classifier, wherein the target training text probability distribution can indicate whether the first sentence matches the second sentence; The initial model is trained according to the target training text probability distribution and the matching results included in the training text pairs to obtain a text matching model.

3. The training method according to claim 2, characterized in that: The training of the initial model according to the training text pair to obtain a text matching model further includes: generating a keyword probability distribution according to the first keyword information and the second keyword information included in the target training text, wherein the keyword probability distribution can indicate whether the first keyword information matches the second keyword information; and generating an intent probability distribution according to the first intent information and the second intent information included in the target training text, wherein the intent probability distribution can indicate whether the first intent information matches the second intent information; The step of training the initial model according to the target training text probability distribution and the matching results of the training text pairs to obtain a text matching model comprises: The initial model is trained according to the target training text probability distribution, the matching results included in the training text pairs, the keyword probability distribution and the intention probability distribution to obtain a text matching model.

4. The training method according to claim 3, characterized in that: The initial model also includes a second classifier and a third classifier; The generating a keyword probability distribution according to the first keyword information and the second keyword information included in the target training text includes: Generate a keyword feature vector of the target training text according to the first keyword information, the second keyword information and the pre-trained language model included in the target training text; Inputting the keyword feature vector into the second classifier to obtain a keyword probability distribution generated by the second classifier; The generating the intent probability distribution according to the first intent information and the second intent information included in the target training text includes: Generate an intent feature vector of the target training text according to the first intent information, the second intent information and the pre-trained language model included in the target training text; The intention feature vector is input into the third classifier to obtain the intention probability distribution generated by the third classifier.

5. The training method according to claim 3 or 4, characterized in that: The initial model is trained according to the target training text probability distribution, the matching results included in the training text pair, the keyword probability distribution and the intention probability distribution to obtain a text matching model, including: Determining a target training text loss according to the target training text probability distribution and the matching results included in the training text pair; Determine a keyword information classification loss according to the keyword probability distribution and the target training text probability distribution, and determine an intent information classification loss according to the intent probability distribution and the target training text probability distribution; The initial model is trained according to the target training text loss, the keyword information classification loss and the intent information classification loss to obtain a text matching model.

6. The training method according to claim 5, characterized in that: The determining of the keyword information classification loss according to the keyword probability distribution and the target training text probability distribution, and the determining of the intent information classification loss according to the intent probability distribution and the target training text probability distribution, include: The divergence distance between the keyword probability distribution and the target training text probability distribution is determined as the keyword information classification loss, and the divergence distance between the intent probability distribution and the target training text probability distribution is determined as the intent information classification loss.

7. The training method according to claim 4, characterized in that: The step of generating a keyword feature vector of the target training text according to the first keyword information, the second keyword information and the pre-trained language model included in the target training text includes: Generating a keyword mask structure in the pre-trained language model according to the first sentence, the second sentence, the first keyword information, and the second keyword information included in the target training text; Generating a keyword feature vector of the target training text according to the target training text and the keyword mask structure; The step of generating the intent feature vector of the target training text according to the first intent information, the second intent information and the pre-trained language model included in the target training text includes: Generate an intent mask structure in the pre-trained language model according to the first sentence, the second sentence, the first intent information, and the second intent information included in the target training text; An intention feature vector of the target training text is generated according to the target training text and the intention mask structure.

8. The training method according to claim 1, characterized in that: The step of obtaining a plurality of training text pairs comprises: Acquire a plurality of sentence pairs, wherein the sentence pairs include a first sentence and a second sentence; For each sentence pair, first keyword information and first intention information are identified from the first sentence, second keyword information and second intention information are identified from the second sentence, and matching results of the first sentence and the second sentence are obtained, the first sentence, the first keyword information and the first intention information are determined as a first training text, the second sentence, the second keyword and the second intention information are determined as a second training text, and the first training text, the second training text and the matching results are determined as a set of training text pairs.

9. A text matching method, characterized in that: The text matching method comprises: Acquire a text pair to be matched, wherein the text pair to be matched includes a first sentence to be matched and a second sentence to be matched; According to the to-be-matched text pair and a preset text matching model, the matching results of the first to-be-matched sentence and the second to-be-matched sentence are determined; wherein the text matching model is generated according to the training method of the text matching model described in any one of claims 1-8.

10. The text matching method according to claim 9, characterized in that: The text matching model includes a pre-trained language model and a first classifier; The determining, according to the text to be matched and a preset text matching model, a matching result of the first sentence to be matched and the second sentence to be matched includes: Performing format conversion on the to-be-matched text pair to obtain a target text that conforms to the text format corresponding to the pre-trained language model; Inputting the target text into the pre-trained language model to obtain a target text feature vector output by the pre-trained language model; Inputting the target text feature vector into the first classifier to obtain a target text probability distribution output by the first classifier, wherein the target text probability distribution can indicate whether the first sentence to be matched matches the second sentence to be matched; According to the target text probability distribution, a matching result of the first sentence to be matched and the second sentence to be matched is determined.

11. A training device for a text matching model, characterized in that: The training device comprises: A first acquisition module is configured to acquire a plurality of training text pairs, wherein each of the training text pairs includes a first training text, a second training text, and a matching result, the first training text includes a first sentence, first keyword information, and first intent information, the second training text includes a second sentence, second keyword information, and second intent information, and the matching result is used to indicate whether the first sentence and the second sentence match; The training module is configured to train the initial model according to the training text pairs to obtain a text matching model.

12. A text matching device, characterized in that: The text matching device comprises: A second acquisition module is configured to acquire a to-be-matched text pair, wherein the to-be-matched text pair includes a first to-be-matched sentence and a second to-be-matched sentence; The first determination module is configured to determine the matching results of the first sentence to be matched and the second sentence to be matched according to the text pair to be matched and a preset text matching model; wherein the text matching model is generated according to the training method of the text matching model according to any one of claims 1-8.

13. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the text matching model training method as described in any one of claims 1 to 8, or to execute the text matching method as described in claim 9 or 10.

14. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps of the training method of the text matching model described in any one of 1-8 are implemented, or the steps of the text matching method described in claim 9 or 10 are implemented.