Model training methods, equipment and storage media

Abstract

This application provides a model training method, device, and storage medium. Specifically, by using the predicted relevance score between the first text and the positive sample in the training sample pair, and the pairing loss of the maximum predicted score of each sample in the sample set, it is equivalent to all samples in the sample set being paired with positive samples. Therefore, the accuracy of the trained text matching model in distinguishing between top-1 and negative samples close to top-1 can be improved, thereby contributing to improving the accuracy of top-1 in text matching. On the other hand, during model training, a pairing loss based on the relevance of training sample pairs is also introduced. This inherits the ranking relationship of the pairing method. For candidate texts containing partially overlapping classifications or having hierarchical relationships, it can improve the accuracy of ranking candidate texts containing partially overlapping classifications or having hierarchical relationships, thereby improving the accuracy of top-k in text matching.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a model training method, device and storage medium. Background Technology

[0002] Text matching is frequently encountered in people's daily lives, work, and studies. For example, in government affairs, citizens can submit requests to various government departments through multiple channels (such as public hotlines, online platforms, or letters). The government system needs to match the citizen's request information with the event type to quickly and accurately determine which government department should have jurisdiction over the event.

[0003] For example, in document organization and classification scenarios, a document organization system can match document content with document categories to quickly and accurately determine the category to which a document belongs. Similarly, in information retrieval, search engines need to match user-provided keywords with database content to quickly and accurately determine the information the user is looking for.

[0004] Regardless of the text matching scenario, higher accuracy in text matching leads to a better user experience. Therefore, improving the accuracy of text matching is a technical problem that those skilled in the art continue to research. Summary of the Invention

[0005] This application provides a model training method, apparatus, and storage medium to improve the accuracy of text matching using a text matching model.

[0006] This application provides a model training method, including:

[0007] Obtain multiple training sample pairs; the multiple training sample pairs include: multiple first texts and sampling sample sets corresponding to each of the multiple first texts; the sampling sample sets include: positive samples and negative samples of the first texts;

[0008] Obtain the comparison relationship between the correlations of the plurality of training sample pairs; the correlation of any training sample pair in the plurality of training sample pairs refers to the correlation between the first text contained in any training sample pair and the positive or negative samples of the first text contained in any training sample pair.

[0009] Using the text matching model to be trained, calculate a first prediction score of the correlation between the first text and the positive sample, and a second prediction score of the correlation between the first text and the negative sample;

[0010] Using the text matching model to be trained, the third prediction score of the relevance of the first training sample pair and the fourth prediction score of the relevance between the second training sample pairs are calculated respectively; the first training sample pair and the second training sample pair are two training sample pairs associated with the same comparison relationship.

[0011] Based on the first difference between the first predicted score and the maximum predicted score corresponding to the sampled sample set, and the second difference between the third and fourth predicted scores corresponding to the same comparison relationship, the model parameters of the text matching model to be trained are adjusted to obtain the target text matching model; wherein, the maximum predicted score corresponding to the sampled sample set is the maximum value between the first predicted score and the second predicted score.

[0012] This application embodiment also provides a computing device, including: a memory and a processor; wherein, the memory is used to store computer programs;

[0013] The processor is coupled to the memory and is used to execute the computer program to perform the steps in the above-described model training method.

[0014] This application also provides a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps in the above-described model training method.

[0015] In this embodiment of the application, by using the prediction score of the correlation between the first text and the positive sample in the training sample pair and the pairing loss of the maximum prediction score of each sample in the sampled sample set, the relative size of the ranking within the sampled sample set is guaranteed. This is equivalent to all samples in the sampled sample set being paired with positive samples. Therefore, the accuracy of the trained text matching model in distinguishing between top 1 and negative samples that are close to top 1 can be improved, which in turn helps to improve the accuracy of top 1 in text matching.

[0016] On the other hand, during model training, a comparison pairing loss based on the relevance of training sample pairs is introduced. This inherits the ranking relationship of the pairing method. Compared with the single-point method, for candidate texts that contain partially overlapping classifications or have hierarchical relationships, it can improve the accuracy of ranking candidate texts that contain partially overlapping classifications or have hierarchical relationships, thereby improving the accuracy of topk in text matching. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 A schematic flowchart illustrating the model training method provided in this application embodiment;

[0019] Figure 2 A flowchart illustrating the text matching method provided in this application embodiment;

[0020] Figure 3 A flowchart illustrating the event classification method provided in this application embodiment;

[0021] Figure 4 A schematic diagram of the structure of a computing device provided in an embodiment of this application. Detailed Implementation

[0022] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0023] For tasks such as determining the semantic category of a given text description, text classification algorithms are commonly used. Classification algorithms are well-suited for situations where the number of categories is small, the categories are relatively fixed, and there is no overlap between categories. However, in practical applications, when classifying text descriptions, the large number of text categories, potential overlap between categories, or hierarchical relationships can cause text classification algorithms to fail to accurately classify the text.

[0024] The method of determining the semantic classification of text description information based on text matching algorithms compares the text description information with the text category names to determine the category to which the text description information belongs. This method has advantages over text classification algorithms. Text matching algorithms are generally implemented by text matching models. The text to be matched and various texts in a text library are input into the text matching model. The text matching model calculates the relevance score between the text to be matched and the texts in the text library. Then, based on the relevance score between the text to be matched and the texts in the text library, the target text that matches the text to be matched can be determined from the text library.

[0025] Before online use, the text matching model needs to be trained. Traditional methods for training or learning text matching models include point-wise and pair-wise methods. Among them, the point-wise method outputs 1 for the correct text and 0 for all other texts, which is not effective for matching texts with partially overlapping or similar semantics among candidate texts.

[0026] The matching method represents the relative comparison between the correct text and other texts. It can rank multiple similar texts higher, but it does not limit the absolute score of the correct text. Therefore, the accuracy of top-1 matching is often not as good as point-wise matching. Here, top-1 refers to the text that ranks first when matched with the text to be matched.

[0027] For example, suppose the texts to be matched are Q1 and Q2; the correct texts that match Q1 and Q2 are D1 and D2, respectively. The texts in the text library are D1, D2, ..., Dn, where n represents the total number of texts in the text library.

[0028] For the single-point method, the constructed training sample pairs are (Q1, D1), (Q1, D2)...(Q1, Dn), and (Q2, D1), (Q2, D2)...(Q2, Dn). The relevance scores of the training sample pairs are shown in Table 1 below:

[0029] Table 1 Training sample pairs for the single-point method

[0030] Text to be matched Text in the text library Relevance score Q1 D1 1 Q1 D2 0 Q1 …… 0 Q2 D1 0 Q2 D2 1 Q2 …… 0

[0031] The loss function L1 of the single-point method can be expressed as:

[0032]

[0033] In equation (1), y represents the true relevance score of the training sample pair, which is the pre-labeled relevance score. Here, y = 1 or y = 0. For positive samples of the text to be matched, y = 1; for negative samples of the text to be matched, y = 0. This represents the predicted relevance score of the training sample pairs calculated by the text matching model to be trained.

[0034] Therefore, the text matching model trained by the single method outputs 1 for the correct text to be matched and 0 for all other texts. It does not perform well in matching texts with some overlapping or similar semantics among the candidate texts.

[0035] For the pairing method, multiple training sample pairs can be constructed. D1 is a positive sample of Q1; other texts in the text library are negative samples of Q1, i.e., (Q1, D1) is a positive sample pair; Q1 and other texts form a negative sample pair. D2 is a positive sample of Q2; other texts in the text library are negative samples of Q2, i.e., (Q2, D2) is a positive sample pair; Q2 and other texts form a negative sample pair. For the same text to be matched (e.g., Q1), the relevance score of the positive sample pair is greater than the relevance score of the negative sample pair; and there is also a comparison relationship in the relevance scores of the negative sample pairs. For example, in Table 2, (Q1, D2) > (Q1, D3), indicating that neither D2 nor D2 is the best match for Q1, but the relevance of D2 to Q1 is greater than the relevance of D3 to Q1, meaning that the matching effect of D2 to Q1 is better than that of D3.

[0036] Table 2. Comparison of Correlation between Paired Training Sample Pairs

[0037]

[0038] The loss function L2 of the pairwise method can be expressed as:

[0039] L2=max(0,(m-(S pos -S neg (2).

[0040] In equation (2), S pos S represents the relevance score of the positive sample pairs output by the text matching model to be trained; neg This represents the relevance score of the negative sample pairs output by the text matching model to be trained. m is a hyperparameter, a fixed value. For example, m = 0.5, etc. The loss function L2 takes values ​​of 0 and (m - (S...). pos -S neg The maximum value in the L2 loss function is used to ensure the relevance score S of positive sample pairs. pos The correlation score of the negative sample pair is greater than or equal to m.

[0041] Because the pairwise method does not limit the absolute score of the correct text, top-1 accuracy is often not as good as point-wise accuracy.

[0042] To improve the accuracy of text matching, in some embodiments of this application, the highest relevance score between positive samples and the first text is guaranteed by maximizing the sample set. At the same time, the relative size of the ranking within the sample set is guaranteed by the pairing loss of the predicted relevance score between the first text and the positive sample and the maximum predicted score of each sample in the sample set. This is equivalent to all samples in the sample set being paired with positive samples. Therefore, the accuracy of the trained text matching model in distinguishing between top 1 and negative samples that are close to top 1 can be improved, which in turn helps to improve the accuracy of top 1 in text matching.

[0043] On the other hand, during model training, a comparison pairing loss based on the relevance of training sample pairs is introduced. This inherits the ranking relationship of the pairing method. Compared with the single-point method, for candidate texts that contain partially overlapping classifications or have hierarchical relationships, it can improve the accuracy of ranking candidate texts that contain partially overlapping classifications or have hierarchical relationships, thereby improving the accuracy of topk in text matching.

[0044] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0045] It should be noted that the same reference numerals denote the same object in the following figures and embodiments. Therefore, once an object is defined in one figure or embodiment, it does not need to be discussed further in subsequent figures and embodiments.

[0046] Figure 1 This is a schematic flowchart illustrating the model training method provided in an embodiment of this application. Figure 1 As shown, the training method for this model mainly includes:

[0047] 101. Obtain multiple training sample pairs; the multiple training sample pairs include: multiple first texts and the sampling sample sets corresponding to each of the multiple first texts; the sampling sample sets include: positive samples and negative samples of the first texts.

[0048] 102. Obtain the correlation comparison relationship of training sample pairs; The correlation of any training sample pair A among multiple training sample pairs refers to the correlation between the first text contained in training sample pair A and the positive or negative samples of the first text contained in training sample pair A.

[0049] 103. Using the text matching model to be trained, calculate the first prediction score of the correlation between the first text and the positive sample, and the second prediction score of the correlation between the first text and the negative sample.

[0050] 104. Using the text matching model to be trained, calculate the third prediction score of the relevance of the first training sample pair and the fourth prediction score of the relevance of the second training sample pair respectively; the first training sample and the second training sample are two training sample pairs associated with the same comparison relationship.

[0051] 105. Based on the first difference between the first predicted score and the maximum predicted score corresponding to the sampled sample set, and the second difference between the third and fourth predicted scores corresponding to the same comparison relationship, adjust the model parameters of the text matching model to be trained to obtain the target text matching model. The maximum predicted score corresponding to the sampled sample set is the maximum value between the first and second predicted scores.

[0052] In this embodiment, to train the text matching model, multiple training sample pairs can be obtained in step 101. "Multiple" refers to two or more. In this embodiment, the specific number of training sample pairs is not limited. For the accuracy of the text matching model, the number of training sample pairs is generally hundreds, thousands, or even tens of thousands, or more. Multiple training sample pairs include: multiple first texts and their respective corresponding sample sets. The sample sets include: positive and negative samples of the first texts. Each training sample pair includes: the first text and the positive or negative sample corresponding to the first text contained in the training sample pair. In this embodiment, the first text is the text to be matched in the training sample pair; the sample set is used to provide candidate texts to match the text to be matched.

[0053] In this embodiment, the first text and the texts in the corresponding sample set are heterogeneous texts. Heterogeneous text is defined in contrast to homogeneous text. Homogeneous text refers to text whose language organization and descriptive information are of the same dimension. For example, text 1 and text 2 are both document content; or text 1 and text 2 are both used to describe the same event, etc., then text 1 and text 2 are homogeneous texts. Conversely, heterogeneous text refers to text whose language organization and descriptive information are not of the same dimension. For example, text 1 is document content, and text 2 is the category to which the document content belongs; or text 1 is used to describe the content of a certain event, and text 2 is the event category, etc.

[0054] The text matching model is applied in different scenarios, and the content of the first text in the training sample pairs, i.e., the texts in the sample set, varies. In some embodiments, the text matching model is applied in government affairs processing scenarios to classify events reported by citizens. Accordingly, the first text can be an event description text, used to describe the specific content of the event. Each text in the sample set can be an event category, i.e., the category to which the event belongs. For example, if the event description text is "My neighbor's dog barks every day, seriously affecting my rest, please coordinate a solution," the event category could be "pet noise," "neighborhood noise," "social noise," "medical issues," "medical insurance card issues," "housing provident fund issues," and "housing provident fund transfer," etc.

[0055] In other embodiments, the text matching model is used in document classification scenarios. Accordingly, the first text may be the content of a document; the texts in the sample set may be document categories, which are collections of things that have one or more common attributes in document classification.

[0056] In this application, the specific implementation method for obtaining training sample pairs is not limited. In some embodiments, multiple first texts can be obtained through offline collection; and a sampling sample set can be constructed for the first texts according to the application scenario. Alternatively, historical texts to be matched in the application scenario of the text matching model can be obtained as first texts; and historical texts matched by the historical texts to be matched in the application scenario can be obtained as positive samples; then, historical texts matched by other historical texts to be matched can be obtained as negative samples, etc.

[0057] For example, in a text classification scenario, the first text can be an event description text; the positive and negative samples in the sample set represent event categories. Then, historical event description texts from a text classification scenario (such as a government affairs processing scenario) can be obtained as the first text; and the historical event categories to which each historical event description text is classified can be obtained from this scenario as positive samples for that historical event description text; subsequently, other event categories can be obtained from the event category library as negative samples for that historical event description text. The positive and negative samples of the historical event description text constitute the sample set for that historical event description text. Historical event description texts and positive samples form positive sample pairs; historical event description texts and each negative sample form negative sample pairs; positive and negative sample pairs constitute training sample pairs.

[0058] Of course, in some embodiments, the positive and negative samples of the first text can also be manually labeled. To reduce the cost of manual labeling, the positive samples of the first text can be manually labeled, while the negative samples of the first text can be automatically collected using negative sample sampling. For example, other texts besides the positive samples can be collected from a text library as negative samples of the first text. For text classification scenarios (such as government affairs processing), the text library can be an event category library.

[0059] The above embodiments are merely illustrative examples of how to obtain training sample pairs, and are not intended to limit the scope of the invention.

[0060] In this embodiment, to make the text matching model applicable to scenarios where candidate texts partially overlap or have a hierarchical relationship of inclusion, step 102 involves obtaining a comparison of the relevance of multiple training sample pairs. The relevance of any training sample pair A refers to the relevance between the first text contained in training sample pair A and the positive or negative samples of that first text. The comparison of relevance refers to a comparison of the magnitude of the relevance.

[0061] In event classification scenarios, candidate text can be event categories. Event categories may partially overlap or have a hierarchical relationship of inclusion and being included. Hierarchical relationships exist between categories. A parent category can be divided into subcategories according to different dimensions; the relationship between parent and subcategories is the hierarchical relationship. A parent category is the superior category of a subcategory, and a subcategory is the subordinate category of that parent category. Specifically, an N-level category can be divided into one or more (N+1)-level categories, where the N-level category is the parent category of the (N+1)-level categories, and the (N+1)-level categories are the subcategories of the N-level category. N is a positive integer.

[0062] For example, event categories could include "pet noise," "neighborhood noise," "social noise," "medical issues," "medical insurance card issues," "housing provident fund issues," and "housing provident fund transfers," etc. Among these, "pet noise" and "neighborhood noise" have semantic overlap; that is, pet noise can also be neighborhood noise, and vice versa. Both "pet noise" and "neighborhood noise" fall under the category of "social noise." To improve text matching accuracy, it is necessary to use a text matching model to select the top k most relevant texts from these partially overlapping candidate texts or those with a hierarchical relationship of inclusion. Here, top k refers to the k texts with the highest relevance to the text to be matched, matched by the text matching model. k is a positive integer, i.e., k ≥ 1.

[0063] Based on this, in step 102 of this embodiment, a comparison of the relevance of training sample pairs can be introduced. This comparison of the relevance of training sample pairs mainly refers to a comparison of the magnitude of their relevance. For example, the first text Q in the training sample pair... iThe first text sample set includes: "My neighbor's dog barks every day, seriously affecting my rest. Please coordinate a solution." The sample set includes: "pet noise," "neighborhood noise," "social noise," "medical issues," "medical insurance card issues," "housing provident fund issues," and "housing provident fund transfers," etc. Therefore, the correlation comparison relationship between the training sample pairs can be expressed as: Q i The correlation with "pet noise" is greater than Q. i Correlation with "neighborhood noise"; Q i The correlation with "pet noise" is greater than Q. i Correlation with "social noise"; Q i The correlation with "social noise" is greater than Q. i Relevance to "medical issues", etc.

[0064] In text classification scenarios, to address the issue of overlapping text categories or hierarchical relationships involving inclusion and being included, which can lead to low accuracy in determining the top-k categories of the text to be matched by the text matching model, the negative samples in the sampling sample set of the first text can include: a first negative sample belonging to the same category and level as the positive sample; a second negative sample belonging to the same category but at a different level as the positive sample; and a third negative sample belonging to a different category as the positive sample. Specifically, the first negative sample can be a sample belonging to the same first-level category and level as the positive sample. The second negative sample can be a sample belonging to the same first-level category but at a different level as the positive sample. The third negative sample is a sample belonging to a different first-level category than the positive sample.

[0065] In this context, a first-level category refers to the primary classification of text, representing the largest granularity of text classification. First-level categories have no parent category. They are independent of each other and do not overlap. For example, in handling government affairs, the first-level category of an event might be its broad category, such as noise pollution, social security, housing provident fund, or urban construction. Similarly, in document classification, the first-level categories can be divided into natural sciences, social sciences, and humanities. In patent classification, the first-level category of a document can be the category corresponding to the main patent classification number. For instance, the main classification number might be categorized as follows: A for necessities of human life; B for operations and transportation; C for chemistry and metallurgy; D for textiles and papermaking; E for fixed buildings; F for mechanical engineering; G for physics; and H for electricity.

[0066] Accordingly, the correlation comparison of training sample pairs may include: the correlation comparison of training sample pairs at the same level and the correlation comparison of training sample pairs across levels. Specifically, the correlation comparison of training sample pairs at the same level may include: the correlation between a positive sample and the first text, and the correlation between a positive sample and the first negative sample and the first text; and the correlation comparison between each negative sample at the same level in the second and third negative samples and the first text.

[0067] Among them, the correlation between the positive sample and the first text is greater than the correlation between the first negative sample and the first text; the correlation between each negative sample at the same level in the second and third negative samples and the first text is as follows: the correlation between negative sample 1 in the second negative sample and the first text is greater than the correlation between negative sample 2 in the third negative sample and the first text. Negative sample 1 and negative sample 2 are negative samples at the same level in the second and third negative samples, respectively.

[0068] Accordingly, the comparison of the relevance of training sample pairs across levels can include: the relevance between a positive sample and the first text, and the comparison of the relevance between a second negative sample and the first text at different levels within the same category. The relevance between a positive sample and the first text is greater than the relevance between a second negative sample and the first text.

[0069] For example, the first text might be "My neighbor's dog barks every day, seriously affecting my rest. Please help resolve this." The positive sample is "pet noise." The category hierarchy for "pet noise" is: Noise → Social noise → Neighborhood noise, pet noise, etc., with the category hierarchy decreasing progressively. "Neighborhood noise" and "pet noise" belong to the same level of noise. Correspondingly, negative samples for this first text could include: "Neighborhood noise" (the first negative sample belonging to the same level of the first-level category as the positive sample), "Social noise," and "Noise" (the second negative sample belonging to a different level of the same level of the first-level category as the positive sample). Of course, negative samples for the first text could also include: social security issues, social security card issues, housing provident fund issues, and housing provident fund transfers, etc., which belong to different level of the first text (i.e., the aforementioned third negative sample).

[0070] The correlation comparison between training sample pairs at the same level can include: S(pet noise) > S(neighborhood noise), S(social life noise) > S(medical security), and S(social life noise) > S(housing provident fund issue), etc. Here, S(X) represents the correlation between text X and the first text, or it can represent the correlation score between text X and the first text. Specifically, S(pet noise) > S(neighborhood noise) is the correlation between a positive sample and the first text, compared to the correlation between the first negative sample and the first text; S(social life noise) > S(medical security) and S(social life noise) > S(housing provident fund issue) represent the correlation comparison between each negative sample at the same level in the second and third negative samples and the first text, respectively.

[0071] Accordingly, the correlation comparison between training sample pairs across levels can include: S(pet noise) > S(social noise), etc. S(pet noise) > S(social noise) represents the correlation between a positive sample and the first text, and the correlation comparison between a second negative sample and the first text at a different level within the same first-level category.

[0072] In the embodiment where the negative samples of the first sample set mentioned above include: a first negative sample belonging to the same level of the same first-level category as the positive sample, a second negative sample belonging to a different level of the same first-level category as the positive sample, and a third negative sample belonging to a different first-level category as the positive sample, when obtaining training sample pairs in step 101, the first text and the category name of the target level to which the first text belongs can be obtained; wherein, the category name of the target level to which the first text belongs is pre-labeled. Further, from the pre-established category text library, the first category name of other levels belonging to the same first-level category as the category name of the target level, the second category name of the same level belonging to the same first-level category as the category name of the target level, and the third category name belonging to a different first-level category as the category name of the target level can be obtained; then, using the category level name of the first text as the positive sample of the first text, and using the first category name, the second category name, and the third category name as the negative samples of the first text, sample pairs between the first text and the positive samples and sample pairs between the first text and each negative sample are constructed to obtain training sample pairs.

[0073] In this application embodiment, the specific implementation of obtaining the comparison relationship of the correlation of training sample pairs is not limited. In some embodiments, the comparison relationship of the correlation of training sample pairs may be manually labeled. Accordingly, step 102 is implemented as: obtaining the comparison relationship of the correlation of pre-labeled training sample pairs.

[0074] In other embodiments, the correlation comparison relationship of training sample pairs can also be automatically determined by the computing device. For embodiments where the computing device automatically determines the correlation comparison relationship of training sample pairs, the category name of the target level to which the first text belongs can be pre-labeled, and the category names of the text and the hierarchical relationship between the category names are pre-established, i.e., known. The category hierarchy of the text is: first-level category → second-level category → ... → n-level category, with the hierarchy decreasing progressively and the granularity corresponding to each level becoming increasingly refined. For example, the hierarchical relationship of the noise category is: noise (first-level category) → social noise (second-level category) → neighborhood noise, pet noise (third-level category), etc., with the category hierarchy decreasing progressively.

[0075] Accordingly, based on the category name of the target level to which the first text belongs, and the pre-established hierarchical relationship between category names, the parent category name of the target level under the same first-level category, the second category name belonging to the same first-level category and at the same level as the target level, and the third category name belonging to a different first-level category from the negative samples of the first text can be obtained. Furthermore, it can be determined that the relevance between the category name of the target level to which the first text belongs and the first text is greater than that of the parent category name of the target level under the same first-level category; and that the relevance between the category name of the target level to which the first text belongs and the first text is greater than that of the second category name; furthermore, it can be determined that the relevance between the second category name and the first text is greater than that of the third category name; and that the relevance between the category name of the target level to which the first text belongs and the first text is greater than that of the third category name.

[0076] The correlation comparison among training sample pairs at the same level includes: the correlation between the category name of the target level to which the first text belongs and the first text is greater than the correlation between the second category name and the first text; the correlation between the second category name and the first text is greater than the correlation between the category name of the target level in the third category name and the first text; and the correlation between the category name of the target level to which the first text belongs and the first text is greater than the correlation between the target level in the third category name and the first text.

[0077] The correlation comparison of training sample pairs across levels includes: the correlation between the category name of the target level to which the first text belongs and the first text is greater than the correlation between the parent category name of the target level under the same first-level category; the correlation between the second category name and the first text is greater than the correlation between the category name of the third category (not at the target level) and the first text; and the correlation between the category name of the target level to which the first text belongs and the first text is greater than the correlation between the third category (not at the target level) and the first text.

[0078] The above embodiments illustrating the comparative relationship of obtaining the correlation between training sample pairs are merely illustrative and do not constitute a limitation.

[0079] Furthermore, in step 103, the text matching model to be trained can be used to calculate a first prediction score S of the relevance between the first text and its positive samples. pos The second prediction score S, which is the correlation between the first text and its negative samples. j Where j = 1, 2, ..., k, and k represents the number of samples in the sample set. j ∈ Φ k Φ k This represents a training sample pair consisting of the first text and the sampled sample set.

[0080] In this embodiment, the text matching model to be trained is the text matching model of the current step, which can predict the relevance score between texts, that is, calculate the predicted relevance score between texts. The text matching model to be trained can be the initial model of model training; it may also be a model that has been learned at a certain stage using a step-by-step learning method, etc.

[0081] Ladder-style learning refers to the learning process of first obtaining a large pre-trained model based on unsupervised pre-training on general data, then further training a domain-specific pre-trained model based on historical data within a broad domain (data constructed by humans or machines, which may contain many errors), and finally fine-tuning a high-quality model based on accurate data labeled by humans in a specific domain.

[0082] In this embodiment, the text matching model can be a neural network model, and the specific implementation of the text matching model is not limited. The text matching model can be a Bidirectional Encoder Representations from Transformers (BERT) model, a Generative Pre-training (GPT) model, or an Embedding from Language Model (ELMo) model, etc.

[0083] The Transformer is a deep learning component that, unlike Recurrent Neural Networks (RNNs) which depend on sequence, can process entire sequences in parallel. This allows for the analysis of larger datasets and accelerates model training. The Transformer uses an attention mechanism to collect contextual information about words and encodes it as rich vectors representing that context, thus processing (rather than processing individually) words related to all other words in a sentence simultaneously. The Transformer model learns how to derive the meaning of a given word from every other word in the sentence. BERT is essentially a bidirectional feature encoder for the Transformer, representing the current input as feature representations (or encoding, or representation), i.e., word vector representations.

[0084] In step 104 of this embodiment, the text matching model to be trained can also be used to calculate the third prediction score S of the relevance of the first training sample pairs associated with the same comparison relation. a The fourth prediction score S of the relevance of the second training sample pair associated with the same comparison relationship. b The first and second training sample pairs associated with the same comparison relationship are the two training sample pairs that have a comparison relationship in step 102. For example, if the comparison relationship is: the relevance between the category name of the target level to which the first text belongs and the first text is greater than the relevance between the second category name and the first text; then the first training sample pair can be the training sample pair consisting of the category name of the target level to which the first text belongs and the first text; the second training sample pair can be the training sample pair consisting of the second category name and the first text.

[0085] In step 104, the first training sample pair can be defined as a training sample pair with high correlation within the same comparison relationship; the second training sample pair consists of training sample pairs with low correlation within the same comparison relationship. That is, S... a >S b .

[0086] The text matching models to be trained in steps 104 and 103 are the same. The model training process is generally a step-by-step learning process, where the architecture of the model obtained at each stage is the same, but the model parameters differ. The text matching models to be trained in steps 103 and 104 can be text matching models from any stage.

[0087] After calculating the first predicted score of the correlation between the first text and the positive samples, and the second predicted score of the correlation between the first text and the negative samples, the maximum pairing method within the sample set can be used, that is, the maximum predicted score (maxS) of each sample within the sample set. jThe predicted score S of the correlation between the first text and the positive sample pos A comparison is made. Specifically, in step 105, the maximum predicted score (max S) between the second predicted score and the first predicted score can be used. j ) and the first predicted score S pos The first difference between them, and the third predicted score S corresponding to the same comparison relationship. a With the fourth predicted score S b The second difference between the two values ​​is used to adjust the model parameters of the text matching model to be trained in order to obtain the target text matching model.

[0088] In this embodiment, maximizing the sample set ensures the highest relevance score between the positive sample and the first text. At the same time, the pairing loss between the predicted relevance score between the first text and the positive sample and the maximum predicted score of each sample in the sample set ensures the relative size of the ranking within the sample set. This is equivalent to all samples in the sample set being paired with positive samples. Therefore, the accuracy of the trained text matching model in distinguishing between top 1 and negative samples close to top 1 can be improved, which in turn helps to improve the accuracy of top 1 in text matching.

[0089] On the other hand, during model training, a comparison pairing loss based on the relevance of training sample pairs is introduced. This inherits the ranking relationship of the pairing method. Compared with the single-point method, for candidate texts that contain partially overlapping classifications or have hierarchical relationships, it can improve the accuracy of ranking candidate texts that contain partially overlapping classifications or have hierarchical relationships, thereby improving the accuracy of topk in text matching.

[0090] In this embodiment, a first loss value is determined based on a set first difference threshold m1 and the first difference (the difference between the maximum predicted score and the first predicted score in the second and first predicted scores), where the first loss value is greater than or equal to 0. A second loss value is then determined based on a set second difference threshold m2 and the second difference (the difference between the third and fourth predicted scores corresponding to the same comparison relationship), where the second loss value is greater than or equal to 0. The first loss value being greater than or equal to 0 ensures that the difference between the predicted score of a positive sample and the maximum predicted score corresponding to the sample set is at least m1, thus maximizing the sample set to guarantee the highest predicted score for the positive sample. The second loss value being greater than or equal to 0 ensures that the difference between the predicted scores of highly correlated training sample pairs and less correlated training sample pairs in the same comparison relationship is at least m2, potentially enabling accurate ranking of candidate texts with multiple granularities.

[0091] Specifically, if the difference between the set first difference threshold m1 and the first difference is greater than 0, then the first loss value is equal to the difference between the set first difference threshold m1 and the first difference; if the difference between the set first difference threshold m1 and the first difference is less than or equal to 0, then the first loss value is equal to 0. Correspondingly, if the difference between the set second difference threshold m2 and the second difference is greater than 0, then the second loss value m2 is equal to the difference between the set second difference threshold m2 and the second difference; if the difference between the set second difference threshold m2 and the second difference is less than or equal to 0, then the second loss value is equal to 0.

[0092] Furthermore, the model parameters of the text matching model to be trained can be adjusted with the goal of minimizing the sum of the first and second loss values ​​to obtain the target text matching model.

[0093] The loss function corresponding to the first loss value can be expressed as:

[0094]

[0095] The loss function corresponding to the second loss value can be expressed as:

[0096] L a =max(0, m2-(S) a -S b (4).

[0097] The loss function corresponding to the sum of the first and second loss values ​​is:

[0098] L3 = L m +L a (5).

[0099] In the above formula (3), m1 is the set first difference threshold, which is a fixed value. S pos The first predicted score is the correlation between the first text and the positive sample; maxS j Let S be the maximum predicted score corresponding to the sample set. The maximum predicted score corresponding to the sample set is the maximum of the first predicted score and the second predicted score; the second predicted score is the predicted score of the correlation between the first text and each negative sample in the sample set. Equation (3) above can make the predicted score S of the positive sample... pos The maximum prediction score maxS corresponding to the sample set j The difference is at least m1, and the highest prediction score for positive samples can be guaranteed by maximizing the sampling set.

[0100] In equation (4) above, m2 is the set second difference threshold, which is a fixed value. a The third prediction score represents the relevance of the first training sample pair associated with the same comparison relation; Sb The fourth prediction score represents the relevance of the first training sample pair associated with the same comparison relation. Where S... a >S b Equation (4) above can ensure that the difference between the predicted scores of training sample pairs with high correlation in the same comparison relationship and the predicted scores of training sample pairs with low correlation in the same comparison relationship is at least m2, which may effectively and accurately rank candidate texts with multiple granularities.

[0101] The comparison relationships between training sample pairs include: comparison relationships of the relevance of training sample pairs at the same level and comparison relationships of the relevance of training sample pairs across levels. Since comparison relationships are constructed for texts with partial semantic overlap or containing or having hierarchical relationships during the model training phase, the trained text matching model can solve the problem of classifying candidate texts that contain partial overlap or have hierarchical relationships, and can improve the accuracy of ranking candidate texts that contain partial overlap or have hierarchical relationships.

[0102] In this embodiment, gradient descent can be used to adjust the model parameters of the text matching model to be trained. Specifically, assuming θ is the set of model parameters of the text matching model to be trained, an initial value is randomly set. Further, the partial derivative of each model parameter is calculated to obtain a set of partial derivatives, i.e., a gradient set. Further, new model parameters can be obtained by updating the gradients corresponding to each model parameter in the gradient set, and this process is repeated until the model parameters minimize the loss function. The text matching model corresponding to the new model parameters is the text matching model to be trained in the next iteration.

[0103] After training, the text matching model can be used for text matching. Specifically, such as... Figure 2 As shown, text matching methods may include:

[0104] 201. Obtain the text to be matched.

[0105] 202. Input the text to be matched and each text in the text library into the target text matching model to obtain the correlation between the text to be matched and each text in the text library.

[0106] 203. Based on the correlation between the text to be matched and the texts in the text library, determine the target text corresponding to the text to be matched from the text library.

[0107] For the text to be matched obtained in step 201, in step 202, the text to be matched and each text in the text library can be input into the target text matching model to obtain the correlation between the text to be matched and each text in the text library.

[0108] Furthermore, in step 203, the target text corresponding to the text to be matched can be determined from the text library based on the relevance between the text to be matched and the texts in the text library. The target text corresponding to the text to be matched can be the text ranked in the top k of relevance to the text to be matched, where k is a positive integer.

[0109] That is, the target text can be the text with the highest relevance to the text to be matched, or the text ranked in the kth position of relevance to the text to be matched, etc.

[0110] In this embodiment, during the training phase, the target text matching model uses a pairing loss calculated by combining the predicted relevance score of the first text in the training sample pair with the positive sample and the maximum predicted score of each sample in the sampled sample set. This effectively pairs all samples in the sampled sample set with positive samples. Therefore, when using the target text to train the model for text matching, the accuracy of the trained text matching model in distinguishing between top-1 and negative samples close to top-1 can be improved, thus contributing to the improvement of the top-1 accuracy in text matching. Furthermore, during model training, a pairing loss based on the relevance of training sample pairs is also introduced. This inherits the ranking relationship of the pairing method. Compared to the single-point method, for candidate texts containing partially overlapping classifications or hierarchical relationships, the text matching model can improve the accuracy of ranking candidate texts containing partially overlapping classifications or hierarchical relationships during text matching, thereby improving the top-k accuracy in text matching.

[0111] The target text matching model provided in this application is applicable to various heterogeneous text matching scenarios, such as text classification scenarios. The following example illustrates this using event classification as an example. In an event classification scenario, the text to be matched can be event description text. Each text in the text library can be an event category. Correspondingly, the target text is the event category to which the event description text belongs. For example... Figure 3 As shown, event classification methods may include:

[0112] 301. Obtain the description text of the event to be processed.

[0113] 302. Input the event description text and the names of the event categories in the event category library into the target text matching model to obtain the correlation between the event description text and the names of the event categories in the event category library.

[0114] 303. Based on the correlation between the event description text and the name of the event category in the event category library, determine the target event category to which the event description text belongs from the event category library.

[0115] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 101 and 102 can be device A; or the execution subject of step 101 can be device A, and the execution subject of step 102 can be device B; and so on.

[0116] Furthermore, some processes described in the above embodiments and accompanying drawings include multiple operations that appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as 101, 102, etc., are merely used to distinguish different operations and do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel.

[0117] Accordingly, embodiments of this application also provide a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, cause one or more processors to perform the steps in the above-described model training method, and / or text matching method, and / or event classification method.

[0118] Figure 4 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application. Figure 4 As shown, the computing device may include: a memory 40a and a processor 40b; wherein the memory 40a is used to store computer programs.

[0119] Processor 40b is coupled to memory 40a for executing a computer program to: acquire multiple training sample pairs; the multiple training sample pairs include: multiple first texts and sampling sample sets corresponding to each of the multiple first texts; the sampling sample sets include: positive samples and negative samples of the first texts; acquire a comparison relationship between the relevance of the multiple training sample pairs; the relevance of any training sample pair in the multiple training sample pairs refers to: the relevance between the first text contained in any training sample pair A and the positive or negative samples of the first text contained in that training sample pair A; and, using the text matching model to be trained, calculate a first prediction score for the relevance between the first text and the positive samples, and a first prediction score for the relevance between the first text and the negative samples. The second prediction score of the relevance; using the text matching model to be trained, the third prediction score of the relevance of the first training sample pair and the fourth prediction score of the relevance of the second training sample pair are calculated respectively; the first training sample pair and the second training sample pair are two training sample pairs associated with the same comparison relation; and, based on the first difference between the first prediction score and the maximum prediction score corresponding to the sampled sample set, and the second difference between the third prediction score and the fourth prediction score corresponding to the same comparison relation, the model parameters of the text matching model to be trained are adjusted to obtain the target text matching model; wherein, the maximum prediction score corresponding to the sampled sample set is the maximum value of the first prediction score and the second prediction score.

[0120] Optionally, the negative samples in the sample set include: the first negative sample belonging to the same category and level as the positive sample, the second negative sample belonging to the same category but different level as the positive sample, and the third negative sample belonging to a different category as the positive sample.

[0121] The correlation comparison of training sample pairs includes: the correlation comparison of training sample pairs at the same level and the correlation comparison of training sample pairs across different levels.

[0122] The correlation comparison of training sample pairs at the same level includes: the correlation between positive samples and the first text, and the correlation between positive samples and the first negative samples and the first text; and the correlation comparison between each negative sample at the same level in the second and third negative samples and the first text.

[0123] The correlation comparison of training sample pairs across levels includes: the correlation between positive samples and the first text, and the correlation comparison between the second negative sample and the first text.

[0124] Optionally, the first text is the event description text; the positive and negative samples in the sample set are the event categories.

[0125] Optionally, when the processor 40b adjusts the model parameters of the text matching model to be trained based on the first difference between the first predicted score and the maximum predicted score corresponding to the sampled sample set, and the second difference between the third and fourth predicted scores corresponding to the same comparison relationship, it specifically performs the following: determining a first loss value based on the difference between the set first difference threshold and the first difference; the first loss value is greater than or equal to 0; determining a second loss value based on the difference between the set second difference threshold and the second difference; the second loss value is greater than or equal to 0; and adjusting the model parameters of the text matching model to be trained with the goal of minimizing the sum of the first loss value and the second loss value to obtain a target text matching model.

[0126] Furthermore, when determining the first loss value based on the difference between a set first difference threshold and a first difference, the processor 40b specifically performs the following: if the difference between the set first difference threshold and the first difference is greater than 0, then the first loss value is equal to the difference between the set first difference threshold and the first difference; if the difference between the set first difference threshold and the first difference is less than or equal to 0, then the first loss value is equal to 0. Correspondingly, when determining the second loss value based on the difference between a set second difference threshold and a second difference, the processor 40b includes: if the difference between the set second difference threshold and the second difference is greater than 0, then the second loss value is equal to the difference between the set second difference threshold and the second difference; if the difference between the set second difference threshold and the second difference is less than or equal to 0, then the second loss value is equal to 0.

[0127] In some embodiments, when acquiring training sample pairs, the processor 40b is specifically configured to: acquire a first text and the category name of the target level to which the first text belongs; the category name of the target level to which the first text belongs is pre-labeled; from a pre-established category text library, acquire the first category name of other levels belonging to the same first-level category as the category name of the target level, the second category name of the same level belonging to the same first-level category as the category name of the target level, and the third category name belonging to a different first-level category than the category name of the target level; and, using the category level name of the first text as a positive sample of the first text, and using the first category name, the second category name, and the third category name as negative samples of the first text, construct sample pairs between the first text and positive samples and sample pairs between the first text and each negative sample to obtain training sample pairs.

[0128] Accordingly, when processor 40b obtains the correlation comparison relationship of training sample pairs, it specifically performs the following: based on the category name of the target level to which the first text belongs and the pre-established hierarchical relationship between category names, it obtains from the negative samples the parent category name of the target level under the same first-level category, the second category name belonging to the same first-level category and at the same level as the target level, and the third category name belonging to a different first-level category from the target level; determines that the correlation between the category name of the target level and the first text is greater than the correlation between the parent category name of the target level and the first text; determines that the correlation between the category name of the target level and the first text is greater than the correlation between the second category name and the first text; determines that the correlation between the parent category name of the target level and the first text and the first text is greater than the correlation between the third category name and the first text; determines that the correlation between the second category name and the first text is greater than the correlation between the third category name and the first text; and determines that the correlation between the category name of the target level and the first text is greater than the correlation between the third category name and the first text.

[0129] In other embodiments, the processor 40b is further configured to: acquire the text to be matched; input the text to be matched and each text in the text library into a target text matching model to obtain the correlation between the text to be matched and each text in the text library; and determine the target text corresponding to the text to be matched from the text library based on the correlation between the text to be matched and each text in the text library.

[0130] Optionally, the text to be matched is the event description text; each text in the text library is an event category; and the target text is the event category to which the event description text belongs.

[0131] In some alternative implementations, such as Figure 4 As shown, the computing device may further include components such as a communication component 40c and a power supply component 40d. In some embodiments, the computing device may be implemented as a terminal device such as a computer or workstation. Accordingly, the computing device may further include components such as a display component 40e and an audio component 40f. Figure 4 The diagram only shows some components and does not mean that the computing device must contain them. Figure 4 The inclusion of all components does not imply that a computing device can only include... Figure 4 The components shown.

[0132] The computing device provided in this embodiment, by using the predicted relevance score between the first text and the positive sample in the training sample pair, and the pairing loss of the maximum predicted score of each sample in the sampled sample set, is equivalent to all samples in the sampled sample set being paired with positive samples. Therefore, it can improve the accuracy of the trained text matching model in distinguishing between the top 1 and negative samples that are close to the top 1, thereby helping to improve the accuracy of the top 1 in text matching. On the other hand, during model training, a pairing loss for comparing the relevance of training sample pairs is also introduced. This inherits the ranking relationship of the pairing method. Compared with the single-point method, for candidate texts that contain partially overlapping classifications or have hierarchical relationships, it can improve the accuracy of ranking candidate texts that contain partially overlapping classifications or have hierarchical relationships, thereby improving the accuracy of the top k in text matching.

[0133] In this embodiment, the memory is used to store computer programs and can be configured to store various other data to support operation on its host device. The processor can execute the computer programs stored in the memory to implement corresponding control logic. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Electrically Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0134] In the embodiments of this application, the processor can be any hardware processing device capable of executing the above-described method logic. Optionally, the processor can be a central processing unit (CPU), graphics processing unit (GPU), or microcontroller unit (MCU); it can also be a programmable device such as a field-programmable gate array (FPGA), programmable array logic (PAL), general array logic (GAL), or complex programmable logic device (CPLD); or an application-specific integrated circuit (ASIC) chip; or an advanced RISC machine (ARM) or system on chip (SoC), etc., but is not limited thereto.

[0135] In this embodiment, the communication component is configured to facilitate wired or wireless communication between its host device and other devices. The device housing the communication component can access wireless networks based on communication standards, such as Wireless Fidelity (WiFi), 2G or 3G, 4G, 5G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In another exemplary embodiment, the communication component may also be implemented based on Near Field Communication (NFC), Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wide Band (UWB), Bluetooth (BT), or other technologies.

[0136] In embodiments of this application, the display component may include a liquid crystal display (LCD) and a touch panel (TP). If the display component includes a touch panel, the display component can be implemented as a touchscreen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation.

[0137] In this embodiment, a power supply component is configured to provide power to various components of the device in which it resides. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply component resides.

[0138] In embodiments of this application, the audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC), which is configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals can be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals. For example, in devices with voice interaction capabilities, voice interaction with the user can be achieved through the audio component.

[0139] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0140] It should also be noted that the terms "first" and "second" in this article are used to distinguish different messages, devices, modules, etc., and do not represent a chronological order, nor do they limit "first" and "second" to different types.

[0141] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, compact disc read-only memory (CD-ROM), optical storage, etc.) containing computer-usable program code.

[0142] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (or systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0143] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0144] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0145] In a typical configuration, a computing device includes one or more processors (such as a CPU), input / output interfaces, network interfaces, and memory.

[0146] Memory may include non-persistent storage in computer-readable media, such as random-access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0147] Computer storage media are readable storage media, also known as removable media. Removable and non-removable media can be used to store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, Digital Video Disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device. As defined in this article, computer-readable media do not include transient computer-readable media, such as modulated data signals and carrier waves.

[0148] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the aforementioned element.

[0149] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A model training method, characterized in that, include: Obtain multiple training sample pairs; The plurality of training sample pairs include: a plurality of first texts and a sampling sample set corresponding to each of the plurality of first texts; the sampling sample set includes: positive samples and negative samples of the first texts; Obtain the comparison relationship between the correlations of the plurality of training sample pairs; the correlation of any training sample pair in the plurality of training sample pairs refers to the correlation between the first text contained in any training sample pair and the positive or negative samples of the first text contained in any training sample pair. Using the text matching model to be trained, calculate a first prediction score of the correlation between the first text and the positive sample, and a second prediction score of the correlation between the first text and the negative sample; Using the text matching model to be trained, the third prediction score of the relevance of the first training sample pair and the fourth prediction score of the relevance of the second training sample pair are calculated respectively. The first training sample pair and the second training sample pair are two training sample pairs associated with the same comparison relationship. Based on the first difference between the first predicted score and the maximum predicted score corresponding to the sampled sample set, and the second difference between the third and fourth predicted scores corresponding to the same comparison relationship, the model parameters of the text matching model to be trained are adjusted to obtain the target text matching model; wherein, the maximum predicted score corresponding to the sampled sample set is the maximum value between the first predicted score and the second predicted score.

2. The method according to claim 1, characterized in that, The negative samples in the sampling sample set include: a first negative sample belonging to the same category and level as the positive sample, a second negative sample belonging to the same category but a different level as the positive sample, and a third negative sample belonging to a different category as the positive sample; The correlation comparison of the training sample pairs includes: the correlation comparison of training sample pairs at the same level and the correlation comparison of training sample pairs across levels. The comparison of the correlation between training sample pairs at the same level includes: the correlation between the positive sample and the first text, and the correlation between the first negative sample and the first text; and the comparison of the correlation between each negative sample at the same level in the second negative sample and the third negative sample and the first text. The comparison of the correlation between the training sample pairs across the levels includes: the correlation between the positive sample and the first text, and the correlation between the second negative sample and the first text.

3. The method according to claim 1 or 2, characterized in that, Based on the first difference between the first predicted score and the maximum predicted score corresponding to the sampled sample set, and the second difference between the third and fourth predicted scores corresponding to the same comparison relationship, the model parameters of the text matching model to be trained are adjusted to obtain the target text matching model, including: A first loss value is determined based on the difference between a set first difference threshold and the first difference; the first loss value is greater than or equal to 0. The second loss value is determined based on the difference between the set second difference threshold and the second difference; the second loss value is greater than or equal to 0. With the goal of minimizing the sum of the first loss value and the second loss value, the model parameters of the text matching model to be trained are adjusted to obtain the target text matching model.

4. The method according to claim 3, characterized in that, The step of determining the first loss value based on the difference between a set first difference threshold and the first difference includes: If the difference between the set first difference threshold and the first difference is greater than 0, then the first loss value is equal to the difference between the set first difference threshold and the first difference. If the difference between the set first difference threshold and the first difference is less than or equal to 0, then the first loss value is equal to 0; The step of determining the second loss value based on the difference between the set second difference threshold and the second difference includes: If the difference between the set second difference threshold and the second difference is greater than 0, then the second loss value is equal to the difference between the set second difference threshold and the second difference. If the difference between the set second difference threshold and the second difference is less than or equal to 0, then the second loss value is equal to 0.

5. The method according to claim 2, characterized in that, The acquisition of training sample pairs includes: Obtain the first text and the category name of the target level to which the first text belongs; the category name of the target level to which the first text belongs is pre-labeled; From the pre-established category text library, obtain the first category name of other levels that belongs to the same first-level category as the category name of the target level, the second category name of the same level that belongs to the same first-level category as the category name of the target level, and the third category name that belongs to a different first-level category than the category name of the target level. Using the category level name of the first text as the positive sample of the first text, and using the first category name, the second category name, and the third category name as the negative samples of the first text, sample pairs between the first text and the positive samples and sample pairs between the first text and each negative sample are constructed to obtain the training sample pairs.

6. The method according to claim 5, characterized in that, The step of obtaining the correlation comparison relationship of the training sample pairs includes: Based on the category name of the target level to which the first text belongs and the hierarchical relationship between the pre-established category names, the parent category name of the target level under the same first-level category, the second category name belonging to the same first-level category and at the same level as the target level, and the third category name belonging to a different first-level category from the negative sample are obtained. The relevance between the category name of the target level and the first text is determined to be greater than the relevance between the parent category name of the target level and the first text. The relevance between the category name of the target level and the first text is determined to be greater than the relevance between the second category name and the first text; The correlation between the parent category name of the target level category name and the first text is determined to be greater than the correlation between the third category name and the first text. The relevance between the second category name and the first text is determined to be greater than the relevance between the third category name and the first text; as well as, The relevance between the category name of the target level and the first text is determined to be greater than the relevance between the third category name and the first text.

7. The method according to claim 2, characterized in that, The first text is the event description text; the positive and negative samples in the sample set are the event categories.

8. The method according to any one of claims 1, 5-7, characterized in that, include: Get the text to be matched; The text to be matched and each text in the text library are input into the target text matching model to obtain the correlation between the text to be matched and each text in the text library; Based on the correlation between the text to be matched and the texts in the text library, the target text corresponding to the text to be matched is determined from the text library.

9. The method according to claim 8, characterized in that, The text to be matched is the event description text; each text in the text library is an event category; the target text is the event category to which the event description text belongs.

10. A computing device, characterized in that, include: A memory and a processor; wherein the memory is used to store computer programs; The processor is coupled to the memory for executing the computer program to perform the steps of the method according to any one of claims 1-9.

11. A computer-readable storage medium storing computer instructions, characterized in that, When the computer instructions are executed by one or more processors, the one or more processors cause the processors to perform the steps of the method according to any one of claims 1-9.

Images