Text recognition model training method, text recognition method and equipment

Abstract

The invention provides a text recognition model training method, a text recognition method and equipment, and the text recognition model training method comprises the steps: carrying out the recognition of a first sample text and a second sample text in a sample text pair based on a text recognition model, and obtaining a first text vector and a second text vector; according to a first label and a second label, a loss function of the sample text pair is determined, the first label is used for representing whether the first sample text and the second sample text are similar in terms of semantics, and the second label is used for representing whether the first sample text and the second sample text are similar in terms of text; and determining a loss value according to the first text vector, the second text vector and the loss function, and training a text recognition model according to the loss value. According to the embodiment of the invention, the accuracy of similarity calculation between texts can be improved.

Description

Technical Field

[0001] The present application relates to the field of text processing technology, and in particular to a text recognition model training method, a text recognition method and a device. Background Art

[0002] Text similarity calculation is an important issue in natural language processing, which aims to quantify the similarity between texts. Text similarity calculation can be used in multiple tasks such as text classification, information retrieval, cluster analysis, etc.

[0003] In related technologies, deep learning models are generally used to calculate the similarity between texts. The specific processing process is as follows: through the deep learning model, the text is first vectorized to obtain the sentence vector representation of the text, and then the Euclidean distance, cosine similarity and other methods are used to calculate the similarity between the texts. Summary of the invention

[0004] The present application provides a text recognition model training method, a text recognition method and a device, which can improve the accuracy of similarity calculation between texts.

[0005] In a first aspect, the present application provides a text recognition model training method, comprising:

[0006] Based on the text recognition model, recognize the first sample text and the second sample text in the sample text pair to obtain a first text vector and a second text vector;

[0007] Determining a loss function of the sample text pair according to a first label and a second label, wherein the first label is used to indicate whether the first sample text and the second sample text are semantically similar, and the second label is used to indicate whether the first sample text and the second sample text are textually similar;

[0008] A loss value is determined according to the first text vector, the second text vector and the loss function, and the text recognition model is trained according to the loss value.

[0009] In a second aspect, the present application provides a text recognition method, comprising:

[0010] Inputting a to-be-processed text pair into a pre-trained text recognition model to obtain a first to-be-processed text vector and a second to-be-processed text vector, wherein the to-be-processed text pair includes a first to-be-processed text and a second to-be-processed text, the first to-be-processed text vector is obtained based on the first to-be-processed text, and the second to-be-processed text vector is obtained based on the second to-be-processed text;

[0011] Determine a second similarity between the first to-be-processed text vector and the second to-be-processed text vector, and obtain text similarity between the first to-be-processed text and the second to-be-processed text;

[0012] Wherein, the text recognition model is trained using the method described in the first aspect.

[0013] In a third aspect, the present application provides a text recognition model training device, comprising:

[0014] A recognition module, based on a text recognition model, recognizes a first sample text and a second sample text in a sample text pair to obtain a first text vector and a second text vector;

[0015] A first processing module, configured to determine a loss function of the sample text pair according to a first label and a second label, wherein the first label is used to indicate whether the first sample text and the second sample text are semantically similar, and the second label is used to indicate whether the first sample text and the second sample text are textually similar;

[0016] The second processing module is used to determine a loss value according to the first text vector, the second text vector and the loss function, and train the text recognition model according to the loss value.

[0017] In a fourth aspect, the present application provides a text recognition device, comprising:

[0018] A third processing module is used to input the to-be-processed text pair into a pre-trained text recognition model to obtain a first to-be-processed text vector and a second to-be-processed text vector, wherein the to-be-processed text pair includes a first to-be-processed text and a second to-be-processed text, the first to-be-processed text vector is obtained based on the first to-be-processed text, and the second to-be-processed text vector is obtained based on the second to-be-processed text;

[0019] A fourth processing module, used for determining a second similarity between the first to-be-processed text vector and the second to-be-processed text vector, and obtaining a text similarity between the first to-be-processed text and the second to-be-processed text;

[0020] Wherein, the text recognition model is trained using the method described in the first aspect.

[0021] In a fifth aspect, the present application provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, and one or more of the computer programs are executed by the at least one processor so that the at least one processor can execute the above-mentioned text recognition model training method, or execute the above-mentioned text recognition method.

[0022] In a sixth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-mentioned text recognition model training method, or the above-mentioned text recognition method.

[0023] In the seventh aspect, the present application provides a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying a computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above-mentioned text recognition model training method, or executes the above-mentioned text recognition method.

[0024] In the embodiment provided by the present application, the first label is used to indicate whether the first sample text and the second sample text are similar in semantics, and the second label is used to indicate whether the first sample text and the second sample text are similar in text. The present application considers whether the first sample text and the second sample text in the sample text pair are similar in semantics and whether the first sample text and the second sample text are similar in text. By integrating whether the text level is similar, the training data is refined and the appearance of difficult samples is reduced. Moreover, in the present application, based on the text recognition model, the first sample text and the second sample text in the sample text pair are recognized to obtain the first text vector and the second text vector. According to the first label and the second label, the loss function of the sample text pair is determined, and different loss functions are designed for different types of sample text pairs. Then, according to the first text vector, the second text vector and the loss function, the loss value is determined, and the text recognition model is trained according to the loss value. Different types of sample text pairs calculate the loss value through different loss functions and train the text recognition model. The model is easier to learn for various types of sample text pairs, which reduces the training difficulty of the text recognition model, improves the performance of the text recognition model, and improves the accuracy of the similarity between texts calculated by the text recognition model.

[0025] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings are used to provide a further understanding of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. The above and other features and advantages will become more apparent to those skilled in the art by describing detailed example embodiments with reference to the accompanying drawings. In the accompanying drawings:

[0027] Figure 1 A flowchart of a text recognition model training method provided in an embodiment of the present application;

[0028] Figure 2 A flow chart of a method for determining a second preset value and a third preset value provided in an embodiment of the present application;

[0029] Figure 3 A schematic diagram of text recognition model training provided in an embodiment of the present application;

[0030] Figure 4 A flowchart of a text recognition method provided in an embodiment of the present application;

[0031] Figure 5 A schematic diagram of text recognition provided in an embodiment of the present application;

[0032] Figure 6 A block diagram of a text recognition model training device provided in an embodiment of the present application;

[0033] Figure 7 A block diagram of a text recognition device provided in an embodiment of the present application;

[0034] Figure 8 A block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to enable those skilled in the art to better understand the technical solution of the present application, the exemplary embodiments of the present application are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0036] In the absence of conflict, the embodiments of the present application and the features therein may be combined with each other.

[0037] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0038] The terms used herein are only used to describe specific embodiments and are not intended to limit the present application. As used herein, the singular forms "a" and "the" are also intended to include plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "including" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded. "Connected" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

[0039] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this application, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined herein.

[0040] In the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals. The use of user data in this technical solution complies with relevant national laws and regulations (for example, the "Information Security Technology Personal Information Security Specification", etc.). For example: corresponding prescribed measures are taken for personal information access control; prescribed restrictions are imposed on the display of personal information; the purpose of using personal information does not exceed the scope of direct or reasonable association; when using personal information, clear identity reference is eliminated to avoid precise positioning of specific individuals.

[0041] The inventor analyzed the relevant technologies and found that:

[0042] Similarity calculation is an important issue in natural language processing. Similarity calculation can be used in multiple tasks such as text classification, information retrieval, cluster analysis, etc. Many applications in the industry have the need to measure the similarity between texts semantically. Generally, such requirements are collectively referred to as semantic matching.

[0043] In related technologies, semantic matching between texts is usually performed in the following two ways:

[0044] Method 1

[0045] The use of algorithms such as TF-IDF, Jaccard coefficient, and edit distance mainly solves the matching problem at the vocabulary level, or the semantic similarity problem at the vocabulary level. In fact, vocabulary-based matching algorithms have great limitations. For example, although "driver" and "driver" are not similar in literal sense, they actually express the same meaning.

[0046] Among them, TF-IDF (Term Frequency–Inverse Document Frequency) is a commonly used weighting technology used in information retrieval and data mining.

[0047] Jaccard coefficient, also known as Jaccard similarity coefficient, is an indicator to measure the similarity between data, such as set similarity, text similarity, etc. The Jaccard coefficient is defined as the ratio of the number of elements in the intersection of two sets A and B to the number of elements in the union of A and B.

[0048] Edit distance, also known as Levenshtein distance, refers to the minimum number of edit operations required to convert one string into another. The greater the distance between two strings, the more different the two strings are. Permitted edit operations include replacing one character with another, inserting a character, and deleting a character.

[0049] Method 2

[0050] When using deep learning methods, the text is generally vectorized first. After obtaining the sentence vector representation of the text, the Euclidean distance, cosine similarity and other methods are used to calculate the similarity between the two texts.

[0051] Specifically, for example, BERT (Bidirectional Encoder Representation from Transformers) is used to vectorize text. BERT is a pre-trained language model that uses the bidirectional encoder representation of Transformer. The pre-trained BERT representation can be fine-tuned through an additional output layer and is suitable for building state-of-the-art models for a wide range of tasks. BERT is one of the most advanced natural language processing technologies. It learns the contextual information of sentences from a large-scale corpus through pre-training and represents the learning results as vectors. These vectors can be used to calculate the similarity between two sentences with high accuracy. Another example is the GPT-3 model (artificial intelligence language model). GPT-3 is a language model based on deep learning with extremely strong generation capabilities. The GPT-3 model already has the ability to calculate sentence similarity and can be pre-trained on large-scale data sets to generate high-quality sentence vectors. For another example, text can be vectorized through text sentence vector representation methods such as word2vec (a related model used to generate word vectors) and glove (a word representation tool based on global word frequency statistics).

[0052] However, when training the semantic matching model, semantically similar text pairs are labeled as 1, and semantically dissimilar text pairs are labeled as 0, and then the model is trained. However, there are two situations for semantically similar text pairs: semantically similar and texts are similar; semantically similar and texts are dissimilar. Similarly, there are two situations for semantically dissimilar text pairs: semantically dissimilar and texts are similar; semantically dissimilar and texts are dissimilar. Therefore, in the data labeled 1, the text pairs with "semantically similar but dissimilar texts" are difficult samples for the model and are not easy to learn, resulting in poor model performance; similarly, in the data labeled 0, the text pairs with "semantically dissimilar but similar texts" are difficult samples for the model and are not easy to learn, resulting in poor model performance. The division of the data used for model training is too simple and direct, making it difficult to optimize the model to the best performance. Moreover, during model training, the loss function is difficult to learn the input data, resulting in poor model performance.

[0053] Therefore, the inventors have discovered that the accuracy of similarity calculation between texts in the related art is poor. To solve the above problem, the present application provides a text recognition model training method, a text recognition method and a device.

[0054] The text recognition model training method and text recognition method according to the embodiment of the present application can be performed by an electronic device such as a terminal device or a server. The terminal device can be a vehicle-mounted device, a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The server can be an independent physical server, a server cluster composed of multiple physical servers, or a cloud server capable of cloud computing. The method can be implemented by a processor calling a computer-readable program instruction stored in a memory. Alternatively, the method can be executed by a server.

[0055] Figure 1 A flowchart of a text recognition model training method provided in an embodiment of the present application. Figure 1 , the method comprising:

[0056] Step 101: Based on a text recognition model, recognize a first sample text and a second sample text in a sample text pair to obtain a first text vector and a second text vector.

[0057] In some embodiments, the text recognition model may be a BERT model or other models capable of generating text vectors, such as a RoBERTa model (A Robustly Optimized BERT Pretraining Approach), a LSTM model (Long Short-Term Memory), and the like.

[0058] In some embodiments, text vectorization is accomplished by representing the text as a vector in a high-dimensional space, where each dimension of the text vector may represent a word, a character, an n-gram, a semantic feature, or other text feature.

[0059] Step 102, determining a loss function of a sample text pair according to a first label and a second label, wherein the first label is used to indicate whether the first sample text and the second sample text are semantically similar, and the second label is used to indicate whether the first sample text and the second sample text are textually similar.

[0060] In some embodiments, whether the first sample text and the second sample text are semantically similar refers to whether the first sample text and the second sample text are similar at a deep semantic level, rather than whether they are similar on the surface of the text; whether the first sample text and the second sample text are textually similar refers to whether the first sample text and the second sample text are similar on the surface of the text. For example, whether they are similar on the surface of the text may include whether the vocabulary is repeated or similar, and may also include whether the sentence structure is similar.

[0061] In some embodiments, semantic similarity refers to the degree of closeness between two texts in meaning or implications. Semantic similarity focuses on whether the information or concepts conveyed by the text are similar or identical, rather than just the similarity of vocabulary or sentence structure. For example, "dog" and "canine" are semantically similar because they refer to the same animal, although their lexical forms are different. Semantic similarity is often used to assess the closeness of texts at the semantic level. Text similarity focuses more on the similarity of texts in surface features such as vocabulary, grammar, or sentence structure. Text similarity is often assessed based on factors such as the degree of vocabulary matching in the text, the length of the sentence, and the consistency of the sentence structure. For example, "I like dogs" and "I like dogs" may have high text similarity because their vocabulary and structure are very similar, except that a synonym is replaced.

[0062] In some embodiments, the first label and the second label are obtained by manual annotation.

[0063] In some embodiments, the value of the first label includes a first label value and a second label value, the first label value indicates that the first sample text and the second sample text are semantically similar, and the second label value indicates that the first sample text and the second sample text are semantically dissimilar; the value of the second label includes a third label value and a fourth label value, the third label value indicates that the first sample text and the second sample text are textually similar, and the fourth label value indicates that the first sample text and the second sample text are textually dissimilar.

[0064] According to the first label and the second label of the sample text pairs, the sample text pairs are divided into four types: semantically similar and textually similar; semantically similar and textually dissimilar; semantically dissimilar and textually similar; semantically dissimilar and textually dissimilar.

[0065] For example, the first label is represented by L1, the first label value can be set to 1, the second label value can be set to 0, L1=1 means that the first sample text and the second sample text are semantically similar, and L1=0 means that the first sample text and the second sample text are semantically dissimilar. The second label is represented by L2, the third label value can be set to a, and the fourth label value can be set to b. L2=a means that the first sample text and the second sample text are textually similar, and L2=b means that the first sample text and the second sample text are textually dissimilar. As shown in Table 1, the values ​​of the first label and the second label of the semantically similar text pair are shown.

[0066] Table 1 Values ​​of the first label and the second label of semantically similar text pairs

[0067] Semantically similar text pairs <![CDATA[First label L1]]> <![CDATA[Second label L2]]> <![CDATA[t1]]> 1 a <![CDATA[t2]]> 1 b <![CDATA[t3]]> 1 b … … … <![CDATA[t n ]]> 1 a

[0068] In Table 1, the number of semantically similar text pairs in the training set is n, that is, the number of sample text pairs with the first label L1=1 in the training set is n, and the sample text pairs with the first label L1=1 in the training set are represented by t1, t2, t3, ..., t n It means that the first label L1 of the semantically similar text pairs in the training set is 1, and the second label L2 is either equal to a or b.

[0069] As shown in Table 2, these are the values ​​of the first label and the second label for semantically dissimilar text pairs.

[0070] Table 2 Values ​​of the first label and the second label of semantically dissimilar text pairs

[0071] Semantically dissimilar text pairs <![CDATA[First label L1]]> <![CDATA[Second label L2]]> <![CDATA[k1]]> 0 b <![CDATA[k2]]> 0 a <![CDATA[k3]]> 0 b … … … <![CDATA[k m ]]> 0 a

[0072] In Table 2, the number of semantically dissimilar text pairs in the training set is m, that is, the number of sample text pairs with the first label L1=0 in the training set is m, and the sample text pairs with the first label L1=0 in the training set are represented by k1, k2, k3, ..., k m It means that the first label L1 of the semantically similar text pairs in the training set is 0, and the second label L2 is either equal to a or equal to b.

[0073] In some embodiments, step 102 includes:

[0074] When the value of the first label is the first label value and the value of the second label is the third label value, determining the loss function of the sample text pair to be the first loss function;

[0075] When the value of the first label is the first label value and the value of the second label is the fourth label value, determining the loss function of the sample text pair to be the second loss function;

[0076] When the value of the first label is the value of the second label and the value of the second label is the value of the third label, determining the loss function of the sample text pair to be the third loss function;

[0077] When the value of the first label is the second label value and the value of the second label is the fourth label value, the loss function of the sample text pair is determined to be the fourth loss function.

[0078] In an exemplary embodiment, the first loss function, the second loss function, the third loss function and the fourth loss function are different.

[0079] The sample text pairs are divided into four types by the first label and the second label of the sample text pairs: semantically similar and textually similar; semantically similar and textually dissimilar; semantically dissimilar and textually similar; semantically dissimilar and textually dissimilar. The loss function of the sample text pairs "semantically similar and textually similar" is set as the first loss function, the loss function of the sample text pairs "semantically similar and textually dissimilar" is set as the second loss function, the loss function of the sample text pairs "semantically dissimilar and textually similar" is set as the third loss function, and the loss function of the sample text pairs "semantically dissimilar and textually similar" is set as the fourth loss function. Different loss functions are set for different types of sample text pairs. Different types of sample text pairs use different loss functions to calculate loss values ​​and train text recognition models. The model is easier to learn for various types of sample text pairs, which reduces the difficulty of training the text recognition model, improves the performance of the text recognition model, and improves the accuracy of the similarity between texts calculated by the text recognition model.

[0080] In some embodiments, the loss function is a function used to measure the difference between the model prediction result and the actual result, and the loss value is the specific calculation result of the loss function on a specific data set. The loss value is a dynamic indicator that changes continuously as the model training progresses; while the loss function is a fixed function form used to calculate the loss value during the entire training process. The first loss function is used to calculate the loss value of sample text pairs that are "semantically similar and textually similar". The second loss function is used to calculate the loss value of sample text pairs that are "semantically similar but textually dissimilar". The third loss function is used to calculate the loss value of sample text pairs that are "semantically dissimilar and textually similar". The fourth loss function is used to calculate the loss value of sample text pairs that are "semantically dissimilar and textually dissimilar".

[0081] Step 103, determining a loss value according to the first text vector, the second text vector and the loss function, and training a text recognition model according to the loss value.

[0082] In some embodiments, the loss value of the first loss function includes the difference between the first preset value and the first similarity, the loss value of the second loss function includes the difference between the first similarity and the second preset value, the loss value of the third loss function includes the difference between the third preset value and the first similarity, and the loss value of the fourth loss function includes the difference between the first similarity and the fourth preset value;

[0083] Train the text recognition model based on the loss value, including:

[0084] Adjust the parameters of the text recognition model, and when the loss value is less than the preset threshold, obtain the trained text recognition model;

[0085] The first similarity refers to the similarity between the first text vector and the second text vector; the first preset value is greater than the second preset value, the second preset value is greater than the third preset value, and the third preset value is greater than the fourth preset value.

[0086] The loss value of the first loss function includes the difference between the first preset value and the first similarity. The parameters of the text recognition model are adjusted. When the loss value is less than the preset threshold, a trained text recognition model is obtained. During the training process of the text recognition model, the first similarity of the sample text pairs that are "semantically similar and textually similar" can be made close to the first preset value.

[0087] The loss value of the second loss function includes the difference between the first similarity and the second preset value. The parameters of the text recognition model are adjusted. When the loss value is less than the preset threshold, a trained text recognition model is obtained. During the training process of the text recognition model, the first similarity of the sample text pairs that are "semantically similar but textually dissimilar" can be made close to the second preset value.

[0088] The loss value of the third loss function includes the difference between the third preset value and the first similarity. The parameters of the text recognition model are adjusted. When the loss value is less than the preset threshold, a trained text recognition model is obtained. During the training process of the text recognition model, the first similarity of the sample text pairs that are "semantically dissimilar but textually similar" can be made close to the third preset value.

[0089] The loss value of the fourth loss function includes the difference between the first similarity and the fourth preset value. The parameters of the text recognition model are adjusted. When the loss value is less than the preset threshold, a trained text recognition model is obtained. During the training process of the text recognition model, the first similarity of sample text pairs that are "semantically dissimilar and textually dissimilar" can be made close to the fourth preset value.

[0090] Since the sample text pairs that are "semantically similar and textually similar" and the sample text pairs that are "semantically similar but textually dissimilar" are both semantically similar, the calculated first similarities of the sample text pairs are both relatively large; the sample text pairs that are "semantically dissimilar and textually similar" and the sample text pairs that are "semantically dissimilar and textually dissimilar" are both semantically dissimilar, and the calculated first similarities of the sample text pairs are both relatively small. In order to make the first similarities of these four types of sample text pairs easily reach the training goal, reduce the training difficulty, and make the first similarities of these four types of sample text pairs have discrimination, in the training process, it is hoped that the first similarity of the sample text pairs that are "semantically similar and textually similar" is less than or equal to the first preset value, and close to the first preset value; the first similarity of the sample text pairs that are "semantically similar but textually dissimilar" is greater than or equal to the second preset value, less than the first preset value, and close to the second preset value; the first similarity of the sample text pairs that are "semantically dissimilar but textually similar" is less than or equal to the third preset value, greater than the fourth preset value, and close to the third preset value; the first similarity of the sample text pairs that are "semantically dissimilar and textually similar" is greater than or equal to the fourth preset value, and close to the fourth preset value. In this way, the first similarity of the sample text pairs that are "semantically similar and textually similar" and the first similarity of the sample text pairs that are "semantically similar but textually dissimilar" are both relatively large and have distinguishability, while the first similarity of the sample text pairs that are "semantically dissimilar and textually similar" and the first similarity of the sample text pairs that are "semantically dissimilar and textually similar" are both relatively small and have distinguishability.

[0091] The loss value of the first loss function includes the difference between the first preset value and the first similarity, the loss value of the second loss function includes the difference between the first similarity and the second preset value, the loss value of the third loss function includes the difference between the third preset value and the first similarity, and the loss value of the fourth loss function includes the difference between the first similarity and the fourth preset value. It can be achieved that during the training process, the first similarity of the sample text pairs that are "semantically similar and textually similar" is less than or equal to the first preset value, and is close to the first preset value; the first similarity of the sample text pairs that are "semantically similar but textually dissimilar" is greater than or equal to the second preset value, less than the first preset value, and close to the second preset value; the first similarity of the sample text pairs that are "semantically dissimilar and textually similar" is less than or equal to the third preset value, greater than the fourth preset value, and close to the third preset value; the first similarity of the sample text pairs that are "semantically dissimilar and textually similar" is greater than or equal to the fourth preset value, and is close to the fourth preset value.

[0092] Since the first preset value is greater than the second preset value, the second preset value is greater than the third preset value, and the third preset value is greater than the fourth preset value, the first similarity of the sample text pairs that are "semantically similar and textually similar" is close to the first preset value, the first similarity of the sample text pairs that are "semantically similar but textually dissimilar" is close to the second preset value, the first similarity of the sample text pairs that are "semantically dissimilar and textually similar" is close to the third preset value, and the first similarity of the sample text pairs that are "semantically dissimilar and textually similar" is close to the fourth preset value, which can be used in the training process of the text recognition model. In the process, the first similarity of the sample text pairs that are "semantically similar and textually similar" is made greater than the first similarity of the sample text pairs that are "semantically similar and textually dissimilar", the first similarity of the sample text pairs that are "semantically dissimilar and textually similar", and the first similarity of the sample text pairs that are "semantically dissimilar and textually similar" is made greater than the first similarity of the sample text pairs that are "semantically dissimilar and textually similar", so that the first similarities of different types of sample text pairs have discrimination. During training, the first similarities of different types of sample text pairs are made discriminative by training the model, and during recognition, the text similarity between the first to-be-processed text and the second to-be-processed text of different types of to-be-processed text pairs also has discrimination, which is very important in scenarios where multiple texts need to be sorted according to text similarity.

[0093] In some embodiments, the first loss function includes the difference between the first preset value and the first similarity of the sample text pair; the second loss function includes the maximum value of the second preset value and the first similarity of the sample text pair; the third loss function includes the minimum value of the third preset value and the first similarity of the sample text pair; the fourth loss function includes the difference between the first similarity of the sample text pair and the fourth preset value.

[0094] The first loss function includes the difference between the first preset value and the first similarity of the sample text pair, which can achieve that during the training process, the first similarity of the sample text pair that is "semantically similar and textually similar" is less than or equal to the first preset value and close to the first preset value.

[0095] The second loss function includes the maximum value of the second preset value and the first similarity of the sample text pair. When the second preset value is greater than or equal to the first similarity, the second loss function includes the second preset value, which is a fixed value, and the model parameters are not updated, and the sample text pairs whose first similarity is less than or equal to the second preset value are discarded. When the second preset value is less than the first similarity, the second loss function includes the first similarity, and optimizes the first similarity toward the second preset value. It can be achieved that during the training process, the first similarity of the sample text pairs that are "semantically similar but textually dissimilar" is greater than or equal to the second preset value, less than the first preset value, and close to the second preset value.

[0096] The third loss function includes the minimum value of the third preset value and the first similarity of the sample text pair. When the first similarity is greater than or equal to the third preset value, the third loss function includes the third preset value, which is a fixed value, and the model parameters are not updated, and such sample text pairs whose first similarity is greater than or equal to the third preset value are discarded. When the first similarity is less than the third preset value, the third loss function includes the first similarity, and optimizes the first similarity toward the third preset value. In the training process, it can be achieved that the first similarity of the sample text pairs that are "semantically dissimilar but textually similar" is less than or equal to the third preset value, greater than the fourth preset value, and close to the third preset value.

[0097] The fourth loss function includes the difference between the first similarity of the sample text pair and a fourth preset value, which can ensure that during the training process, the first similarity of the sample text pairs that are "semantically dissimilar and textually dissimilar" is greater than or equal to the fourth preset value and close to the fourth preset value.

[0098] In some embodiments, the fourth loss function includes the difference between the first similarity of the sample text pair and the fourth preset value, and the fourth loss function may include the square of the difference between the first similarity of the sample text pair and the fourth preset value.

[0099] In some embodiments, the loss function is calculated as follows:

[0100]

[0101] Among them, loss refers to the loss value, L1 refers to the first label, the first label value is 1, the second label value is 0, L2 refers to the second label, the third label value is a, the fourth label value is b, vec A Refers to the text vector of the first sample text A, vec B refers to the text vector of the second sample text B, cos(vec A ,vec B ) refers to the cosine similarity between the text vector of the first sample text A and the text vector of the second sample text B, that is, the first similarity, max represents the maximum value, min represents the minimum value, the first preset value is 1, the second preset value is γ2, the third preset value is γ1, the fourth preset value is 0, 1>γ2>γ1>0.

[0102] When L1=1 and L2=a, the sample text pair is a sample text pair that is "similar in both semantics and text", cos(vec A ,vec B ) is larger and closer to 1, the smaller the loss value is and the better the model performance is, which means that it is more in line with the principle of “semantically similar and textually similar”. Therefore, in the training process of the text recognition model, the first similarity of the sample text pairs that are “semantically similar and textually similar” is close to 1.

[0103] When L1=1 and L2=b, the sample text pair is a sample text pair that is "semantically similar but textually dissimilar", which is a difficult sample. Since the sample text pair is textually dissimilar, the first similarity of the sample text pair that is "semantically similar but textually dissimilar" is difficult to approach 1. In order to reduce the difficulty of model training, the second loss function includes the maximum value of γ2 and the first similarity of the sample text pair, which can make the first similarity of the sample text pair that is "semantically similar but textually dissimilar" close to the second preset value γ2 during the training process of the text recognition model. 1>γ2, so the difficulty of model training can be reduced, and 1>γ2, so the first similarity of the sample text pair that is "semantically similar and textually similar" and the first similarity of the sample text pair that is "semantically similar but textually dissimilar" have distinguishability.

[0104] When L1=0 and L2=a, the sample text pair is a sample text pair that is "semantically dissimilar but textually similar", which is a difficult sample. Since the sample text pairs are textually similar, the first similarity of the sample text pair that is "semantically dissimilar but textually similar" is difficult to approach 0. In order to reduce the difficulty of model training, the third loss function includes the minimum value of the third preset value γ1 and the first similarity of the sample text pair, which can make the first similarity of the sample text pair that is "semantically dissimilar but textually similar" close to the third preset value γ1 during the training process of the text recognition model. γ1>0, so the difficulty of model training can be reduced. Moreover, γ1>0, the first similarity of the sample text pair that is "semantically dissimilar but textually similar" and the first similarity of the sample text pair that is "semantically dissimilar but textually similar" have distinguishability. Moreover, γ2>γ1, the sample text pairs that are "semantically similar but textually dissimilar" are semantically similar, and the sample text pairs that are "semantically dissimilar but textually similar" are semantically dissimilar. During the training process of the text recognition model, the first similarity of the sample text pairs that are "semantically similar but textually dissimilar" is greater than the first similarity of the sample text pairs that are "semantically dissimilar but textually similar". The first similarity of the sample text pairs that are "semantically similar but textually dissimilar" and the first similarity of the sample text pairs that are "semantically dissimilar but textually similar" have distinguishability.

[0105] When L1 = 0 and L2 = b, the sample text pair is a sample text pair that is "semantically dissimilar and textually dissimilar", cos(vec A ,vec B ) is closer to 0, the closer the loss value is to 0, the better the performance of the model, and cos(vec A ,vec B ) is closer to 0, which means that it is more consistent with "semantically dissimilar and textually dissimilar". Therefore, in the first training process, the first similarity of the sample text pairs that are "semantically dissimilar and textually dissimilar" is close to 0.

[0106] In some embodiments, determining a loss value based on the first text vector, the second text vector and a loss function includes: determining a first similarity between the first text vector and the second text vector based on the first text vector and the second text vector; and determining the loss value based on the first similarity and the loss function.

[0107] According to the first text vector and the second text vector, the first similarity between the first text vector and the second text vector is determined, which is to calculate the value of the first similarity of the specific sample text pair, and the loss function includes the parameter of the first similarity. According to the first similarity and the loss function, the loss value is determined, which is to bring the calculated value of the first similarity of the specific sample text pair into the parameter of the first similarity in the loss function to obtain the loss value of the specific sample text pair. According to the loss value, the text recognition model can be trained, and the training of the text recognition model is realized.

[0108] In some embodiments, determining a first similarity between the first text vector and the second text vector based on the first text vector and the second text vector includes: calculating a cosine similarity between the first text vector and the second text vector; and using the cosine similarity as a first similarity of a sample text pair.

[0109] Cosine similarity is the most common similarity calculation method. The cosine similarity between the first text vector and the second text vector is calculated and used as the first similarity of the sample text pair. Moreover, during recognition, when determining the second similarity between the first to-be-processed text vector and the second to-be-processed text vector, the cosine similarity between the first to-be-processed text vector and the second to-be-processed text vector is also used as the second similarity. The similarity calculation method is kept consistent during training and recognition, and the recognition effect will be better.

[0110] In some embodiments, a first similarity between the first text vector and the second text vector is determined based on the first text vector and the second text vector, and a similarity calculation method such as Manhattan distance and Euclidean distance may also be used.

[0111] In some embodiments, determining the loss value based on the first similarity and the loss function refers to a process of substituting the numerical value of the first similarity into the loss function to obtain the loss value.

[0112] In some embodiments, Figure 2 A flow chart of a method for determining a second preset value and a third preset value provided in an embodiment of the present application. Figure 2 In the process of determining the second preset value and the third preset value, the process includes:

[0113] Step 201, determining the edit distance of each sample text pair in the training set.

[0114] In the exemplary embodiment, the edit distance, also known as the Levenshtein distance, refers to the minimum number of edit operations required to convert one string into another string between two strings. The greater the distance between the two strings, the more different the two strings are. The permitted edit operations include replacing one character with another, inserting a character, and deleting a character.

[0115] Step 202: Divide the edit distance by the total number of characters to obtain a quotient as the distance between each sample text pair on the text, wherein the total number of characters is the sum of the number of characters of the first sample text and the second sample text in the sample text pair.

[0116] In the exemplary embodiment, the distance between the first sample text A and the second sample text B in the sample text pair is represented by d. Where D(A→B) represents the edit distance between the first sample text A and the second sample text B, l A refers to the number of characters in the first sample text A, l B Refers to the number of characters in the second sample text B.

[0117] in, is a weight coefficient used to balance the deviation of D(A→B) caused by the difference in the number of characters between the first sample text A and the second sample text B.

[0118] The smaller the textual distance between the first sample text and the second sample text in the sample text pair, the greater the textual similarity between the first sample text and the second sample text; the larger the textual distance between the first sample text and the second sample text, the smaller the textual similarity between the first sample text and the second sample text.

[0119] Step 203: taking the average value of the distance between the first target text pair on the text as the second preset value, wherein the first target text pair refers to a sample text pair in the training set whose first label value is the second label value.

[0120] In an exemplary embodiment, the second preset value is γ2, d i It refers to the textual distance between the first sample text A and the second sample text B in the i-th first target text pair, L1 refers to the first label, the first label value is 1, and the second label value is 0, m is the number of sample text pairs (i.e., first target text pairs) with L1=0 in the training set, and γ2 refers to the average textual distance between the first sample text A and the second sample text B in the sample text pairs with L1=0.

[0121] Step 204: taking the average value of the distance between the second target text pairs on the text as the third preset value, wherein the second target text pairs refer to sample text pairs in the training set whose first label value is the first label value.

[0122] In an exemplary embodiment, the third preset value is γ1, d i It refers to the textual distance between the first sample text A and the second sample text B in the i-th second target text pair, L1 refers to the first label, the first label value is 1, and the second label value is 0, n is the number of sample text pairs (i.e., second target text pairs) with L1=1, and γ1 refers to the average textual distance between the first sample text A and the second sample text B in the sample text pairs with L1=1.

[0123] In some embodiments, the first preset value is 1, the second preset value is γ2, the third preset value is γ1, and the fourth preset value is 0. The second preset value and the third preset value need to be determined, and the first preset value and the fourth preset value can be directly set manually.

[0124] In some embodiments, the implementation method of obtaining the value of the second label of the sample text pair through the second target text pair includes but is not limited to the following methods:

[0125] Method 1

[0126] Determine a first subtext pair and a second subtext pair from a second target text pair, set the value of the second label of the first subtext pair to a third label value, and set the value of the second label of the second subtext pair to a fourth label value, wherein a distance of the first subtext pair on the text is less than a third preset value, and a distance of the second subtext pair on the text is greater than or equal to the third preset value.

[0127] In an exemplary embodiment, the text distance between the first sample text and the second sample text may be calculated by using the edit distance or other text distance calculation methods, such as the Jaccard coefficient.

[0128] In the exemplary embodiment, the third preset value is γ1, L1 refers to the first label, the first label value is 1, the second label value is 0, γ1 refers to the average distance between the first sample text A and the second sample text B in the sample text pair (i.e., the second target text pair) with L1=1, d i It refers to the distance between the first sample text A and the second sample text B in the i-th second target text pair. L2 refers to the second label. The third label value is a, and the fourth label value is b. i L2=a,d for the second target text pair (i.e., the first subtext pair) with <γ1 i≥γ1 for the second target text pair (i.e., the second subtext pair). The smaller the distance between the first sample text and the second sample text, the greater the textual similarity between the first sample text and the second sample text. The value of the second label of the sample text pair is determined to be more accurate by the average textual distance γ1 of the first sample text A and the second sample text B in the sample text pair with L1=1.

[0129] Method 2

[0130] Sort the distances of the second target text pairs on the text from small to large, set the value of the second label of the first half of the second target text pair to the third label value, and set the value of the second label of the second half of the second target text pair to the fourth label value.

[0131] In the exemplary embodiment, L1 refers to the first label, the first label value is 1, the second label value is 0, n is the number of sample text pairs (i.e., second target text pairs) with L1=1, and the sample text pairs t1, t2, t3, ..., t n In the example, the distance d between the first sample text and the second sample text is sorted from small to large, L2=a for the first n / 2 sample text pairs, and L2=b for the last n / 2 sample text pairs. The smaller the distance between the first sample text and the second sample text, the greater the similarity between the first sample text and the second sample text. Sort the distances between the second target text pairs from small to large, set the value of the second label of the first half of the second target text pairs to the third label value, and set the value of the second label of the second half of the second target text pairs to the fourth label value, and divide them according to the number of second target texts to determine whether the value of the second label of the sample text pairs is more accurate.

[0132] In some embodiments, the implementation method of obtaining the value of the second label of the sample text pair through the first target text pair includes but is not limited to the following methods:

[0133] Method 1

[0134] A third subtext pair and a fourth subtext pair are determined from the first target text pair, a value of a second label of the third subtext pair is set to a third label value, and a value of a second label of the fourth subtext pair is set to a fourth label value, wherein a distance of the third subtext pair on the text is less than a second preset value, and a distance of the fourth subtext pair on the text is greater than or equal to the second preset value.

[0135] In an exemplary embodiment, the text distance between the first sample text and the second sample text may be calculated by using the edit distance or other text distance calculation methods, such as the Jaccard coefficient.

[0136] In the exemplary embodiment, the second preset value is γ2, L1 refers to the first label, the first label value is 1, the second label value is 0, γ2 refers to the average distance between the first sample text A and the second sample text B in the sample text pair (i.e., the first target text pair) with L1=0, d i It refers to the distance between the first sample text A and the second sample text B in the i-th first target text pair. L2 refers to the second label. The third label value is a, and the fourth label value is b. i L2=a,d for the first target text pair (i.e., the third sub-text pair) with <γ2 i ≥γ2, the L2=b of the first target text pair (i.e., the fourth subtext pair). The smaller the distance between the first sample text and the second sample text, the greater the textual similarity between the first sample text and the second sample text. The value of the second label of the sample text pair is determined to be more accurate by the average textual distance γ2 of the first sample text A and the second sample text B in the sample text pair with L1=0.

[0137] Method 2

[0138] Sort the distances of the first target text pairs on the text from small to large, set the second label of the first target text pair in the first half to the third label value, and set the second label of the first target text pair in the second half to the fourth label value.

[0139] In the exemplary embodiment, L1 refers to the first label, the first label value is 1, the second label value is 0, m is the number of sample text pairs (i.e., first target text pairs) with L1=0 in the training set, and k1, k2, k3, ..., k m In the example, the distance d between the first sample text and the second sample text is sorted from small to large, L2=a for the first m / 2 sample text pairs, and L2=b for the last m / 2 sample text pairs. The smaller the distance between the first sample text and the second sample text, the greater the similarity between the first sample text and the second sample text. Sort the distance between the first target text pairs from small to large, set the second label of the first target text pairs in the first half to the third label value, and set the second label of the second target text pairs in the second half to the fourth label value, and divide them according to the number of first target text pairs to determine the value of the second label of the sample text pairs more accurately.

[0140] In some embodiments, Figure 3 A schematic diagram of text recognition model training provided in an embodiment of the present application. Figure 3In the method, the first sample text and the second sample text are input into the text recognition model to obtain the first text vector and the second text vector output by the text recognition model, the loss value is calculated according to the first text vector and the second text vector, and the text recognition model is trained according to the loss value. Specifically, the loss value is calculated according to the first text vector and the second text vector, including: determining the loss function of the sample text pair according to the first label and the second label of the sample text pair; determining the first similarity of the sample text pair according to the first text vector and the second text vector; determining the loss value according to the first similarity of the sample text pair and the loss function of the sample text pair. In some embodiments, the text recognition model can be a BERT model or other model that can generate text vectors.

[0141] In summary, in the embodiments provided by the present application, the first label is used to indicate whether the first sample text and the second sample text are semantically similar, and the second label is used to indicate whether the first sample text and the second sample text are textually similar. Compared with the related art that divides the training data into two categories, one is semantically similar text pairs, and the other is semantically dissimilar text pairs, in the semantically similar text pairs, the "semantically similar, textually dissimilar" text pairs are difficult samples for the model, and in the semantically dissimilar text pairs, the "semantically dissimilar, textually similar" text pairs are difficult samples for the model. Difficult samples are not easy to learn, and the model training is difficult, resulting in poor model performance. The present application considers both whether the first sample text and the second sample text in the sample text pair are semantically similar and whether the first sample text and the second sample text are textually similar. By integrating the similarity at the text level, the training data is refined and the occurrence of difficult samples is reduced. Moreover, in the present application, based on the text recognition model, the first sample text and the second sample text in the sample text pair are recognized to obtain the first text vector and the second text vector, and the loss function of the sample text pair is determined according to the first label and the second label, and different loss functions are designed for different types of sample text pairs. Then, according to the first text vector, the second text vector and the loss function, the loss value is determined, and the text recognition model is trained according to the loss value. Different types of sample text pairs calculate loss values ​​through different loss functions and train the text recognition model. The model is easier to learn for various types of sample text pairs, which reduces the training difficulty of the text recognition model, improves the performance of the text recognition model, and improves the accuracy of the similarity between texts calculated by the text recognition model.

[0142] Figure 4 A flowchart of a text recognition method provided in an embodiment of the present application. Figure 4 , the method comprising:

[0143] Step 401: input the text pair to be processed into a pre-trained text recognition model to obtain a first text vector to be processed and a second text vector to be processed, wherein the text pair to be processed includes a first text to be processed and a second text to be processed, the first text vector to be processed is obtained based on the first text to be processed, and the second text vector to be processed is obtained based on the second text to be processed.

[0144] Step 402: determine the second similarity between the first to-be-processed text vector and the second to-be-processed text vector, and obtain the text similarity between the first to-be-processed text and the second to-be-processed text.

[0145] Among them, the text recognition model is trained using the text recognition model training method provided in the embodiment of the present application.

[0146] In some embodiments, step 402 may be implemented inside the text recognition model or outside the text recognition model, and this application does not limit this.

[0147] Since the text recognition model training method provided in the embodiment of the present application reduces the training difficulty of the text recognition model, improves the performance of the text recognition model, and improves the accuracy of the similarity between texts calculated by the text recognition model, therefore, by using the text recognition model trained by the text recognition model training method provided in the embodiment of the present application, a first text vector to be processed and a first text vector to be processed are obtained, the second similarity between the first text vector to be processed and the second text vector to be processed is determined, and the text similarity between the first text to be processed and the second text to be processed is obtained, and the obtained text similarity can be calculated more accurately.

[0148] In some embodiments, Figure 5 A schematic diagram of text recognition provided in an embodiment of the present application. Figure 5 In the method, the first text to be processed and the second text to be processed are input into the text recognition model to obtain the first text vector to be processed and the second text vector to be processed, and the second similarity between the first text vector to be processed and the second text vector to be processed is calculated, and the second similarity is used as the text similarity between the first text to be processed and the second text to be processed.

[0149] In some embodiments, determining the second similarity between the first to-be-processed text vector and the second to-be-processed text vector may be calculating the cosine similarity between the first to-be-processed text vector and the second to-be-processed text vector. In some embodiments, the text recognition model may be a BERT model or other model capable of generating text vectors.

[0150] In the exemplary embodiment, after step 402, there are multiple application scenarios for the text similarity between the first to-be-processed text and the second to-be-processed text. For example, in a dialogue robot scenario, the most similar text is generally obtained and output; in a speech clustering scenario, it is generally necessary to sort according to the similarity value and then output a certain amount of the most similar text.

[0151] In the exemplary embodiment, the FAQ (frequently-asked questions) dialogue system is the most commonly used method in intelligent customer service. After the customer enters a query in the intelligent dialogue system, the background will recognize the semantic information of the text and return the answer. The algorithm logic behind it is usually to first collect business data and user log data, maintain a FAQ question-answer library in the form of "question-answer" question-answer pairs; then expand the standard questions in the question-answer pairs to expand multiple similar questions for semantic matching; when the query is entered, the system will first perform a rough recall, query the FAQ library and query the question list (similar question list) with a high degree of matching with the query text; then use a more accurate model to fine-sort the question list (here the text recognition method of the present invention can be used), and if it is greater than the set threshold, the relevant answer will be returned to the customer.

[0152] It can be understood that the above-mentioned various method embodiments mentioned in this application can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, this application will not go into details. Those skilled in the art can understand that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined by its function and possible internal logic.

[0153] In addition, the present application also provides a text recognition device, a text recognition model training device, an electronic device, a computer-readable storage medium, and a computer program product, all of which can be used to implement any text recognition method or text recognition model training method provided by the present application. The corresponding technical solutions and descriptions are referred to in the corresponding records of the method part and will not be repeated here.

[0154] Figure 6 A block diagram of a text recognition model training device provided in an embodiment of the present application.

[0155] Reference Figure 6 , an embodiment of the present application provides a text recognition model training device, the text recognition model training device comprising:

[0156] A recognition module 601 is used to recognize the first sample text and the second sample text in the sample text pair based on a text recognition model to obtain a first text vector and a second text vector;

[0157] A first processing module 602 is used to determine a loss function of the sample text pair according to a first label and a second label, wherein the first label is used to indicate whether the first sample text and the second sample text are semantically similar, and the second label is used to indicate whether the first sample text and the second sample text are textually similar;

[0158] The second processing module 603 is used to determine a loss value according to the first text vector, the second text vector and the loss function, and train the text recognition model according to the loss value.

[0159] Each module in the above-mentioned text recognition model training device can be implemented in whole or in part by software, hardware and their combination. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0160] Figure 7 A block diagram of a text recognition device provided in an embodiment of the present application.

[0161] Reference Figure 7 , an embodiment of the present application provides a text recognition device, the text recognition device comprising:

[0162] The third processing module 701 is used to input the to-be-processed text pair into a pre-trained text recognition model to obtain a first to-be-processed text vector and a second to-be-processed text vector, wherein the to-be-processed text pair includes a first to-be-processed text and a second to-be-processed text, the first to-be-processed text vector is obtained based on the first to-be-processed text, and the second to-be-processed text vector is obtained based on the second to-be-processed text;

[0163] The fourth processing module 702 is used to determine the second similarity between the first to-be-processed text vector and the second to-be-processed text vector, and obtain the text similarity between the first to-be-processed text and the second to-be-processed text;

[0164] Among them, the text recognition model is trained using the text recognition model training method provided in the embodiment of the present application.

[0165] Each module in the above-mentioned text recognition device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0166] Figure 8A block diagram of an electronic device provided in an embodiment of the present application.

[0167] Reference Figure 8 An embodiment of the present application provides an electronic device, which includes: at least one processor 801; at least one memory 802, and one or more I / O interfaces 803, connected between the processor 801 and the memory 802; wherein the memory 802 stores one or more computer programs that can be executed by at least one processor 801, and the one or more computer programs are executed by at least one processor 801 so that the at least one processor 801 can execute the above-mentioned text recognition model training method, or execute the above-mentioned text recognition method.

[0168] Each module in the above electronic device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0169] The embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-mentioned text recognition model training method, or the above-mentioned text recognition method. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0170] An embodiment of the present application also provides a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above-mentioned text recognition model training method, or the above-mentioned text recognition method.

[0171] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable storage medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium).

[0172] As is known to those of ordinary skill in the art, the term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable program instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technology, portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. In addition, it is known to those of ordinary skill in the art that communication media typically contain computer-readable program instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and may include any information delivery medium.

[0173] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0174] The computer program instructions for performing the operation of the present application can be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions can be executed completely on a user's computer, partially on a user's computer, executed as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet). In some embodiments, by using the state information of a computer-readable program instruction to personalize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA) or a programmable logic array (PLA), the electronic circuit can execute a computer-readable program instruction, thereby realizing various aspects of the present application.

[0175] The computer program product described herein may be implemented in hardware, software, or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK), etc.

[0176] Various aspects of the present application are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0177] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0178] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0179] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can also be executed in the opposite order sometimes, depending on the function involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a dedicated hardware-based system that performs the function or action of the specification, or can be realized by a combination of special-purpose hardware and computer instructions.

[0180] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for limiting purposes. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly noted, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, those skilled in the art will appreciate that various changes in form and detail may be made without departing from the scope of the present application as set forth in the appended claims.

Claims

1. A text recognition model training method, characterized in that: include: Based on the text recognition model, recognize the first sample text and the second sample text in the sample text pair to obtain a first text vector and a second text vector; Determining a loss function of the sample text pair according to a first label and a second label, wherein the first label is used to indicate whether the first sample text and the second sample text are semantically similar, and the second label is used to indicate whether the first sample text and the second sample text are textually similar; A loss value is determined according to the first text vector, the second text vector and the loss function, and the text recognition model is trained according to the loss value.

2. The text recognition model training method according to claim 1, characterized in that: The value of the first label includes a first label value and a second label value, the first label value indicates that the first sample text and the second sample text are semantically similar, and the second label value indicates that the first sample text and the second sample text are semantically dissimilar; the value of the second label includes a third label value and a fourth label value, the third label value indicates that the first sample text and the second sample text are textually similar, and the fourth label value indicates that the first sample text and the second sample text are textually dissimilar; The step of determining the loss function of the sample text pair according to the first label and the second label includes: When the value of the first label is the first label value and the value of the second label is the third label value, determining the loss function of the sample text pair to be the first loss function; When the value of the first label is the first label value and the value of the second label is the fourth label value, determining the loss function of the sample text pair to be the second loss function; When the value of the first label is the second label value and the value of the second label is the third label value, determining the loss function of the sample text pair to be a third loss function; When the value of the first label is the second label value and the value of the second label is the fourth label value, the loss function of the sample text pair is determined to be a fourth loss function.

3. The text recognition model training method according to claim 2, characterized in that: The loss value of the first loss function includes the difference between the first preset value and the first similarity, the loss value of the second loss function includes the difference between the first similarity and the second preset value, the loss value of the third loss function includes the difference between the third preset value and the first similarity, and the loss value of the fourth loss function includes the difference between the first similarity and the fourth preset value; The step of training the text recognition model according to the loss value comprises: Adjusting the parameters of the text recognition model, and obtaining a trained text recognition model when the loss value is less than a preset threshold; Among them, the first similarity refers to the similarity between the first text vector and the second text vector; the first preset value is greater than the second preset value, the second preset value is greater than the third preset value, and the third preset value is greater than the fourth preset value.

4. The text recognition model training method according to claim 3, characterized in that: The method further comprises: Determine the edit distance for each sample text pair in the training set; The quotient obtained by dividing the edit distance by the total number of characters is used as the distance of each sample text pair on the text, wherein the total number of characters is the sum of the number of characters of the first sample text and the second sample text in the sample text pair; Taking the average value of the distance between the first target text pair on the text as the second preset value, wherein the first target text pair refers to the sample text pair in the training set whose value of the first label is the value of the second label; The average value of the distance between the second target text pairs on the text is used as the third preset value, wherein the second target text pair refers to a sample text pair in the training set whose first label value is the first label value.

5. The text recognition model training method according to claim 1, characterized in that: The determining the loss value according to the first text vector, the second text vector and the loss function comprises: Determining a first similarity between the first text vector and the second text vector according to the first text vector and the second text vector; A loss value is determined according to the first similarity and the loss function.

6. A text recognition method, characterized in that: include: Inputting a to-be-processed text pair into a pre-trained text recognition model to obtain a first to-be-processed text vector and a second to-be-processed text vector, wherein the to-be-processed text pair includes a first to-be-processed text and a second to-be-processed text, the first to-be-processed text vector is obtained based on the first to-be-processed text, and the second to-be-processed text vector is obtained based on the second to-be-processed text; Determine a second similarity between the first to-be-processed text vector and the second to-be-processed text vector, and obtain text similarity between the first to-be-processed text and the second to-be-processed text; Wherein, the text recognition model is trained by using the method described in any one of claims 1 to 5.

7. A text recognition model training device, characterized in that: include: A recognition module, based on a text recognition model, recognizes a first sample text and a second sample text in a sample text pair to obtain a first text vector and a second text vector; A first processing module, configured to determine a loss function of the sample text pair according to a first label and a second label, wherein the first label is used to indicate whether the first sample text and the second sample text are semantically similar, and the second label is used to indicate whether the first sample text and the second sample text are textually similar; The second processing module is used to determine a loss value according to the first text vector, the second text vector and the loss function, and train the text recognition model according to the loss value.

8. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, and the one or more computer programs are executed by the at least one processor so that the at least one processor can execute the text recognition model training method described in any one of claims 1 to 5, or execute the text recognition method described in claim 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When executed by a processor, the computer program implements the text recognition model training method as described in any one of claims 1 to 5, or executes the text recognition method as described in claim 6.

10. A computer program product, characterized in that It includes a computer-readable code, or a non-volatile computer-readable storage medium carrying a computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the text recognition model training method as described in any one of claims 1 to 5, or executes the text recognition method as described in claim 6.

Images