Text processing model training method and device, computer device, and storage medium

Abstract

Embodiments of the present application provide a text processing model training method and device, computer equipment and a storage medium, which can obtain original text information of training text; perform feature extraction on the original text information to obtain original text features of the original text information; determine target similar text information corresponding to the original text information according to the original text features; and train a text processing model according to the original text information and the target similar text information to obtain a trained text processing model, which is used for recognizing text. Since the embodiments of the present application can determine the target similar text information corresponding to the original text information, training the text processing model based on the original text information and the target similar text information can improve the generalization performance and accuracy of the obtained trained text processing model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a text processing model training method and device, computer equipment and a storage medium. BACKGROUND

[0002] Currently, most text business scenarios need to use a text processing model to process text. Training samples are needed in the process of training the text processing model. If there is relevant text information in the training samples, it can be considered that there is a large amount of homogenized text information in the training data. Homogenization refers to the existence of the same or similar text information in the training samples.

[0003] For example, a training data set is divided into an A subset and a B subset, each training sample in the training data set is composed of four kinds of text information, and the four kinds of text information are text information 1, text information 2, text information 3 and text information 4. If each training sample in the A subset contains text information 1 and text information 2, text information 1 and text information 2 are called relevant text information, and it can be determined that there is homogenized text information in the A subset. If each training sample in the B subset contains text information 3 and text information 4, text information 3 and text information 4 are called relevant text information, and it can be determined that there is homogenized text information in the B subset.

[0004] In this case, using training samples with relevant text information to train the text processing model can cause the trained text processing model to have low generalization performance and low accuracy. SUMMARY

[0005] The embodiments of the present application provide a text processing model training method and device, computer equipment and a storage medium, which can improve the generalization performance and accuracy of the trained text processing model.

[0006] A text processing model training method comprises:

[0007] Obtaining original text information of training text;

[0008] Performing feature extraction on the original text information to obtain original text features of the original text information;

[0009] According to the original text features, determining target similar text information corresponding to the original text information;

[0010] According to the original text information and the target similar text information, training a text processing model to obtain a trained text processing model, wherein the trained text processing model is used for recognizing text.

[0011] Accordingly, an embodiment of the present application provides a text processing model training apparatus, comprising:

[0012] The acquisition unit can be configured to acquire original text information of a training text.

[0013] The extraction unit can be configured to perform feature extraction on the original text information to obtain original text features of the original text information.

[0014] The determination unit can be configured to determine target similar text information corresponding to the original text information according to the original text features.

[0015] The training unit can be configured to train the text processing model according to the original text information and the target similar text information to obtain a trained text processing model, and the trained text processing model is used for text recognition.

[0016] In some embodiments, the determination unit can be specifically configured to determine reference text features matching the dimension of the original text features, and determine the target similar text information corresponding to the original text information according to the original text features and the reference text features.

[0017] In some embodiments, the determination unit can be specifically configured to calculate difference information between the original text features and the reference text features according to the original text features and the reference text features, and determine the target similar text information corresponding to the original text information according to the difference information.

[0018] In some embodiments, the determination unit can be specifically configured to calculate target distance information between the original text features and the reference text features in a mapping space according to the original text features and the reference text features, and take the target distance information as the difference information.

[0019] In some embodiments, the determination unit can be specifically configured to acquire reference text information matching the dimension of the original text information, and perform feature extraction on the reference text information to obtain reference text features matching the dimension of the original text features.

[0020] In some embodiments, the training unit can be specifically configured to perform prediction on the original text information to obtain predicted original text information, perform prediction on the target similar text information to obtain predicted similar text information, and train the text processing model according to the original text information, the predicted original text information, the target similar text information, and the predicted similar text information to obtain the trained text processing model.

[0021] In some embodiments, the training unit can be specifically configured to calculate a first loss value between the original text information and the predicted original text information according to the original text information and the predicted original text information; calculate a second loss value between the target similar text information and the predicted similar text information according to the target similar text information and the predicted similar text information; and train the text processing model according to the first loss value and the second loss value to obtain the trained text processing model.

[0022] In some embodiments, the text processing model training apparatus can further include an encoding training unit, which can be specifically configured to obtain a label corresponding to the original text information of the training text; and perform encoding training on an initial encoder of an initial text processing model based on the original text information and the label to obtain the text processing model.

[0023] On this basis, the extraction unit can specifically perform feature extraction on the original text information by using the text processing model to obtain original text features of the original text information.

[0024] In addition, an embodiment of the present application further provides a computer device, which includes a memory and a processor; the memory stores a computer program, and the processor is configured to run the computer program in the memory to perform the operations in any of the text processing model training methods provided by the embodiments of the present application.

[0025] In addition, an embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is adapted to be loaded by a processor to execute any of the text processing model training methods provided by the embodiments of the present application.

[0026] The embodiments of the present application can obtain original text information of a training text; perform feature extraction on the original text information to obtain original text features of the original text information; determine target similar text information corresponding to the original text information according to the original text features; and train a text processing model according to the original text information and the target similar text information to obtain a trained text processing model, which is used for text recognition. Since the embodiments of the present application can determine the target similar text information corresponding to the original text information, training the text processing model based on the original text information and the target similar text information can improve the generalization performance and accuracy of the trained text processing model. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0028] Figure 1 is a scene schematic diagram of the text processing model training method provided by the embodiments of the present application;

[0029] Figure 2 is a flowchart of the text processing model training method provided by the embodiments of the present application;

[0030] Figure 3 is a flowchart of determining the target similar text information corresponding to the original text information according to the original text features provided by the embodiments of the present application;

[0031] Figure 4 is a flowchart of determining the target similar text information corresponding to the original text information according to the original text features and the reference text features provided by the embodiments of the present application;

[0032] Figure 5 is a flowchart of the text processing model training method provided by the embodiments of the present application;

[0033] Figure 6 is a schematic diagram of encoding and training the initial encoder of the initial text processing model based on the original text information and the label to obtain the text processing model provided by the embodiments of the present application;

[0034] Figure 7 is a schematic diagram of obtaining the target similar text information provided by the embodiments of the present application;

[0035] Figure 8 is a flowchart of training the text processing model according to the original text information and the target similar text information to obtain the trained text processing model provided by the embodiments of the present application;

[0036] Figure 9 is a structural schematic diagram of the text processing model training device provided by the embodiments of the present application;

[0037] Figure 10 is a structural schematic diagram of the computer device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0038] With reference to the drawings, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the scope of the present application.

[0039] The embodiment of the present application provides a text processing model training method and device, computer equipment and a computer readable storage medium. The text processing model training device can be integrated in the computer equipment, which can be a server or a terminal device.

[0040] The server can be a stand-alone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and basic cloud computing services such as big data and artificial intelligence platforms. The terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a smart television, a smart vehicle device, etc., but is not limited thereto. The terminal and the server can be directly or indirectly connected through wired or wireless communication, which is not limited in the present application.

[0041] The embodiment of the present application relates to artificial intelligence (AI). Artificial intelligence is the use of digital computers or computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which aims to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0042] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software technologies. Artificial intelligence basic technologies generally include sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0043] For example, see Figure 1For example, the text processing model training apparatus is integrated in a computer device, and the embodiments of the present application can obtain original text information of training text; perform feature extraction on the original text information to obtain original text features of the original text information; determine target similar text information corresponding to the original text information according to the original text features; train the text processing model according to the original text information and the target similar text information to obtain a trained text processing model, and the trained text processing model is used for text recognition.

[0044] The original text information is text information of the training text mapped into a dimensional space, and the original text information can be expressed in the form of a matrix.

[0045] The target similar text information can be text information similar to the original text information. When the target similar text information contains some information same as the original text information, the target similar text information and the original text information can be considered similar. Or, when the similarity between the target similar text information and the original text information meets a preset similarity threshold, the target similar text information and the original text information can be considered similar.

[0046] The text processing model can be a GAN neural network model or a Transformer neural network model.

[0047] The following will be described in detail. It should be noted that the order of the following embodiments is not limited as the preferred order of the embodiments.

[0048] The present embodiment will be described from the perspective of a text processing model training apparatus, which can be integrated in a computer device, which can be a server, a terminal, or other devices.

[0049] As shown in Figure 2 The specific process of the text processing model training method is as follows, which can be executed by a computer device:

[0050] S101, obtaining original text information of training text.

[0051] The original text information is text information of the training text mapped into a dimensional space, and the original text information can be expressed in the form of a matrix.

[0052] The embodiments of the present application can perform feature extraction on the training text by the text processing model to obtain the original text information of the training text.

[0053] Specifically, the text processing model in this embodiment may have an embedding layer. Based on this, the text processing model in this embodiment can map the training text into the dimensional space through the embedding layer, thereby extracting features from the training text and obtaining the original text information of the training text.

[0054] This application embodiment can also use other methods to obtain the original text information of the training text. For example, this application embodiment can use other text extraction models to extract features from the training text to obtain the original text information of the training text, and store the extracted original text information in the original text set. Based on this, in this application embodiment, the computer device can directly extract the original text information from the original text set.

[0055] S102. Extract features from the original text information to obtain the original text features.

[0056] The embodiments of this application can use a text processing model to extract features from the original text information, thereby obtaining the original text features of the original text information.

[0057] In this embodiment, the text processing model may include an encoder and a decoder. This embodiment uses the encoder of the text processing model to encode the original text information, obtaining the original text features.

[0058] Before extracting features from the original text information to obtain the original text features, in order to improve the accuracy of the extracted original text features, this embodiment of the application can train the initial encoder of the initial text processing model to obtain the text processing model. The encoder of the text processing model is obtained by training the initial encoder of the initial text processing model.

[0059] Specifically, in this application embodiment, the process of encoding and training the initial encoder of the initial text processing model can be as follows: the computer device obtains the labels corresponding to the original text information of the training text; based on the original text information and labels, the initial encoder of the initial text processing model is encoded and trained to obtain the text processing model.

[0060] In this embodiment, the initial encoder of the initial text processing model is used to initially encode the original text information to obtain initial encoded features; the initial encoded features are classified to obtain classification results; the initial loss value between the classification results and the labels is calculated; and based on the initial loss value, the initial encoder of the initial text processing model is trained to obtain the text processing model.

[0061] In the embodiment of the present application, the initial encoding features can be classified in the form of binary classification. Of course, the present application is not limited to the binary classification method.

[0062] S103, determining target similar text information corresponding to the original text information according to the original text features.

[0063] The target similar text information refers to text information similar to the original text information. When the target similar text information contains some information same as the original text information, the target similar text information and the original text information can be considered similar. Or, when the similarity between the target similar text information and the original text information meets a preset similarity threshold, the target similar text information and the original text information can be considered similar.

[0064] There are various ways to determine the target similar text information corresponding to the original text information according to the original text features in the embodiment of the present application.

[0065] For example, the embodiment of the present application can obtain a preset feature with the same dimension as the original text feature based on the dimension of the original text feature; and fuse the preset feature and the original text feature to obtain the target similar text information.

[0066] The preset feature can be represented in the form of a matrix. For example, the original text feature is a three-dimensional matrix, and based on this, the obtained preset feature is also a three-dimensional matrix. The embodiment of the present application can add the original text feature and the preset feature or subtract the original text feature and the preset feature to fuse the preset feature and the original text feature, thereby obtaining the target similar text information.

[0067] Of course, the way of fusing the original feature and the preset feature in the embodiment of the present application is not limited to the addition or subtraction method, which can be set according to specific needs.

[0068] For another example, as shown in the following figure, Figure 3 The process of determining the target similar text information corresponding to the original text information according to the original text features in the embodiment of the present application can be as follows:

[0069] A1, determining a reference text feature matching the dimension of the original text feature.

[0070] The reference text feature can be expressed in the form of a matrix.

[0071] When the dimension of the reference text feature is the same as the dimension of the original text feature, the reference text feature can be considered to match the original text feature; when the reference text feature has at least two, the reference text feature is fused to obtain a fused feature, and when the dimension of the fused feature is the same as the dimension of the original text feature, the reference text feature can be considered to match the original text feature.

[0072] For example, when the reference text feature has one, the reference text feature is an n*n matrix, the original text feature is an n*n matrix, and n is a positive integer. In this case, the reference text feature can be considered to match the original text feature.

[0073] For example, the reference text feature can be multiplied; for example, the reference text feature can be divided.

[0074] For example, when the reference text feature has two, for example, the reference text feature includes a first reference text feature and a second reference text feature. The first reference text feature is a 1*n matrix, the second reference text feature is an n*1 matrix, and the original text feature is an n*n matrix. n is a positive integer. The first reference text feature and the second reference text feature are multiplied to obtain a fused feature, and the fused feature is an n*n matrix. Based on this, the original text feature and the reference text feature can be considered to match.

[0075] The embodiments of the present application determine the reference text feature corresponding to the dimension of the original text feature as follows:

[0076] Specifically, the computer device obtains reference text information matching the dimension of the original text information; the reference text information is extracted to obtain reference text features matching the dimension of the original text features.

[0077] The reference text information can be expressed in the form of a matrix. The reference text information can be text information randomly assigned by the computer device, and the reference text information can be text information randomly initialized by the text processing model.

[0078] When the dimension of the reference text information is the same as the dimension of the original text information, the reference text information can be considered to match the original text information; when the reference text information has at least two, the reference text information is fused to obtain a fused information, and when the dimension of the fused information is the same as the dimension of the original text information, the reference text information can also be considered to match the original text information.

[0079] For example, when the reference text information has one, the reference text information is an m*m matrix, the original text information is an m*m matrix, and m is a positive integer. In this case, the reference text information and the original text information can be matched.

[0080] For example, when the reference text information has two, for example, the reference text information includes first reference text information and second reference text information. The first reference text information is a 1*m matrix, the second reference text information is an m*1 matrix, and the original text information is an m*m matrix. m is a positive integer. In this case, the original text feature and the reference text feature can be matched by multiplying the first reference text information and the second reference text information to obtain fusion information, and the fusion feature is an n*n matrix.

[0081] A2, determining target similar text information corresponding to the original text information according to the original text feature and the reference text feature.

[0082] There are various ways to determine the target similar text information corresponding to the original text information according to the original text feature and the reference text feature. For example, the original text feature and the reference text feature can be fused to obtain the target similar text information.

[0083] In addition to the above manner, the embodiments of the present application can also use other manners, for example, as shown in Figure 4 The embodiments of the present application can determine the target similar text information corresponding to the original text information according to the original text feature and the reference text feature in the following manner:

[0084] B1, calculating the difference information between the original text feature and the reference text feature according to the original text feature and the reference text feature.

[0085] The embodiments of the present application can calculate the similarity between the original text feature and the reference text feature, and take the similarity between the original text feature and the reference text feature as the difference information between the original text feature and the reference text feature.

[0086] There are various ways to calculate the similarity between the original text feature and the reference text feature, for example, cosine similarity, Gaussian kernel similarity.

[0087] The embodiments of the present application can specifically calculate the target distance information between the original text feature and the reference text feature in the mapping space according to the original text feature and the reference text feature; and take the target distance information as the difference information.

[0088] In the formula (1), the embodiments of the present application calculate the target distance information between the original text feature and the reference text feature in the mapping space.

[0089] d = ||f1-f2|| Formula (1)

[0090] Wherein, d represents target distance information, f1 represents original text feature, and f2 represents reference text feature.

[0091] B2, according to the difference information, determine the target similar text information corresponding to the original text information.

[0092] Wherein, the difference information is fed back to the input layer of the encoder of the text processing model, so as to obtain the target similar text information.

[0093] The method of feeding back the difference information to the input layer of the encoder of the text processing model in the embodiment of the application can be a back propagation method, that is, a BP algorithm. Through the back propagation method, the parameters of the text processing model can be updated.

[0094] S104, according to the original text information and the target similar text information, training the text processing model to obtain the trained text processing model.

[0095] Wherein, the trained text processing model is used for text recognition. For example, the trained text processing model is used for title recognition.

[0096] Wherein, the text processing model can be a GAN neural network model, and can be a Transformer neural network model.

[0097] The embodiment of the application can use the original text information and the target similar text information to train the text processing model at the same time. The text processing model can be a self-encoder, including an encoder and a decoder. In this way, the encoder and the decoder can learn the features that have not been learned before sample reconstruction, that is, the original text features of the original text information can be learned for targeted sample enhancement.

[0098] The embodiment of the application can train the text processing model according to the original text information and the target similar text information to obtain the trained text processing model in the following way:

[0099] Specifically, the computer device predicts the original text information to obtain the predicted original text information; predicts the target similar text information to obtain the predicted similar text information; according to the original text information, the predicted original text information, the target similar text information, and the predicted similar text information, trains the text processing model to obtain the trained text processing model.

[0100] Wherein, the embodiment of the application trains the text processing model according to the original text information, the predicted original text information, the target similar text information, and the predicted similar text information, and obtains the trained text processing model as follows:

[0101] Specifically, a first loss value between the original text information and the predicted original text information is calculated according to the original text information and the predicted original text information; a second loss value between the target similar text information and the predicted similar text information is calculated according to the target similar text information and the predicted similar text information; and the text processing model is trained according to the first loss value and the second loss value to obtain the trained text processing model.

[0102] Wherein, the embodiment of the application trains the text processing model until the discriminator of the text processing model cannot distinguish the difference between the original text information and the predicted original text information, and the discriminator of the text processing model cannot distinguish the difference between the target similar text information and the predicted similar text information.

[0103] The embodiment of the application can obtain the original text information of the training text; perform feature extraction on the original text information to obtain the original text feature of the original text information; determine the target similar text information corresponding to the original text information according to the original text feature; and train the text processing model according to the original text information and the target similar text information to obtain the trained text processing model. Since the embodiment of the application can determine the target similar text information corresponding to the original text information, training the text processing model based on the original text information and the target similar text information can improve the generalization performance and accuracy of the obtained trained text processing model.

[0104] According to the method described in the above embodiment, the following examples will be further described in detail.

[0105] In this embodiment, the text processing model training device is specifically integrated in a computer device, which is a server.

[0106] As shown in FIG. Figure 5 A text processing model training method, and the specific process is as follows:

[0107] S201, the computer device obtains the original text information of the training text and the label corresponding to the original text information.

[0108] In order to improve the accuracy of the original text feature extracted by the embodiment of the application, the embodiment of the application can perform encoding training on the initial encoder of the initial text processing model.

[0109] In this embodiment, a supervised training method can be used to train the initial encoder of the initial text processing model. This allows the text processing model obtained after training the initial encoder of the initial text processing model to accurately extract the original text features that are strongly correlated with the labels. Based on this, this embodiment obtains the original text information of the training text and the labels corresponding to the original text information to train the initial encoder of the initial text processing model.

[0110] The embodiments of this application can extract features from the training text through the embedding layer of the initial text processing model, thereby obtaining the original text information of the training samples.

[0111] In this application embodiment, the training text can be title text or paragraph text. For example, taking title text as an example, the training text can include multiple types of training text, such as titles suspected of containing words of personal attack, or titles suspected of containing words of violation.

[0112] Based on the above, the embodiments of this application can divide the training text into multiple training subsets. Taking title text as an example, the training text includes training text of multiple text types. The training text is divided into different training subsets according to the text type, with each text type corresponding to one training subset.

[0113] Based on this, embodiments of this application can extract features from training texts according to different training subsets, thereby obtaining the original text information set corresponding to each text category. The original text information set corresponding to each text category includes several original text information items corresponding to each text category. This application implementation can train the initial encoder of the initial text processing model by encoding according to the original text information set and labels corresponding to each text category.

[0114] Based on the above, embodiments of this application can store the original text information set corresponding to each text type. Furthermore, embodiments of this application can also use a visual method to select the original text information set, thereby improving the training efficiency of the text processing model and the initial text processing model.

[0115] Specifically, in the embodiments of this application, the visualization method can be as follows: a computer device displays a text information selection page, the text information selection page includes at least one set of original text information of at least one text type, each set of original text information of one text type corresponds to an original text set of one text type; in response to the selection operation of the original text information set option, at least one original text information set option is selected from the original text information set option of at least one text type to obtain at least one original text information set, that is, to obtain the original text information.

[0116] S202, the computer device encodes and trains the initial encoder of the initial text processing model based on the original text information and the label to obtain the text processing model.

[0117] The initial text processing model in the embodiment of the application can be a Transformer neural network model.

[0118] The embodiment of the application can use the initial encoder of the initial text processing model to initially encode the original text information to obtain initial encoding features, classify the initial encoding features to obtain classification results, calculate an initial loss value between the classification results and the label, and encode and train the initial encoder of the initial text processing model based on the initial loss value to obtain the text processing model.

[0119] The embodiment of the application can update the network parameters of the initial text processing model through gradient back propagation to obtain the text processing model.

[0120] As shown in the figure, Figure 6 The embodiment of the application takes the first original text information and the second original text information as examples for illustration.

[0121] The first original text information and the second original text information can both be text information of a title suspected to contain a personal attack word category, and the training text corresponding to the first original text information and the training text corresponding to the second original text information are different in specific words.

[0122] The first original text information and the second original text information are input into the initial encoder of the initial text processing model, the initial encoder of the initial text processing model initially encodes the first original text information to obtain first initial encoding features, and the initial encoder of the initial text processing model initially encodes the second original text information to obtain second initial encoding features. The embodiment of the application classifies the first initial encoding features through the softmax layer of the initial text processing model to obtain first classification results, and classifies the second initial encoding features through the softmax layer of the initial text processing model to obtain second classification results. Based on this, the embodiment of the application encodes and trains the initial encoder of the initial text processing model based on the first classification results, the second classification results, and the label to obtain the text processing model.

[0123] The softmax layer in the embodiment of the application can be set to one layer, i.e., the same layer of softmax classifies the original text information; the embodiment of the application can also use different softmax layers to classify original text information of different text categories.

[0124] S203, the computer device obtains reference text information matching the dimension of the original text information.

[0125] The reference text information can be expressed in matrix form. This reference text information can be text information randomly assigned by a computer device, or text information randomly initialized by a text processing model.

[0126] In this case, when the dimension of the reference text information is the same as the dimension of the original text information, the reference text information and the original text information can be said to match. When there are at least two reference text information, the reference text information is fused to obtain fused information. When the dimension of the fused information is the same as the dimension of the original text information, the reference text information and the original text information can also be said to match.

[0127] For example, when there is one reference text information, the reference text information is an m×m matrix, and the original text information is an m×m matrix, where m is a positive integer. In this case, the reference text information and the original text information can be said to match.

[0128] For example, when there are two reference text information, such as a first reference text information and a second reference text information, the first reference text information is a 1×m matrix, the second reference text information is an m×1 matrix, and the original text information is an m×m matrix, where m is a positive integer. In this embodiment, the first and second reference text information are multiplied to obtain fused information, and the fused features are an n×n matrix. Based on this, this situation can be called original text feature and reference text feature matching.

[0129] S204. The computer equipment uses a text processing model to extract features from the original text information and the reference text information, thereby obtaining the original text features of the original text information and the reference text features of the reference text information.

[0130] like Figure 7 As shown in the embodiments of this application, the text processing model may include an encoder and a decoder. In this embodiment, the encoder of the text processing model encodes the original text information to obtain the original text features; the encoder of the text processing model encodes the reference text information to obtain the original text features.

[0131] S205. The computer device determines the target similar text information corresponding to the original text information based on the original text features and the reference text features.

[0132] When the reference text information is text information randomly initialized by the text processing model, if the following objective process is repeated multiple times, the reference text features corresponding to the reference text information can be made very similar to the original text features of the original text information. At the same time, the reference text information only contains information learned on the encoder of the text processing model.

[0133] The specific process is as follows: input randomly initialized text information and original text information into the text processing model for forward propagation to obtain reference text features and original text features; based on the reference text features and original text features, calculate the difference information between the original text features and the reference text features; based on the difference information, perform backpropagation in the text processing model to update the network parameters of the text processing model, thereby updating the randomly initialized text information, and obtaining the updated randomly initialized text information, which is also the target similar text information corresponding to the original text information.

[0134] In this context, target-similar text information refers to text information that is similar to the original text information. When the target-similar text information contains some information that is the same as the original text information, it can be said that the target-similar text information and the original text information are similar; or, when the similarity between the target-similar text information and the original text information meets a preset similarity threshold, it can be said that the target-similar text information and the original text information are similar.

[0135] This application embodiment has multiple methods for determining target similar text information corresponding to the original text information based on the original text features.

[0136] For example, embodiments of this application can obtain preset features with the same dimensions as the original text features based on the dimensions of the original text features; and fuse the preset features with the original text features to obtain target similar text information.

[0137] For example, such as Figure 7 As shown in the embodiment of this application, the method for determining the target similar text information corresponding to the original text information based on the original text features and the reference text features can be as follows: the computer device calculates the difference information between the original text features and the reference text features based on the original text features and the reference text features; and determines the target similar text information corresponding to the original text information based on the difference information.

[0138] In this embodiment, the difference information is backpropagated to the input layer of the encoder of the text processing model to obtain the target similar text information. The method used in this embodiment to backpropagate the difference information to the input layer of the encoder of the text processing model can be backpropagation, i.e., the BP algorithm. This embodiment uses this backpropagation method to update the parameters of the text processing model.

[0139] In the embodiment of the application, the target distance information between the original text feature and the reference text feature in the mapping space can be calculated according to the original text feature and the reference text feature, and the target distance information is used as the difference information.

[0140] In the embodiment of the application, the target distance information between the original text feature and the reference text feature in the mapping space can be calculated according to the original text feature and the reference text feature, and the target distance information is used as the difference information.

[0141] In the embodiment of the application, the target distance information between the original text feature and the reference text feature in the mapping space can be calculated according to the original text feature and the reference text feature, and the target distance information is used as the difference information.

[0142] In the embodiment of the application, the target distance information between the original text feature and the reference text feature in the mapping space can be calculated according to the original text feature and the reference text feature, and the target distance information is used as the difference information.

[0143] In the embodiment of the application, the target distance information between the original text feature and the reference text feature in the mapping space can be calculated according to the original text feature and the reference text feature, and the target distance information is used as the difference information.

[0144] Specifically, the encoder of the text processing model can be trained in a traditional classification manner, based on which, the encoder of the text processing model can extract the encoded original text feature of the original text information and the encoded similar text feature of the target similar text information. When training the autoencoder, the minimization of the reconstruction loss can automatically optimize the encoder and the decoder. If the representations of the original text information and the target similar text information are as different as possible, the encoder of the text processing model learns the text information before the original text information is reconstructed, that is, the encoder of the text processing model learns the target similar text information.

[0145] In the embodiment of the application, the original text information and the target similar text information can be regarded as positive and negative examples after sample enhancement, so that the text processing model captures more information during the training process.

[0146] In the embodiment of the application, the original text information and the target similar text information can be regarded as positive and negative examples after sample enhancement, so that the text processing model captures more information during the training process.

[0147] Specifically, the computer device predicts the original text information to obtain predicted original text information, and predicts the target similar text information to obtain predicted similar text information; and trains the text processing model according to the original text information, the predicted original text information, the target similar text information, and the predicted similar text information, to obtain a trained text processing model.

[0148] As shown in Figure 8 The embodiment of the present application encodes the original text information by using an encoder of the text processing model to obtain encoded original text features, and encodes the target similar text information by using the encoder of the text processing model to obtain encoded similar text features. Then, the embodiment of the present application inputs the encoded original text features into a decoder of the text processing model to decode to obtain original text decoding information, which is the predicted original text information; and inputs the encoded similar text features into the decoder of the text processing model to decode to obtain similar text decoding information, which is the predicted similar text information.

[0149] In the embodiment of the present application, when the reference text information is the text information randomly initialized by the text processing model, the embodiment of the present application uses the reference text information to obtain the target similar text information corresponding to the original text information. When the text processing model encodes the target similar text information and the original text information, the encoded similar text information corresponding to the target similar text information is very similar to the encoded original text features corresponding to the original text information.

[0150] In the embodiment of the present application, the training of the text processing model according to the original text information, the predicted original text information, the target similar text information, and the predicted similar text information can be as follows:

[0151] Specifically, a first loss value between the original text information and the predicted original text information is calculated according to the original text information and the predicted original text information; a second loss value between the target similar text information and the predicted similar text information is calculated according to the target similar text information and the predicted similar text information; and the text processing model is trained according to the first loss value and the second loss value to obtain a trained text processing model.

[0152] The text processing model of the embodiment of the present application can be a text classification model. Based on this, the trained text processing model can be applied to online application of text classification. Especially when there is a large amount of homogenized text information in the training sample, the cold start of the text processing model can improve the accuracy and recall rate of the text processing model for online data classification. At the same time, the reference text information can also be used as negative examples of the original text information of the training sample, thereby improving the generalization performance of the text processing model.

[0153] When the text processing model is a text classification model, the text processing model of the embodiment of the present application can use the encoder of the text processing model as a classifier, and train the encoder of the text processing model by using the original text information and the reference text information in an adversarial manner.

[0154] The embodiment of the present application can obtain original text information of training text; perform feature extraction on the original text information to obtain original text features of the original text information; determine target similar text information corresponding to the original text information according to the original text features; and train the text processing model according to the original text information and the target similar text information to obtain a trained text processing model, which is used for recognizing text. Since the embodiment of the present application can determine the target similar text information corresponding to the original text information, training the text processing model based on the original text information and the target similar text information can improve the generalization performance and accuracy of the obtained trained text processing model.

[0155] In order to better implement the above method, the embodiment of the present application further provides a text processing model training device, which can be integrated in a computer device, such as a server or a terminal.

[0156] For example, as shown in FIG. 3, the text processing model training device can include an obtaining unit 301, an extracting unit 302, a determining unit 303, a training unit 304, and an encoding training unit 305. Figure 9

[0157] (1) The obtaining unit 301;

[0158] The obtaining unit 301 can be used to obtain original text information of training text.

[0159] (2) The extracting unit 302;

[0160] The extracting unit 302 can be used to perform feature extraction on the original text information to obtain original text features of the original text information.

[0161] ​In some embodiments of the application, the extraction unit 302 can be specifically configured to extract features of the original text information by using a text processing model to obtain original text features of the original text information.

[0162] (3) The determination unit 303;

[0163] The determination unit 303 can be configured to determine target similar text information corresponding to the original text information according to the original text features.

[0164] In some embodiments, the determination unit 303 can be specifically configured to determine reference text features matching the dimension of the original text features; and determine target similar text information corresponding to the original text information according to the original text features and the reference text features.

[0165] In some embodiments, the determination unit 303 can be specifically configured to calculate difference information between the original text features and the reference text features according to the original text features and the reference text features; and determine target similar text information corresponding to the original text information according to the difference information.

[0166] In some embodiments, the determination unit 303 can be specifically configured to calculate target distance information between the original text features and the reference text features in a mapping space according to the original text features and the reference text features; and take the target distance information as the difference information.

[0167] In some embodiments, the determination unit 303 can be specifically configured to obtain reference text information matching the dimension of the original text information; and extract features of the reference text information to obtain reference text features matching the dimension of the original text features.

[0168] (4) The training unit 304;

[0169] The training unit 304 can be configured to train the text processing model according to the original text information and the target similar text information to obtain a trained text processing model, and the trained text processing model is used for text recognition.

[0170] The training unit 304 can be specifically configured to predict the original text information to obtain predicted original text information; predict the target similar text information to obtain predicted similar text information; and train the text processing model according to the original text information, the predicted original text information, the target similar text information, and the predicted similar text information to obtain the trained text processing model.

[0171] In some embodiments, the training unit 304 can be specifically configured to calculate a first loss value between the original text information and the predicted original text information according to the original text information and the predicted original text information; calculate a second loss value between the target similar text information and the predicted similar text information according to the target similar text information and the predicted similar text information; and train the text processing model according to the first loss value and the second loss value to obtain the trained text processing model.

[0172] (5) an encoding training unit 305;

[0173] In some embodiments, the text processing model training apparatus further includes an encoding training unit 305, which can be configured to obtain a label corresponding to the original text information of the training text; and perform encoding training on an initial encoder of an initial text processing model based on the original text information and the label to obtain the text processing model.

[0174] As can be seen from the above, the obtaining unit 301 can obtain the original text information of the training text; the extracting unit 302 can be configured to perform feature extraction on the original text information to obtain original text features of the original text information; the determining unit 303 can be configured to determine the target similar text information corresponding to the original text information according to the original text features; and the training unit 304 can be configured to train the text processing model according to the original text information and the target similar text information to obtain the trained text processing model, which is used for text recognition. Since the target similar text information corresponding to the original text information can be determined, the text processing model can be trained based on the original text information and the target similar text information, so that the generalization performance and accuracy of the trained text processing model can be improved.

[0175] The embodiments of the present application also provide a computer device, as shown in Figure 10 The computer device is shown in the structure schematic diagram of the computer device according to the embodiments of the present application, and specifically:

[0176] The computer device can include a processor 401 with one or more processing cores, a memory 402 with one or more computer readable computer readable storage media, a power supply 403, an input unit 404, and the like. Those skilled in the art can understand that the structure of the computer device shown in Figure 10 The structure of the computer device shown in the above does not constitute a limitation on the computer device, and can include more or fewer components than those shown, or combine certain components, or different component arrangements. Among them:

[0177] The processor 401 is the control center of the computer device, connects the various parts of the computer device through various interfaces and lines, and performs various functions and processes data of the computer device by running or executing software programs and / or modules stored in the memory 402 and calling data stored in the memory 402, thereby overall detecting and controlling the computer device. Optionally, the processor 401 can include one or more processing cores; preferably, the processor 401 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, computer program, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 401.

[0178] The memory 402 can be used to store software programs and modules, and the processor 401 executes various functions and data processing by running the software programs and modules stored in the memory 402. The memory 402 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one computer program required by a function (such as a sound playing function, an image playing function, etc.), etc.; the data storage area can store data created according to the use of the computer device, etc. In addition, the memory 402 can include a high-speed reference access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device. Accordingly, the memory 402 can also include a memory controller to provide access of the processor 401 to the memory 402.

[0179] The computer device further includes a power supply 403 for powering various components, and preferably the power supply 403 can be logically connected to the processor 401 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. The power supply 403 can also include one or more than one direct current or alternating current power supply, a recharging system, a power failure detection circuit, a power converter or inverter, a power state indicator, etc. Any component.

[0180] The computer device can also include an input unit 404, which can be used to receive input digital or character information communication, and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0181] Although not shown, the computer device can also include a display unit, etc., which will not be described here. Specifically, in the present embodiment, the processor 401 in the computer device will load the executable file corresponding to the process of one or more computer programs into the memory 402 according to the following instructions, and run the computer program stored in the memory 402 by the processor 401, thereby realizing various functions, such as:

[0182] Obtaining original text information of the training text; performing feature extraction on the original text information to obtain original text features of the original text information; determining target similar text information corresponding to the original text information according to the original text features; and training the text processing model according to the original text information and the target similar text information to obtain a trained text processing model, the trained text processing model being used for text recognition.

[0183] The specific implementation of the above operations can refer to the foregoing embodiments, and will not be described here again.

[0184] Those skilled in the art can understand that all or part of the steps in the various methods of the foregoing embodiments can be completed by a computer program, or by a computer program controlling relevant hardware, the computer program can be stored in a computer readable storage medium and loaded and executed by a processor.

[0185] To this end, an embodiment of the present application provides a computer readable storage medium, which stores a computer program capable of being loaded by a processor to execute any one of the text processing model training methods provided by the embodiments of the present application.

[0186] The specific implementation of the above operations can refer to the foregoing embodiments, and will not be described here again.

[0187] The computer readable storage medium can include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0188] Since the instructions stored in the computer readable storage medium can execute the steps in any one of the text processing model training methods provided by the embodiments of the present application, the beneficial effects that can be achieved by any one of the text processing model training methods provided by the embodiments of the present application can be achieved, and details are described in the foregoing embodiments, which will not be described here again.

[0189] According to an aspect of the present application, a computer program product or a computer program is provided, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to enable the computer device to perform the method provided in any one of the various optional implementation manners provided by the foregoing embodiments.

[0190] The text processing model training method, the computer device, and the computer readable storage medium provided by the embodiments of the present application are described in detail above, and the principles and implementation manners of the present application are described by applying specific examples. The above embodiment description is only used to help understand the method of the present application and its core idea; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range will be changed, and the above description should not be understood as a limitation on the present application.

Claims

1. A method for training a text processing model, characterized in that, include: Obtain the original text information of the training text and the labels corresponding to the original text information; Based on the original text information and the tags, the initial encoder of the initial text processing model is trained to obtain the text processing model. The original text information is extracted using the text processing model described above to obtain the original text features of the original text information; Based on the original text features, target similar text information that is similar to the original text information is determined, wherein the original text information and the target similar text information are positive and negative pairs; The text processing model is used to predict the original text information and the target similar text information, which are positive and negative example pairs, to obtain the original text decoding information corresponding to the original text information and the similar text decoding information corresponding to the target similar text information. Calculate a first loss value between the original text information and the original text decoded information, and a second loss value between the target similar text information and the similar text decoded information; The text processing model is trained based on the first loss value and the second loss value until the discriminator of the text processing model can no longer distinguish the difference between the original text information and the original text decoding information, and the discriminator of the text processing model can no longer distinguish the difference between the target similar text information and the similar text decoding information, thus obtaining a trained text processing model, which is used to recognize text.

2. The text processing model training method according to claim 1, characterized in that, The step of determining target similar text information that is similar to the original text information based on the original text features includes: Determine reference text features that match the original text feature dimensions; Based on the original text features and the reference text features, target similar text information that is similar to the original text information is determined.

3. The text processing model training method according to claim 2, characterized in that, The step of determining target similar text information that is similar to the original text information based on the original text features and the reference text features includes: Based on the original text features and the reference text features, calculate the difference information between the original text features and the reference text features; Based on the difference information, target similar text information that is similar to the original text information is determined.

4. The text processing model training method according to claim 3, characterized in that, The step of calculating the difference information between the original text features and the reference text features based on the original text features and the reference text features includes: Based on the original text features and the reference text features, calculate the target distance information between the original text features and the reference text features in the mapping space; The target distance information is used as the difference information.

5. The text processing model training method according to claim 2, characterized in that, The determination of reference text features that match the original text feature dimensions includes: Obtain reference text information that matches the dimensions of the original text information; Feature extraction is performed on the reference text information to obtain reference text features that match the feature dimensions of the original text.

6. A text processing model training device, characterized in that, include: The acquisition unit is used to acquire the original text information of the training text and the labels corresponding to the original text information; The encoding training unit trains the initial encoder of the initial text processing model based on the original text information and the labels to obtain the text processing model. An extraction unit is used to extract features from the original text information using the text processing model to obtain the original text features of the original text information. The determining unit is configured to determine target similar text information that is similar to the original text information based on the original text features, wherein the original text information and the target similar text information are positive and negative pairs; The training unit is used to predict the original text information and the target similar text information, which are positive and negative example pairs, using the text processing model to obtain the original text decoding information corresponding to the original text information and the similar text decoding information corresponding to the target similar text information. It calculates a first loss value between the original text information and the original text decoding information, and a second loss value between the target similar text information and the similar text decoding information. Based on the first loss value and the second loss value, the text processing model is trained until the discriminator of the text processing model can no longer distinguish the difference between the original text information and the original text decoding information, and the discriminator of the text processing model can no longer distinguish the difference between the target similar text information and the similar text decoding information, thus obtaining a trained text processing model. The trained text processing model is used for text recognition.

7. A computer device, characterized in that, It includes a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to perform the text processing model training method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted for loading by a processor to execute the text processing model training method according to any one of claims 1 to 5.

Images