Model training method, text recognition method and device for text recognition

Abstract

The present disclosure provides a model training method, a text recognition method and a device for text recognition, which relate to artificial intelligence, and in particular to the fields of natural language processing, deep learning and semantic analysis. The specific implementation scheme is as follows: obtaining a first set of texts to be trained including texts without category labels and a second set of texts to be trained including a preset second category label, the second set of texts to be trained including multiple text subsets; training the initial model based on the first set of texts to be trained to obtain a first model; repeating the following steps, where the initial value of i is 1: inputting the i-th text subset into the first model to obtain a trained first model; determining that the trained first model is a new first model, and determining that the value of i is i+1; determining that the new first model obtained when the preset conditions are met is a text recognition model. The obtained text recognition model can accurately identify whether the text is a text with a preset category label.

Description

Technical Field

[0001] The present disclosure relates to fields such as natural language processing, deep learning, and semantic analysis in artificial intelligence, and in particular to a model training method, a text recognition method, and a device for text recognition. Background Art

[0002] With the development of the Internet and media, the need for text review is becoming increasingly prominent. It is necessary to identify text and then determine whether it is illegal, such as pornographic or violent text.

[0003] In the prior art, each text annotated with a category label can be processed based on a shallow neural network model, and then the shallow neural network model can be trained to obtain a model for recognizing the category labels of the text.

[0004] However, in the above method, the training is based on a shallow neural network model. Since the structure of the model is relatively simple, the recognition rate of the trained model is not high, resulting in poor text recognition effect. Summary of the Invention

[0005] The present disclosure provides a model training method, a text recognition method, and an apparatus for obtaining a model that can accurately recognize category labels of text, which are applied to text recognition.

[0006] According to a first aspect of the present disclosure, a model training method for text recognition is provided, comprising:

[0007] Obtaining a first set of texts to be trained and a second set of texts to be trained, wherein each first text to be trained in the first set of texts to be trained is a text without a category label, and the second set of texts to be trained includes a plurality of text subsets, and each second text to be trained in the text subsets is a text with a preset category label; training an initial model based on the first set of texts to be trained to obtain a first model;

[0008] Repeat the following steps until a preset condition is met, wherein the initial value of i is 1 and i is a positive integer greater than or equal to 1: input the i-th text subset into the first model to obtain a trained first model; determine the trained first model as a new first model, and determine the value of i to be i+1; wherein different text subsets have different numbers of texts; and different text subsets have different data sources;

[0009] The new first model obtained when it is determined that the preset condition is met is a text recognition model for identifying the preset category label.

[0010] According to a second aspect of the present disclosure, a text recognition method is provided, comprising:

[0011] Obtaining a text to be recognized, inputting the text to be recognized into a text recognition model, and obtaining a category label for the text to be recognized;

[0012] The text recognition model is a model obtained by using the method provided in the first aspect.

[0013] According to a third aspect of the present disclosure, a model training device for text recognition is provided, comprising:

[0014] A first acquisition unit, configured to acquire a first text set to be trained;

[0015] A second acquisition unit is configured to obtain a second set of texts to be trained, wherein each first text to be trained in the first set of texts to be trained is a text without a category label, and the second set of texts to be trained includes a plurality of text subsets, and each second text to be trained in the text subsets is a text with a preset category label;

[0016] A first training unit is configured to train an initial model based on the first set of texts to be trained to obtain a first model;

[0017] The second training unit is configured to repeat the following steps until a preset condition is met, wherein the initial value of i is 1 and i is a positive integer greater than or equal to 1: inputting the i-th text subset into the first model to obtain a trained first model; determining the trained first model as a new first model, and determining the value of i to be i+1; wherein different text subsets have different numbers of texts; and different text subsets have different data sources;

[0018] The new first model obtained when it is determined that the preset condition is met is a text recognition model for identifying the preset category label.

[0019] According to a fourth aspect of the present disclosure, a text recognition device is provided, comprising:

[0020] An acquisition unit, used for acquiring the text to be recognized;

[0021] A recognition unit, configured to input the text to be recognized into a text recognition model to obtain a category label of the text to be recognized;

[0022] The text recognition model is a model obtained by using the device provided in the third aspect.

[0023] According to a fifth aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above method.

[0024] According to a sixth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the above method.

[0025] According to the seventh aspect of the present disclosure, a computer program product is provided, comprising: a computer program, wherein the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program so that the electronic device executes the method described in the first aspect.

[0026] According to the technical method disclosed in the present invention, the initial model is trained based on other texts without category labels to obtain a first model. Then the following process is repeated: the first model is trained using the first text subset consisting of the second text to be trained with a preset category label and the second text to be trained without a preset category label; the trained first model is trained again using the second text to be trained with a preset category label and the second text to be trained without a preset category label; and so on. Finally, a text recognition model for identifying the preset category label is obtained. Among them, the data sources of different text subsets are different. Based on multiple model trainings and using different text data to train the model, the obtained model can learn semantic information and category label information; and the text recognition model obtained can accurately identify whether the text is under the preset category label. This is conducive to the accurate identification of illegal texts.

[0027] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0029] Figure 1 This is a scenario where the embodiments of the present disclosure can be implemented. Figure 1 ;

[0030] Figure 2This is a scenario where the embodiments of the present disclosure can be implemented. Figure 2 ;

[0031] Figure 3 is a schematic diagram according to a first embodiment of the present disclosure;

[0032] Figure 4 is a schematic diagram according to a second embodiment of the present disclosure;

[0033] Figure 5 is a model training graph according to the second embodiment of the present disclosure;

[0034] Figure 6 is a training graph based on the first text subset according to the second embodiment of the present disclosure;

[0035] Figure 7 is a training graph based on a second text subset according to the second embodiment of the present disclosure;

[0036] Figure 8 is a schematic diagram according to a third embodiment of the present disclosure;

[0037] Figure 9 is a schematic diagram according to a fourth embodiment of the present disclosure;

[0038] Figure 10 is a schematic diagram according to a fifth embodiment of the present disclosure;

[0039] Figure 11 is a schematic diagram according to a sixth embodiment of the present disclosure;

[0040] Figure 12 is a schematic diagram according to a seventh embodiment of the present disclosure;

[0041] Figure 13 is a schematic diagram according to an eighth embodiment of the present disclosure;

[0042] Figure 14 is a schematic diagram according to a ninth embodiment of the present disclosure;

[0043] Figure 15 A schematic block diagram of an example electronic device 1500 is shown, which may be used to implement embodiments of the present disclosure. DETAILED DESCRIPTION

[0044] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0045] With the development of the internet and media, the need for text moderation is becoming increasingly prominent. This requires identifying text and then determining its category label. Based on this category label, the text is then judged as illegal, such as pornographic or violent. The illegal text is then processed. Examples of category labels include pornography, violence, advertising, and so on.

[0046] However, due to the massive amount of user data generated daily on the Internet, human labor is unable to handle the arduous task of identifying text categories. Therefore, a method that can automatically identify text category labels is needed. With the development of current computer and natural language processing technologies, it is possible to train a model to identify text category labels.

[0047] For example, it is necessary to identify the category labels of text in scenarios such as user signatures, user nicknames, comments, messages, instant messaging text, user posts, media information, product information, live video comments, and graphic information to determine whether the text is illegal. This requires a text recognition method; a model for identifying text category labels can be obtained through model training.

[0048] In one example, texts labeled with category labels can be obtained; the texts labeled with category labels are input into a shallow neural network model; each text labeled with category labels is processed based on the shallow neural network model, and then the shallow neural network model is trained once to obtain a model for recognizing the category labels of the text.

[0049] However, in the above method, the training is based on a shallow neural network model. Since the structure of the model is relatively simple, the recognition rate of the trained model is not high, resulting in poor text recognition effect.

[0050] In another example, text labeled with category labels can be obtained; the text labeled with category labels is input into a language model, for example, the language model is a bidirectional encoder representation (Bidirectional Encoder Representation from Transformers, referred to as BERT) model or an enhanced representation through knowledge integration (Enhanced representation through knowledge integration, referred to as ERNIE) model, and then a model for identifying category labels of the text is obtained.

[0051] However, in the above method, since less text data is used, insufficient information transfer will occur during the model migration process, resulting in a low recognition rate of the obtained model.

[0052] In another example, a sensitive dictionary matching method can be used to match words in the text with sensitive words in a sensitive dictionary, where the sensitive dictionary includes multiple sensitive words, to further determine the sensitive words in the text; based on the sensitive words in the text, the category label of the text is determined.

[0053] However, this approach requires significant manpower to build a sensitive dictionary that covers a wide range of sensitive words, which is costly. Furthermore, methods based solely on matching cannot accurately identify the category labels of text; methods based on sensitive dictionary matching also have poor generalization capabilities.

[0054] The present disclosure provides a model training method, a text recognition method, and an apparatus for text recognition, which are applied to fields such as natural language processing, deep learning, and semantic analysis in artificial intelligence to achieve the purpose of obtaining a model that can accurately recognize category labels of text.

[0055] Figure 1 This is a scenario where the embodiments of the present disclosure can be implemented. Figure 1 ,like Figure 1 As shown in FIG, in a scenario where users are performing instant messaging, a user sends message 1 and another user sends message 2. It is necessary to identify the category labels of message 1 and message 2 respectively to identify illegal texts.

[0056] Figure 2 This is a scenario where the embodiments of the present disclosure can be implemented. Figure 2 ,like Figure 2 As shown in the figure, a user posts a message on a forum. The message needs to be categorized to determine whether it is a violation of the rules. Other users can comment on the message, for example, other users post comments 1 and 2. The comments need to be categorized to determine whether they are a violation of the rules.

[0057] Figure 3 is a schematic diagram according to the first embodiment of the present disclosure, as shown in Figure 3 As shown, the model training method for text recognition provided in this embodiment includes:

[0058] S301. Obtain a first set of texts to be trained and a second set of texts to be trained, wherein each first text to be trained in the first set of texts to be trained is a text without a category label, and the second set of texts to be trained includes multiple text subsets, and each second text to be trained in the text subsets is a text with a preset category label.

[0059] For example, the execution subject of this embodiment can be a terminal device, a server, an electronic device, a model training device or device for text recognition, or other devices or equipment that can execute the method of this embodiment. This embodiment is described with the execution subject being an electronic device.

[0060] It is necessary to obtain the text to be trained. In this embodiment, two text sets are obtained, namely the first text set to be trained and the second text set to be trained.

[0061] The first text to be trained set includes a plurality of first texts to be trained; the first text to be trained is a text without a category label, that is, the first text to be trained does not have a category label.

[0062] The second set of texts to be trained includes multiple text subsets, each of which includes multiple second texts to be trained. The first texts to be trained are texts with preset category labels, i.e., the second texts to be trained do not have category labels. It should be noted that the second texts to be trained in the second set of texts to be trained are texts with the same category label (i.e., the preset category label).

[0063] For example, for category label 1, a second set of texts to be trained corresponding to category label 1 is obtained, the second set of texts to be trained includes multiple text subsets, and the second texts to be trained in each text subset have category label 1. Alternatively, for category label 1, a second set of texts to be trained corresponding to category label 1 is obtained, the second set of texts to be trained includes multiple text subsets, part of the second texts to be trained in each text subset has category label 1, and the remaining second texts to be trained in each text subset do not have category label 1 (for example, the remaining second texts to be trained in each text subset may be other category labels, or may not have any category label).

[0064] The first and second text sets to be trained may be acquired in advance; the first and second text sets to be trained are stored on other servers, and then the electronic device may acquire the first and second text sets to be trained.

[0065] Alternatively, the first text set to be trained and the second text set to be trained are acquired in advance; the electronic device stores the acquired first text set to be trained and the second text set to be trained in the electronic device, or in a database corresponding to the electronic device.

[0066] S302: Train the initial model based on the first text set to be trained to obtain a first model.

[0067] For example, a preset initial model is trained based on a first set of unlabeled texts to be trained, thereby obtaining a first model. The initial model may be a BERT model or an ERNIE model.

[0068] In one example, a large amount of unlabeled text is pre-obtained. An ERNIE model or a BERT model is trained based on this large amount of unlabeled text to generate an initial model. The "large amount of unlabeled text" refers to text that is not part of the first or second training text sets. Although the initial model has been trained on a large amount of unlabeled text, it has not yet learned how to identify category labels and cannot be applied to text type labels.

[0069] The first set of unlabeled text to be trained is then input into the initial model to train the initial model, thereby obtaining a first model. The number of texts in the first set of unlabeled texts can be smaller than the number of texts in the aforementioned large number of unlabeled texts. The resulting first model has been trained twice, but both times using texts without labeling. Consequently, it has not learned how to identify category labels, and thus cannot be applied to identifying text type labels.

[0070] S303. Repeat the following steps until a preset condition is met, wherein the initial value of i is 1 and i is a positive integer greater than or equal to 1: input the i-th text subset into the first model to obtain a trained first model; determine the trained first model as a new first model, and determine the value of i to be i+1; wherein the number of texts in different text subsets is different.

[0071] The new first model obtained when it is determined that the preset condition is met is a text recognition model for recognizing preset category labels.

[0072] For example, after step S302 , in order to obtain a model that can be prepared to recognize category labels of texts, the first model outputted in step S302 needs to be trained multiple times.

[0073] The first text subset in the second text set to be trained is input into the first model outputted in step S302 to train the first model. Since the first text subset is configured with preset category labels, part of the second texts to be trained in each text subset has a preset category label, and the remaining second texts to be trained in each text subset do not have the preset category label (for example, the remaining second texts to be trained in each text subset may be other category labels, or may not have any category labels). Thus, the trained first model obtained at this time has learned the ability to recognize category labels.

[0074] Then, the first text subset from the second text set to be trained is input into the first model output based on the first text subset to further train the first model output based on the first text subset; the trained first model is output, i.e., the first model output based on the second text subset (a new first model) is obtained. At this point, the trained first model has again learned the ability to recognize category labels. Because the data sources of the first and second text subsets are different, the first model can be trained at different levels.

[0075] For example, if the number of texts in the first text subset is different from the number of texts in the second text subset, the category label of the second text to be trained in the first text subset can be a category label automatically identified by the system; and the category label of the second text to be trained in the second text subset can be a manually annotated category label.

[0076] Then, the third text subset from the second text set to be trained is input into the first model output based on the second text subset to further train the first model output based on the second text subset; the trained first model is output, i.e., the first model output based on the third text subset (a new first model) is obtained. At this point, the trained first model has again learned the ability to recognize category labels. Because the data sources of the second and third text subsets are different, the first model can be trained at different levels.

[0077] And so on, until the preset condition is met. The preset condition may be that the maximum number of iterations is reached. Alternatively, if the second set of texts to be trained includes N text subsets, where N is a positive integer greater than or equal to 1, then the preset condition may be that i is N+1; that is, all text subsets in the second set of texts to be trained are trained.

[0078] The new first model obtained when the preset conditions are met is a text recognition model. This text recognition model can be used to identify the preset category label. Since step S303 is trained using text with the preset category label, the text recognition model output after step S303 is used to identify the preset category label.

[0079] For example, in step S302, the initial model is trained based on text without category labels to obtain a first model. After step S302, the first model is trained multiple times using positive samples with category label 1 (the second text to be trained with category label 1) and negative samples without category label 1 (the second text to be trained without category label 1), and the resulting text recognition model is used to identify category label 1. After step S302, the first model can also be trained multiple times using positive samples with category label 2 (the second text to be trained with category label 2) and negative samples without category label 2 (the second text to be trained without category label 2), and the resulting text recognition model is used to identify category label 2.

[0080] In this embodiment, the initial model is trained based on other texts without category labels to obtain a first model. The following process is repeated: the first model is trained using the second text to be trained with a preset category label and the first text subset consisting of the second text to be trained without the preset category label; the trained first model is trained again using the second text to be trained with a preset category label and the second text subset consisting of the second text to be trained without the preset category label; and so on. Finally, a text recognition model for identifying preset category labels is obtained. Among them, the data sources of different text subsets are different. Based on multiple model trainings and using different text data to train the model, the obtained model can learn semantic information and category label information; and the text recognition model obtained can accurately identify whether the text is under the preset category label. This is conducive to the accurate identification of illegal texts.

[0081] Figure 4 is a schematic diagram according to the second embodiment of the present disclosure, as shown in Figure 4 As shown, the model training method for text recognition provided in this embodiment includes:

[0082] S401: Obtain a third set of texts to be trained, wherein the third texts to be trained in the third set of texts to be trained are texts without category labels.

[0083] For example, the execution subject of this embodiment can be a terminal device, a server, an electronic device, a model training device or device for text recognition, or other devices or equipment that can execute the method of this embodiment. This embodiment is described with the execution subject being an electronic device.

[0084] The electronic device can obtain the third set of texts to be trained from other online systems or from other databases. The third set of texts to be trained includes multiple third texts to be trained. The third texts to be trained are texts without category labels, that is, each text in the third set of texts to be trained does not have any category label.

[0085] Among them, the third text set to be trained is a data set of billions of levels.

[0086] For example, each third text to be trained in the third text set does not have a category label, that is, there is no distinction between "pornographic text", "advertising text", "violent text", and "terrorist text". There is no need to determine which texts are illegal text data.

[0087] S402: Input the third set of texts to be trained into the enhanced representation model using knowledge integration for training, thereby obtaining an initial model. The number of texts in the third set of texts to be trained is greater than a preset number; the preset number is the sum of the number of texts in the first set of texts to be trained and the number of texts in the second set of texts to be trained. In one example, the initial model is a model obtained by applying the enhanced representation model using knowledge integration to other texts to be trained; the other texts to be trained are texts without category labels, are not in the first set of texts to be trained, and are not in the second set of texts to be trained.

[0088] For example, it can be seen that the texts in the third set of texts to be trained are “other texts to be trained.” The third set of texts to be trained is different from the first set of texts to be trained and the second set of texts to be trained. Figure 5 is a model training diagram according to the second embodiment of the present disclosure, such as Figure 5 As shown, the third set of texts to be trained is input into a preset enhanced representation model based on knowledge integration for processing, thereby training the enhanced representation model based on knowledge integration and obtaining an initial model. Since each third text in the third set of texts to be trained does not have a category label, the resulting initial model does not learn the ability to recognize category labels. However, the initial model does learn semantic information (i.e., semantic knowledge). Semantic information includes semantics, morphology, syntax, and dependency information; dependency information characterizes the dependencies between words. Semantics characterizes the meaning of a sentence. Morphology characterizes the composition, subcombination, and changes in word meaning. Syntax characterizes the components of a sentence and their order of arrangement. Dependency information characterizes the dependencies between words, such as dependencies between adjacent words, dependencies between intervening words (i.e., non-adjacent words), and dependencies between words of different types (e.g., verbs, nouns, adjectives, modal particles, etc.).

[0089] S403: Sending an acquisition instruction to the online system, wherein the acquisition instruction is used to instruct the acquisition of text without a category label; receiving the text without a category label sent by the online system to obtain a first set of texts to be trained. The number of texts in the first set of texts to be trained is greater than the number of texts in the second set of texts to be trained. Each first text to be trained in the first set of texts to be trained is a text without a category label.

[0090] In one example, the first text to be trained is a sentence obtained by segmenting the entire text.

[0091] For example, a first set of training texts is required. The first set of training texts includes multiple first training texts. The first training texts are unlabeled texts, i.e., the first training texts do not have category labels. The first training text set is a data set of tens of millions of words.

[0092] The electronic device may send an acquisition instruction to each online system, wherein the acquisition instruction is used to instruct to obtain text without category labels. Each online system sends the text without category labels to the electronic device, thereby obtaining the first set of text to be trained.

[0093] In one example, the entire text is stored in an online system, and the online system segments the entire text into sentences; then the online system sends each sentence without a category label to an electronic device; the electronic device combines a first set to be trained based on the sentences without a category label; each sentence without a category label is a first text to be trained.

[0094] The number of texts in the first text set to be trained is greater than the number of texts in the second text set to be trained.

[0095] In this step, real text data can be obtained from the online text review system. This is because the online text review system has already accumulated a large amount of text data; thus, a large amount of real text data can be obtained conveniently. Training based on real text data ensures that the model is adaptable to real online text review scenarios. This is because the text used for training is real text data, and the resulting text recognition model needs to recognize online text, and thus the resulting text recognition model can recognize the category labels of online text.

[0096] The data in the first text set to be trained can be called domain data.

[0097] S404: Acquire a second text set to be trained, wherein the second text set to be trained includes multiple text subsets, and the second text to be trained in the text subsets is a text with a preset category label.

[0098] In one example, the preset category labels for the second training text in the first text subset of the second training text set are pseudo labels, where a pseudo label represents a category label obtained by processing the same text in multiple online systems. The preset category labels for the second training text in the second text subset of the second training text set are category labels obtained by identification.

[0099] In one example, the second text to be trained is a sentence obtained by segmenting the entire text.

[0100] Exemplarily, the electronic device needs to obtain a second set of texts to be trained, the second set of texts to be trained including multiple text subsets, each of which includes multiple second texts to be trained. The first texts to be trained are texts with preset category labels, i.e., the second texts to be trained do not have category labels. It should be noted that the second texts to be trained in the second set of texts to be trained are texts with the same category label (i.e., the preset category label).

[0101] The second text set to be trained includes a first text subset (a first text subset) and a second text subset (a second text subset).

[0102] The first text subset includes multiple second texts to be trained; the preset category labels of the second texts to be trained in the first text subset are pseudo labels; wherein, the pseudo label refers to the category label obtained by the electronic device processing the same text in multiple online systems.

[0103] The preset category label of the second text to be trained in the second text subset is the category label obtained by identification. For example, the preset category label of the second text to be trained in the second text subset is the category label obtained by manual identification; or the preset category label of the second text to be trained in the second text subset is the category label obtained by other model identification.

[0104] By distinguishing the sources of each text subset in the second text set to be trained, the preset category label of the second text to be trained in the first text subset is a pseudo label; the preset category label of the second text to be trained in the second text subset is the category label obtained by identification; thereby, the first model can complete multiple trainings based on different text subsets, and each training learns information about category labels obtained in different ways, which facilitates the obtained text recognition model to perform online recognition.

[0105] In one example, an online system stores the entire text and segments it into sentences. The online system then sends each sentence with a category label to an electronic device. The electronic device then assembles a second training set based on the sentences with the preset category labels. Each sentence with the preset category label constitutes a second training text, enabling the first model to be effectively trained based on the sentences.

[0106] In one example, step S404 includes the following steps:

[0107] The first step of step S404 is to send a first acquisition instruction to the online system, wherein the first acquisition instruction is used to instruct to obtain text under multiple category labels; receive the text to be processed sent by the online system, wherein the text to be processed has at least one category label.

[0108] The second step of step S404 is to generate a first text subset in the second text set to be trained based on each text to be processed.

[0109] In one example, the second step of step S404 includes the following processes: for the same text to be processed, determining the category labels determined by different offline systems for the text to be processed; for the same text to be processed, determining the category label with the highest number as the pseudo label of the text to be processed; determining the text to be processed whose pseudo label is a preset category label as the second text to be trained in the first text subset.

[0110] The third step of step S404 is to obtain a second text subset in the second text set to be trained, wherein the preset category labels of the second texts to be trained in the second text subset in the second text set to be trained are category labels identified manually.

[0111] In one example, the number of texts in the first text subset in the second to-be-trained text set is greater than the number of texts in the second text subset in the second to-be-trained text set.

[0112] Exemplarily, when the electronic device acquires the first text subset in the second text set to be trained, the electronic device sends a first acquisition instruction to each online system, wherein the first acquisition instruction is used to instruct the acquisition of text under multiple category labels; thereby, each online system sends the stored real text to be processed to the electronic device, wherein the text to be processed has at least one category label.

[0113] The electronic device needs to obtain text with a preset category label. It then selects or processes each of the received texts to be processed, thereby obtaining a second set of training texts with the preset category label. The electronic device then combines the second set of training texts with the preset category label and the second set of training texts without the preset category label to form a first subset of texts within the second set of training texts. This first subset of texts within the second set of training texts is constructed based on a large amount of real text data from various online systems, providing a large amount of real data for subsequent training.

[0114] In one example, after receiving the texts to be processed sent by each online system, the electronic device can select from each text to be processed and select a second text to be trained with a preset category label; thus, the electronic device will form a first text subset in the second text to be trained set with the second text to be trained with the preset category label and the second text to be trained without the preset category label.

[0115] Alternatively, after the electronic device receives the text to be processed sent by each online system, each online system may store the same text to be processed. Then, for the same text to be processed, the electronic device determines the category label determined by different offline systems for the text to be processed; for the same text to be processed, the electronic device determines the category label with the highest number based on a voting mechanism, and the electronic device determines the category label with the highest number as the pseudo label of the text to be processed. Thus, the electronic device can obtain the text to be processed under each category label; for the preset category label, the electronic device determines the text to be processed with the pseudo label of the preset category label as the second text to be trained in the first text subset. Based on multiple offline systems, a voting mechanism is used to determine the pseudo label of the text to be processed for the same text to be processed. This is a remote supervision method; the text to be processed with the pseudo label of the preset category label is determined as the second text to be trained in the first text subset; this can reduce the error problem of the category label of the online system and reduce the noise of the text to be trained.

[0116] Furthermore, the first text subset in the second text set to be trained may be text data of millions of levels.

[0117] In this step, millions of text data are obtained; and subsequently, the first model can be trained based on millions of text data. Training based on a large amount of data can further reduce the noise of the text to be trained and the noise of the obtained text recognition model.

[0118] It should be noted that the first text subset in the second set of texts to be trained is text data for a preset category label. Each second text to be trained in the first text subset in the second set of texts to be trained has the same preset category label. Alternatively, some of the second texts to be trained in the first text subset in the second set of texts to be trained have the same preset category label, while the remaining second texts to be trained in the first text subset in the second set of texts to be trained do not have the preset category label (they may not have any category label, or may have other category labels).

[0119] When obtaining a second text subset from a second set of texts to be trained, the electronic device can obtain real text data from various online systems; the electronic device displays the obtained real text data and then obtains category labels based on manual annotation. The electronic device determines that the text data with the preset category labels manually annotated are the second texts to be trained in the second text subset. The second text subset is obtained based on manual annotation, and the category labels of each second text to be trained in the second text subset are correct and valid; this is beneficial to the training of the first model, and a text recognition model that accurately recognizes the category labels of texts can be obtained.

[0120] The second text subset in the second set of texts to be trained can be tens of thousands of text data items. The number of text items in the first text subset in the second set of texts to be trained is greater than the number of text items in the second text subset in the second set of texts to be trained. The first text subset in the second set of texts to be trained is determined by a voting mechanism, while the second text subset in the second set of texts to be trained is determined by manual annotation. The second text subset has fewer text items, which can reduce labor costs and time costs and improve training efficiency.

[0121] It should be noted that the second text subset in the second text set to be trained is text data for a preset category label. Each second text to be trained in the second text subset in the second text set to be trained has the same preset category label. Alternatively, some of the second texts to be trained in the second text subset in the second text set to be trained have the same preset category label, while the remaining second texts to be trained in the second text subset in the second text set to be trained do not have the preset category label (they may not have any category label, or may have other category labels).

[0122] S405 . Train the initial model based on the first text set to be trained to obtain a first model, wherein the initial model is a model obtained by using other text pairs without category labels to enhance the representation model through knowledge integration.

[0123] In one example, step S405 includes the following process: masking the first text to be trained in the first text set to be trained based on the initial model to obtain the masked first text to be trained; training the initial model based on the masked first text to be trained to obtain the first model; wherein the first model is used to identify the semantic information of the text.

[0124] In one example, the semantic information includes one or more of the following information: semantics, lexical information, syntax, and dependency information; wherein the dependency information represents the dependency relationship between words.

[0125] Exemplarily, the electronic device hot-starts the parameters of the initial model outputted in step S402, and then uses the initial model outputted in step S402 to complete step S405.

[0126] like Figure 5 As shown, the electronic device inputs the first text set to be trained into the initial model outputted in step S402, which is an ERNIE model that has been trained once. It can be seen that the initial model has the capability of a masked language model.

[0127] Thus, the electronic device masks the first text to be trained without category labels in the first text set to be trained based on the initial model, and then performs masking processing on the words in the first text to be trained, so as to predict the masked words (i.e., the words at the mask) during the training process to train the initial model and obtain the first model.

[0128] In this process, the semantic information of the text can be learned; thus, the first model obtained has the ability to recognize the semantic information of the text. Among them, the semantic information includes: semantics, morphology, syntax, and dependency information; the diversity of semantic information ensures that the first model has the ability to learn rich semantic information. Therefore, since the first model output by step S405 has learned a wealth of general semantic knowledge, the semantic learning ability of the first model can have better performance and processing capabilities in the subsequent training process. Among them, semantics represents the meaning of the sentence. Morphology represents the composition of words, their combination, changes in word meaning, etc. Syntax represents the various components of a sentence and their arrangement order. Dependency information represents the dependency between words, for example, the dependency between adjacent words, the dependency between intervening words (i.e., non-adjacent words), and the dependency between words of different types (types are verbs, nouns, adjectives, modal particles, etc.).

[0129] S406. When the i-th text subset is the first text subset, the first text subset is input into the first model corresponding to the first text subset to obtain a trained first model, wherein the trained first module is used to identify the semantic information of the text, and the trained first model is used to identify the category label of the text.

[0130] The first model corresponding to the first text subset is the first model outputted in step S405 .

[0131] Exemplarily, the electronic device hot-starts the parameters of the first model output in step S405, and then uses the first model output in step S405 to complete this step.

[0132] like Figure 5 As shown, the electronic device inputs the first text subset (the first text subset) into the first model output by step S405. The first model is an ERNIE model that has been trained twice. It can be seen that the initial model has the ability of a mask language model. The first model is trained based on the first text subset with a preset category label to obtain a trained first model. The first model learns semantic knowledge. Semantic knowledge includes: semantics, lexicality, syntax, and dependency information; wherein, dependency information represents the dependency relationship between words. Moreover, since the first text subset is configured with a preset category label, the first model can learn the knowledge of the preset category label, and then the trained first module obtained at this time is used to identify the semantic information of the text, and the trained first model is used to identify the category label of the text.

[0133] In one example, step S406 includes the following steps:

[0134] The first step of step S406 is to input the second text to be trained in the first text subset into the first model for masking to obtain the masked second text to be trained.

[0135] The second step of step S406 is to perform word prediction and category label prediction on the masked second text to be trained based on the first model to obtain the trained first model.

[0136] In one example, the second step of step S406 includes the following processes: predicting the masked words in the second text to be trained after masking based on the first model, and outputting a first loss function; and predicting the category label of the second text to be trained after masking based on the first model to obtain a second loss function; adjusting the parameters of the first model based on the first loss function and the second loss function to obtain the trained first model.

[0137] In one example, "based on the first loss function and the second loss function, adjusting the parameters of the first model to obtain the trained first model" includes: adding the first loss function and the second loss function to obtain the processed loss function; based on the processed loss function, adjusting the parameters of the first model to obtain the trained first model.

[0138] Alternatively, “based on the first loss function and the second loss function, adjusting the parameters of the first model to obtain the trained first model” includes: performing weighted processing on the first loss function and the second loss function to obtain the processed loss function; based on the processed loss function, adjusting the parameters of the first model to obtain the trained first model.

[0139] For example, Figure 6 is a training graph based on the first text subset according to the second embodiment of the present disclosure, such as Figure 6 As shown, a text with a preset category label is input into the first model. The text includes multiple words, namely word 1, word 2, word 4, word 5, word 6, and word 7. Based on the first model, the text is masked, and then word 3 and word 5 are masked. Feature processing is performed on the masked text to obtain vector features. The vectors include vector 0 corresponding to the category label, vector 1 corresponding to word 1, vector 2 corresponding to word 2, vector 3 corresponding to word 3, vector 4 corresponding to word 4, vector 5 corresponding to word 5, and vector 6 corresponding to word 6. The model is trained based on the vector features to obtain predicted category labels, predicted word 3, and predicted word 5. During the training process, a first loss function corresponding to the mask and a second loss function corresponding to the category label are output. The first and second loss functions are then added to obtain a processed loss function. The parameters of the first model are adjusted based on the processed loss function.

[0140] In one example, semantic information includes one or more of the following information: semantics, lexical information, syntax, and dependency information; wherein dependency information represents dependency relationships between words.

[0141] Exemplarily, the electronic device inputs the first text subset (the first text subset) into the first model output by step S405. Since the first model has the ability of a mask language model, the electronic device can mask the second text to be trained in the first text subset, and then mask the words in the second text to be trained in the first text subset to obtain the masked second text to be trained.

[0142] Then, the electronic device performs word prediction and category label prediction on the masked second text to be trained based on the first model output in step S405, thereby predicting the masked words (i.e., the words at the mask) and the category labels during the training process to train the first model and obtain the trained first model.

[0143] It can be seen that the first model at this time has learned semantic knowledge and knowledge of preset category labels, and the trained first module obtained at this time is used to recognize the semantic information of the text, and the trained first model is used to recognize the category labels of the text.

[0144] When the electronic device predicts words and category labels for the masked second text to be trained based on the first model, the electronic device can predict the masked words in the masked second text to be trained based on the first model, predict the masked words (i.e., the words at the mask) during the training process, and then output a first loss function; at the same time, during the training process, the electronic device predicts the category labels of the masked second text to be trained based on the first model, predicts the category labels during the training process, and compares the predicted category labels with the original category labels of the masked second text to be trained, and then outputs a second loss function.

[0145] The electronic device adjusts the parameters of the first model based on the first loss function corresponding to the mask and the second loss function corresponding to the class label, thereby obtaining a trained first model. Thus, during this stage of training, the first model learns more task-related knowledge (semantic knowledge and class label knowledge), which facilitates the first model's ability to fit and generalize text data involved in subsequent training.

[0146] In one example, the electronic device can add a first loss function corresponding to the mask and a second loss function corresponding to the category label to obtain a processed loss function. Based on the processed loss function, the electronic device adjusts the parameters of the first model to obtain a trained first model. Adding the loss functions based on two different directions can improve the first model's ability to simultaneously learn semantic knowledge and category label knowledge.

[0147] In another example, the electronic device can perform weighted processing on the first loss function corresponding to the mask and the second loss function corresponding to the category label. For example, the electronic device sets a first weight for the first loss function and a second weight for the second loss function, where the first weight is smaller than the second weight (because the semantics have been learned multiple times before, the weight of learning the category label can be increased). The electronic device performs weighted addition processing on the first loss function corresponding to the mask and the second loss function corresponding to the category label based on the first weight and the second weight; the electronic device obtains the processed loss function; the electronic device adjusts the parameters of the first model based on the processed loss function, thereby obtaining the trained first model. By performing weighted processing on the first loss function corresponding to the mask and the second loss function corresponding to the category label, the first weight of the first loss function can be set to be smaller than the second weight of the second loss function, which can increase the weight of learning the category label and improve the performance of the first model for learning the category label.

[0148] The data in the first text subset may be referred to as task data.

[0149] S407. When the i-th text subset is the second text subset, the second text subset is input into the first model corresponding to the second text subset to obtain a trained first model, wherein the trained first model is used to identify the category label of the text.

[0150] The first model corresponding to the second text subset is the first model outputted in step S406 .

[0151] The new first model obtained when the preset condition is determined to be met is a text recognition model for identifying the preset category label. The new first model obtained when the preset condition is met is the trained first model outputted in step S407.

[0152] Exemplarily, the electronic device hot-starts the parameters of the first model outputted in step S406, and then uses the first model outputted in step S406 to complete this step.

[0153] like Figure 5 As shown, the electronic device inputs the second text subset into the first model outputted in step S406, which is an ERNIE model that has been trained three times. It can be seen that the first model has the ability of a masked language model and has learned the ability to recognize category labels.

[0154] The electronic device inputs the second text subset (second text subset) into the first model outputted in step S406, and retrains the first model based on the preset category labels set for the second text subset, thereby obtaining a trained first model. This process no longer performs masking on the second text to be trained in the second text subset, but only learns the category labels, thereby reducing the amount of training and improving training efficiency. The resulting trained first model is used to identify the category labels of the texts.

[0155] The data in the second text subset may be referred to as downstream data.

[0156] In one example, step S407 includes the following steps:

[0157] The first step of step S407 is to input the second text to be trained in the second text subset into the first model for feature processing to obtain vector information; wherein the vector information represents the text information of the second text to be trained in the second text subset and the category label information of the second text to be trained in the second text subset.

[0158] The second step of step S407 is to train the first model according to the vector information to obtain a trained first model.

[0159] In one example, Figure 7 is a training graph based on the second text subset according to the second embodiment of the present disclosure, such as Figure 7 As shown, the first model corresponding to the second text subset has a fully connected layer. The second step of step S407 includes the following process: processing the vector information based on the fully connected layer to train the first model to obtain a trained first model; wherein the last layer of the trained first model is a fully connected layer.

[0160] Exemplarily, the electronic device inputs the second text to be trained in the second text subset into the first model outputted in step S406. The electronic device extracts and processes features of the second text to be trained in the second text subset based on the first model to obtain vector information. Since the second text to be trained in the second text subset has a preset category label (in this case, a manually annotated label), the vector information represents the text information of the second text to be trained in the second text subset and the category label information of the second text to be trained in the second text subset. The vector information is a 768-dimensional vector.

[0161] In this step, the second text to be trained in the second text subset is not subjected to masking processing.

[0162] Then, the first model of the electronic device processes the vector information to train the first model to obtain a trained first model.

[0163] In this step, the model is trained based on a second text subset of tens of thousands of words. Each second text to be trained in the second text subset has a preset category label manually annotated, and the data is accurate. Based on the first model obtained in the previous training processes, more accurate training is completed.

[0164] In one example, a fully connected layer is added to the last layer of the first model involved in this step. The electronic device then inputs the resulting vector information into the fully connected layer of the first model for training, thereby adjusting the parameters of the first model. This results in a trained first model. This allows the first model to accurately learn category labels based on the fully connected layer.

[0165] After the above processing is performed on each second text to be trained in the second text subset, the training process of this step is completed, and the obtained trained first model is a text recognition model; the text recognition model is used to identify preset category labels.

[0166] In this embodiment, by first obtaining a first model based on an initial model, the first model learns semantic information. The first model is then trained based on a first set of training texts, numbering tens of millions. Each first training text in the first set is a text without a category label, thereby further learning semantic information. The first model is then retrained based on a subset of first texts, numbering millions. The category labels for each second training text in the first subset are preset category labels. These category labels are labels derived from texts in multiple online systems, allowing for further learning of semantic information and category label knowledge. The first model is then retrained based on a second subset of ten thousand. The category labels for each second training text in the second subset are preset category labels, which are manually annotated labels, allowing for further learning of category label knowledge. Based on the large-scale data training at each stage above, the model can learn semantic information and category label knowledge related to the preset category labels, capture the language patterns of the semantic information under the preset category labels, and thus obtain a text recognition model that can more accurately recognize the category labels of texts. Furthermore, in the process of training the first model based on a first text subset of millions of words, the first model can learn more semantic knowledge and language patterns related to the recognition of preset category labels during the pre-training phase, thereby alleviating the problem of insufficient labeled data when subsequently training the first model based on a second text subset of tens of thousands of words. This allows the first model to focus on learning category labels based on the second text subset of tens of thousands of words. The text recognition model obtained in this embodiment can accurately identify the preset category labels of text.

[0167] Figure 8 is a schematic diagram according to the third embodiment of the present disclosure, as shown in Figure 8 As shown, the text recognition method provided in this embodiment includes:

[0168] S801: Obtain text to be recognized.

[0169] For example, the execution subject of this embodiment can be a terminal device, a server, an electronic device, a model training device or device for text recognition, or other devices or equipment that can execute the method of this embodiment. This embodiment is described with the execution subject being an electronic device.

[0170] The electronic device can obtain the text to be recognized from other online systems or from other databases.

[0171] S802: Input the text to be recognized into a text recognition model to obtain a category label of the text to be recognized; wherein the text recognition model is a model obtained by using the method provided in the above embodiment.

[0172] For example, the electronic device inputs the text to be recognized into the text recognition model obtained based on the above embodiment. Because the text recognition model has been trained multiple times with a large amount of data, the text recognition model can accurately identify the category label of the text. Since each text recognition model is used to identify each category label, the electronic device can input the text to be recognized into the text recognition model corresponding to each category label. The text to be recognized is then recognized based on the text recognition model corresponding to each category label to determine the category label of the text to be recognized.

[0173] For example, the text to be recognized is input into the text recognition model corresponding to category label 1, and then it is determined whether the text to be recognized is category label 1; then the text to be recognized is input into the text recognition model corresponding to category label 2, and then it is determined whether the text to be recognized is category label 2; and so on.

[0174] In this embodiment, the text to be recognized is input into the text recognition model corresponding to each category label, and then the text to be recognized is recognized based on the text recognition model corresponding to each category label to determine the category label of the text to be recognized. The category label of the text to be recognized can be accurately identified.

[0175] Figure 9 is a schematic diagram according to a fourth embodiment of the present disclosure, as shown in Figure 9 As shown, the text recognition method provided in this embodiment includes:

[0176] S901: Acquire an initial text, perform sentence segmentation on the initial text, and obtain multiple sentences; wherein the text to be recognized includes multiple sentences of the initial text.

[0177] For example, the execution subject of this embodiment can be a terminal device, a server, an electronic device, a model training device or device for text recognition, or other devices or equipment that can execute the method of this embodiment. This embodiment is described with the execution subject being an electronic device.

[0178] The electronic device may obtain the initial text from other online systems or from other databases.

[0179] The electronic device first segments the initial text, and then extracts each sentence in the initial text to obtain multiple sentences, so that each sentence can be accurately analyzed later.

[0180] S902: Input the sentences of the text to be recognized into the text recognition model to obtain the category of the sentences. The text recognition model is a model obtained by using the method provided in the above embodiment.

[0181] For example, the electronic device inputs each extracted sentence into the text recognition model corresponding to the preset category label obtained in the above embodiment, thereby identifying the category of each sentence. The category can be the preset category label described above. The category of each sentence in the text to be recognized is then obtained.

[0182] S903: Determine a category label of the text to be recognized based on the category of each sentence in the text to be recognized.

[0183] In one example, step S903 includes the following process: performing weighted calculation on the categories of each sentence in the text to be recognized to determine the category label of the text to be recognized, or determining the category with the highest number of sentence categories as the category label of the text to be recognized.

[0184] For example, the electronic device determines whether the category label of the text to be recognized is the preset category label based on the category of each sentence in the text to be recognized, and then recognizes each sentence of the text to be recognized to accurately identify the category label of the text to be recognized.

[0185] For the same text to be recognized, each sentence in the text to be recognized has a category; the electronic device determines the category with the highest number of categories as the category label for the text to be recognized. For example, the category of sentence 1 in the text to be recognized is category label A, the category of sentence 2 in the text to be recognized is category label A, the category of sentence 3 in the text to be recognized is category label A, the category of sentence 4 in the text to be recognized is category label B, and the category of sentence 5 in the text to be recognized is category label C; category label A can be determined to be the label with the highest number of categories, and then it is determined that the text to be recognized has category label A. Based on the voting mechanism, the category label of the text to be recognized is determined.

[0186] Alternatively, a weight can be set for each sentence in the text to be recognized. The weight is related to the length of the sentence, or the order of the sentences, or the type of sentence (for example, affirmative sentences, interrogative sentences; or, for example, film and television, science and technology). The weight of each sentence is multiplied by the category of each sentence to determine the coefficient of each category of each sentence; for the same category (category of sentence), the sum of the coefficients of the category is calculated. The category with the highest sum of coefficients is determined as the category label of the text to be recognized.

[0187] For example, the category of sentence 1 in the text to be recognized is category label A, and the weight of sentence 1's category is weight a; the category of sentence 2 in the text to be recognized is category label A, and the weight of sentence 2's category is weight b; the sum of the coefficients of category A is x1. The category of sentence 3 in the text to be recognized is category label B, and the weight of sentence 3's category is weight a; the category of sentence 4 in the text to be recognized is category label B, and the weight of sentence 4's category is weight b; the sum of the coefficients of category B is x2. If x2 is greater than x1, category B is determined to be the category label of the text to be recognized.

[0188] S904: If it is determined that the category label of the text to be recognized is an illegal text label, the text to be recognized is shielded, and a prompt message is sent to a preset terminal; wherein the prompt message indicates that the text to be recognized is an illegal text.

[0189] For example, according to the above steps, the text to be recognized is recognized based on the text recognition model corresponding to each preset category label, and an output is given as to whether the text to be recognized has the preset category label. This can then result in obtaining at least one category label for the text to be recognized, or determining that the text to be recognized does not have any category label.

[0190] The aforementioned category labels are all illegal text labels. If the electronic device determines that the text to be recognized has at least one category label, it will block the text to be recognized. The electronic device then sends a prompt message to the preset terminal that sent the text to be recognized, prompting the user that the text to be recognized is illegal. This effectively blocks the illegal text.

[0191] In this embodiment, based on the above embodiment, each extracted sentence is input into the text recognition model corresponding to the preset category label obtained in the above embodiment, and the category of each sentence is identified. According to the category of each sentence in the text to be identified, it is determined whether the category label of the text to be identified is the above preset category label. Then, each sentence of the text to be identified is identified to accurately identify the category label of the text to be identified. And if the electronic device determines that the text to be identified has at least one category label, the electronic device will block the text to be identified. Thereby effectively blocking the illegal text.

[0192] Figure 10 is a schematic diagram according to a fifth embodiment of the present disclosure, as shown in Figure 10 As shown, the model training device 1000 for text recognition provided in this embodiment includes:

[0193] The first acquiring unit 1001 is configured to acquire a first text set to be trained.

[0194] The second acquisition unit 1002 is used for a second set of texts to be trained, wherein each first text to be trained in the first set of texts to be trained is a text without a category label, the second set of texts to be trained includes multiple text subsets, and the second texts to be trained in the text subsets are texts with preset category labels.

[0195] The first training unit 1003 is used to train the initial model based on the first text set to be trained to obtain a first model.

[0196] The second training unit 1004 is used to repeat the following steps until a preset condition is met, wherein the initial value of i is 1 and i is a positive integer greater than or equal to 1: inputting the i-th text subset into the first model to obtain a trained first model; determining the trained first model as a new first model, and determining the value of i to be i+1; wherein the number of texts in different text subsets is different; and the data sources of different text subsets are different.

[0197] The new first model obtained when it is determined that the preset condition is met is a text recognition model for recognizing preset category labels.

[0198] The device of this embodiment can execute the technical solution in the above method. Its specific implementation process and technical principles are the same and will not be repeated here.

[0199] Figure 11 is a schematic diagram according to a sixth embodiment of the present disclosure, as shown in Figure 11 As shown, the model training device 1100 for text recognition provided in this embodiment includes:

[0200] The first acquiring unit 1101 is configured to acquire a first text set to be trained.

[0201] The second acquisition unit 1102 is used for a second set of texts to be trained, wherein each first text to be trained in the first set of texts to be trained is a text without a category label, the second set of texts to be trained includes multiple text subsets, and the second texts to be trained in the text subsets are texts with preset category labels.

[0202] The first training unit 1103 is used to train the initial model based on the first text set to be trained to obtain a first model.

[0203] The second training unit 1104 is used to repeat the following steps until a preset condition is met, wherein the initial value of i is 1 and i is a positive integer greater than or equal to 1: inputting the i-th text subset into the first model to obtain a trained first model; determining the trained first model as a new first model, and determining the value of i to be i+1; wherein the number of texts in different text subsets is different; and the data sources of different text subsets are different.

[0204] The new first model obtained when it is determined that the preset condition is met is a text recognition model for recognizing preset category labels.

[0205] In one example, the preset category label of the second text to be trained in the first text subset in the second text set to be trained is a pseudo label; wherein the pseudo label represents the category label obtained by processing the same text in multiple online systems.

[0206] The preset category label of the second text to be trained in the second text subset in the second text set to be trained is the category label obtained by identification.

[0207] The initial model is a model obtained by using other texts to be trained to enhance the representation model through knowledge integration; wherein the other texts to be trained are texts without category labels, and the other texts to be trained are texts that are not in the first set of texts to be trained, and the other texts to be trained are texts that are not in the second set of texts to be trained.

[0208] In one example, when the second acquiring unit 1102 is used to acquire the first text subset in the second to-be-trained text set, the following steps are performed:

[0209] The first sending module 11021 is used to send a first acquisition instruction to the online system, wherein the first acquisition instruction is used to instruct to acquire texts under multiple category labels.

[0210] The second receiving module 11022 is configured to receive a text to be processed sent by an online system, wherein the text to be processed has at least one category label.

[0211] The generating module 11023 is configured to generate a first text subset in the second text set to be trained based on each text to be processed.

[0212] In one example, the generation module 11023 includes:

[0213] The first determination submodule 110231 is used to determine, for the same text to be processed, the category labels determined by different offline systems for the text to be processed.

[0214] The second determination submodule 110232 is used to determine the category label with the highest number for the same text to be processed, which is the pseudo label of the text to be processed.

[0215] The third determining submodule 110233 is configured to determine that the text to be processed, whose pseudo label is a preset category label, is the second text to be trained in the first text subset.

[0216] In one example, the preset category labels of the second texts to be trained in the second text subset in the second text set to be trained are category labels identified manually.

[0217] In one example, the number of texts in the first text subset in the second to-be-trained text set is greater than the number of texts in the second text subset in the second to-be-trained text set.

[0218] In one example, when the i-th text subset is the first text subset, the second training unit 1104, when inputting the i-th text subset into the first model to obtain the trained first model, is specifically configured to:

[0219] The first text subset is input into the first model to obtain a trained first model, wherein the trained first module is used to identify semantic information of the text, and the trained first model is used to identify category labels of the text.

[0220] In one example, the second training unit 1104 includes:

[0221] The first input module 11041 is used to input the second text to be trained in the first text subset into the first model for masking to obtain the masked second text to be trained.

[0222] The first training module 11042 is used to perform word prediction and category label prediction on the masked second text to be trained based on the first model to obtain a trained first model.

[0223] In one example, the first training module 11042 includes:

[0224] The output submodule 110421 is used to predict the masked words in the second to-be-trained text after masking based on the first model and output a first loss function; and predict the category label of the second to-be-trained text after masking based on the first model to obtain a second loss function.

[0225] The adjustment submodule 110422 is used to adjust the parameters of the first model based on the first loss function and the second loss function to obtain the trained first model.

[0226] In one example, the adjustment submodule 110422 is specifically configured to:

[0227] The first loss function and the second loss function are added to obtain a processed loss function; based on the processed loss function, the parameters of the first model are adjusted to obtain a trained first model.

[0228] In one example, the adjustment submodule 110422 is specifically configured to:

[0229] The first loss function and the second loss function are weighted to obtain a processed loss function; based on the processed loss function, the parameters of the first model are adjusted to obtain a trained first model.

[0230] In one example, semantic information includes one or more of the following information: semantics, lexical information, syntax, and dependency information; wherein dependency information represents dependency relationships between words.

[0231] In one example, when the i-th text subset is the second text subset, the second training unit 1104, when inputting the i-th text subset into the first model to obtain the trained first model, is specifically configured to:

[0232] The second text subset is input into the first model to obtain a trained first model, wherein the trained first model is used to identify category labels of the text.

[0233] In one example, the second training unit 1104 includes:

[0234] The second input module 11043 is used to input the second text to be trained in the second text subset into the first model for feature processing to obtain vector information; wherein the vector information represents the text information of the second text to be trained in the second text subset and the category label information of the second text to be trained in the second text subset.

[0235] The second training module 11044 is used to train the first model according to the vector information to obtain a trained first model.

[0236] In one example, the first model corresponding to the second text subset has a fully connected layer; the second training module 11044 is specifically configured to:

[0237] The vector information is processed based on the fully connected layer to train the first model to obtain a trained first model; wherein the last layer of the trained first model is a fully connected layer.

[0238] In one example, the first obtaining unit 1101 includes:

[0239] The second sending module 11011 is used to send an acquisition instruction to the online system, wherein the acquisition instruction is used to instruct to acquire text without category tags.

[0240] The second receiving module 11012 is used to receive text without category labels sent by the online system to obtain a first set of texts to be trained; wherein the number of texts in the first set of texts to be trained is greater than the number of texts in the second set of texts to be trained.

[0241] In one example, the first training unit 1103 includes:

[0242] The masking module 11031 is configured to perform masking processing on the first text to be trained in the first text to be trained set based on the initial model to obtain the masked first text to be trained.

[0243] The third training module 11032 is used to train the initial model based on the masked first text to be trained to obtain a first model; wherein the first model is used to recognize semantic information of the text.

[0244] In one example, the apparatus provided in this embodiment further includes:

[0245] The third acquiring unit 1105 is configured to acquire a third set of texts to be trained, wherein the third texts to be trained in the third set of texts to be trained are texts without category labels.

[0246] The third training unit 1106 is used to input the third text set to be trained into the enhanced representation model through knowledge integration for training processing to obtain an initial model.

[0247] The number of texts in the third text set to be trained is greater than a preset number; the preset number is the sum of the number of texts in the first text set to be trained and the number of texts in the second text set to be trained.

[0248] In one example, the first text to be trained is a sentence obtained by segmenting the entire text, and the second text to be trained is a sentence obtained by segmenting the entire text.

[0249] The device of this embodiment can execute the technical solution in the above method. Its specific implementation process and technical principles are the same and will not be repeated here.

[0250] Figure 12 is a schematic diagram according to the seventh embodiment of the present disclosure, as shown in Figure 12 As shown, the text recognition device 1200 provided in this embodiment includes:

[0251] The acquiring unit 1201 is configured to acquire the text to be recognized.

[0252] The recognition unit 1202 is used to input the text to be recognized into the text recognition model to obtain the category label of the text to be recognized; wherein the text recognition model is a model obtained by using the above-mentioned model training device for text recognition.

[0253] The device of this embodiment can execute the technical solution in the above method. Its specific implementation process and technical principles are the same and will not be repeated here.

[0254] Figure 13 is a schematic diagram according to an eighth embodiment of the present disclosure, as shown in Figure 13 As shown, the text recognition device 1300 provided in this embodiment includes:

[0255] The acquiring unit 1301 is configured to acquire the text to be recognized.

[0256] The recognition unit 1302 is used to input the text to be recognized into the text recognition model to obtain the category label of the text to be recognized; wherein the text recognition model is a model obtained by using the above-mentioned model training device for text recognition.

[0257] In one example, the text to be recognized includes multiple sentences; the recognition unit 1302 includes:

[0258] The input module 13021 is used to input sentences of the text to be recognized into the text recognition model to obtain the category of the sentence.

[0259] The recognition module 13022 is used to determine the category label of the text to be recognized based on the category of each sentence in the text to be recognized.

[0260] In one example, the recognition module 13022 is specifically configured to: perform weighted calculation on the categories of each sentence of the text to be recognized to determine the category label of the text to be recognized; or determine the category with the highest number of sentence categories as the category label of the text to be recognized.

[0261] In one example, the acquiring unit 1301 is specifically configured to: acquire an initial text, perform sentence segmentation on the initial text, and obtain multiple sentences; wherein the text to be recognized includes multiple sentences of the initial text.

[0262] In one example, the apparatus provided in this embodiment further includes:

[0263] The shielding unit 1303 is configured to shield the text to be recognized if it is determined that the category label of the text to be recognized is an illegal text label, and send a prompt message to a preset terminal; wherein the prompt message indicates that the text to be recognized is an illegal text.

[0264] The device of this embodiment can execute the technical solution in the above method. Its specific implementation process and technical principles are the same and will not be repeated here.

[0265] Figure 14 is a schematic diagram according to the ninth embodiment of the present disclosure, as shown in Figure 14 As shown, the electronic device 1400 in this embodiment may include: a processor 1401 and a memory 1402.

[0266] Memory 1402 is used to store programs. Memory 1402 may include volatile memory, such as random-access memory (RAM), such as static random-access memory (SRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), etc. Memory may also include non-volatile memory, such as flash memory. Memory 1402 is used to store computer programs (such as applications and functional modules that implement the above-mentioned methods), computer instructions, etc. These computer programs and computer instructions may be partitioned and stored in one or more memories 1402. Furthermore, these computer programs, computer instructions, data, etc. may be called by processor 1401.

[0267] The aforementioned computer programs, computer instructions, etc. may be partitioned and stored in one or more memories 1402 , and the aforementioned computer programs, computer instructions, etc. may be called by the processor 1401 .

[0268] The processor 1401 is configured to execute the computer program stored in the memory 1402 to implement the various steps in the method involved in the above embodiment.

[0269] For details, please refer to the relevant description in the previous method embodiment.

[0270] The processor 1401 and the memory 1402 may be independent structures or integrated structures. When the processor 1401 and the memory 1402 are independent structures, the memory 1402 and the processor 1401 may be coupled via a bus 1403 .

[0271] The electronic device of this embodiment can execute the technical solution in the above method. Its specific implementation process and technical principles are the same and will not be repeated here.

[0272] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0273] According to an embodiment of the present disclosure, the present disclosure further provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the solution provided by any of the above embodiments.

[0274] According to an embodiment of the present disclosure, the present disclosure also provides a computer program product, which includes: a computer program, the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the solution provided by any of the above embodiments.

[0275] Figure 15 A schematic block diagram of an example electronic device 1500 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0276] like Figure 15 As shown, device 1500 includes a computing unit 1501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1502 or a computer program loaded from a storage unit 1508 into a random access memory (RAM) 1503. Various programs and data required for the operation of device 1500 can also be stored in RAM 1503. Computing unit 1501, ROM 1502, and RAM 1503 are connected to each other via a bus 1504. An input / output (I / O) interface 1505 is also connected to bus 1504.

[0277] Various components in device 1500 are connected to I / O interface 1505, including an input unit 1506, such as a keyboard and mouse; an output unit 1507, such as various types of displays and speakers; a storage unit 1508, such as a magnetic disk and optical disk; and a communication unit 1509, such as a network card, a modem, a wireless communication transceiver, etc. Communication unit 1509 allows device 1500 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0278] The computing unit 1501 can be various general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 1501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 1501 performs the various methods and processes described above, such as the model training method or text recognition method applied to text recognition. For example, in some embodiments, the model training method or text recognition method applied to text recognition can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as the storage unit 1508. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 1500 via the ROM 1502 and / or the communication unit 1509. When the computer program is loaded into the RAM 1503 and executed by the computing unit 1501, one or more steps of the model training method applied to text recognition or one or more steps of the text recognition method described above can be performed. Alternatively, in other embodiments, the computing unit 1501 may be configured in any other appropriate manner (eg, by means of firmware) to execute a model training method or a text recognition method applied to text recognition.

[0279] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0280] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0281] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0282] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0283] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0284] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. This client-server relationship is established by computer programs running on the respective computers, establishing a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host, a host product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosts and VPS services ("Virtual Private Servers" or simply "VPS"). The server may also be a server in a distributed system or a server integrated with blockchain.

[0285] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0286] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A model training method for text recognition, comprising: Obtaining a first set of texts to be trained and a second set of texts to be trained, wherein each first text to be trained in the first set of texts to be trained is a text without a category label, and the second set of texts to be trained includes a plurality of text subsets, and each second text to be trained in the text subsets is a text with a preset category label; training an initial model based on the first set of texts to be trained to obtain a first model; Repeat the following steps until a preset condition is met, wherein the initial value of i is 1, and i is a positive integer greater than or equal to 1: input the i-th text subset into the first model to obtain a trained first model; determine the trained first model as a new first model, and determine the value of i to be i+1; wherein the number of texts in different text subsets is different; and the data sources of different text subsets are different; wherein the preset category label of the second text to be trained in the first text subset in the second text set to be trained is a pseudo label; and the preset category label of the second text to be trained in the second text subset in the second text set to be trained is a category label obtained by identification; The first text subset and the second text subset are respectively used to train the first model at different stages; the first text subset is used to enable the trained first model to recognize semantic information of the text and recognize the category label of the text; the second text subset is used to enable the trained first model to recognize the category label of the text; The new first model obtained when it is determined that the preset condition is met is a text recognition model for identifying the preset category label.

2. The method according to claim 1, wherein The pseudo-label representation is a category label obtained by processing the same text on multiple online systems; The initial model is a model obtained by using other texts to be trained to enhance the representation model through knowledge integration; wherein, the other texts to be trained are texts without category labels, and the other texts to be trained are texts that are not in the first set of texts to be trained, and the other texts to be trained are texts that are not in the second set of texts to be trained.

3. The method according to claim 2, wherein: Obtaining a first text subset in a second text set to be trained, including: Sending a first acquisition instruction to the online system, wherein the first acquisition instruction is used to instruct to acquire texts under multiple category labels; receiving a text to be processed sent by the online system, wherein the text to be processed has at least one category label; A first text subset in the second text set to be trained is generated based on each of the texts to be processed.

4. The method according to claim 3, wherein: Generating a first text subset in the second text set to be trained based on each of the to-be-processed texts includes: For the same text to be processed, determine the category labels determined by different offline systems for the text to be processed; For the same text to be processed, determine the category label with the highest number as the pseudo label of the text to be processed; The text to be processed whose pseudo label is determined to be the preset category label is the second text to be trained in the first text subset.

5. The method according to any one of claims 2 to 4, wherein: The preset category labels of the second texts to be trained in the second text subset in the second text set to be trained are category labels identified manually.

6. The method according to any one of claims 1 to 4, wherein: The number of texts in the first text subset in the second text set to be trained is greater than the number of texts in the second text subset in the second text set to be trained.

7. The method according to any one of claims 1 to 4, wherein: When the i-th text subset is the first text subset, the i-th text subset is input into the first model to obtain the trained first model, including: The first text subset is input into a first model to obtain a trained first model, wherein the trained first module is used to identify semantic information of the text, and the trained first model is used to identify category labels of the text.

8. The method according to claim 7, wherein: Inputting the first text subset into a first model to obtain a trained first model includes: Inputting the second text to be trained in the first text subset into the first model for masking to obtain the masked second text to be trained; Based on the first model, word prediction and category label prediction are performed on the masked second text to be trained to obtain a trained first model.

9. The method according to claim 8, wherein Performing word prediction and category label prediction on the masked second training text based on the first model to obtain a trained first model, including: Predicting the masked words in the second to-be-trained text after masking based on the first model and outputting a first loss function; and predicting the category label of the second to-be-trained text after masking based on the first model and obtaining a second loss function; Based on the first loss function and the second loss function, the parameters of the first model are adjusted to obtain a trained first model.

10. The method according to claim 9, wherein: Adjusting parameters of the first model based on the first loss function and the second loss function to obtain a trained first model includes: Adding the first loss function and the second loss function to obtain a processed loss function; Based on the processed loss function, the parameters of the first model are adjusted to obtain a trained first model.

11. The method according to claim 10, wherein: Adjusting parameters of the first model based on the first loss function and the second loss function to obtain a trained first model includes: Performing weighted processing on the first loss function and the second loss function to obtain a processed loss function; Based on the processed loss function, the parameters of the first model are adjusted to obtain a trained first model.

12. The method according to any one of claims 1 to 4 or 8 to 11, wherein: When the i-th text subset is the second text subset, the i-th text subset is input into the first model to obtain the trained first model, including: The second text subset is input into the first model to obtain a trained first model, wherein the trained first model is used to identify the category label of the text.

13. The method according to claim 12, wherein: Inputting the second text subset into the first model to obtain a trained first model includes: Inputting the second text to be trained in the second text subset into the first model for feature processing to obtain vector information; wherein the vector information represents text information of the second text to be trained in the second text subset and information about category labels of the second text to be trained in the second text subset; The first model is trained according to the vector information to obtain a trained first model.

14. The method according to claim 13, wherein: The first model corresponding to the second subset of texts has a fully connected layer; Training the first model according to the vector information to obtain a trained first model includes: The vector information is processed based on the fully connected layer to train the first model to obtain a trained first model; wherein the last layer of the trained first model is a fully connected layer.

15. The method according to any one of claims 1-4, 8-11 or 13-14, wherein The step of obtaining a first text set to be trained includes: Sending an acquisition instruction to an online system, wherein the acquisition instruction is used to instruct to acquire text without a category label; Receive the text without category labels sent by the online system to obtain the first set of texts to be trained; wherein the number of texts in the first set of texts to be trained is greater than the number of texts in the second set of texts to be trained.

16. A text recognition method comprising: Obtaining a text to be recognized, inputting the text to be recognized into a text recognition model, and obtaining a category label for the text to be recognized; Wherein, the text recognition model is a model obtained by using the method described in any one of claims 1-15.

17. The method according to claim 16, wherein: The text to be recognized includes a plurality of sentences; the text to be recognized is input into a text recognition model to obtain a category label of the text to be recognized, including: Inputting the sentence of the text to be recognized into the text recognition model to obtain the category of the sentence; According to the category of each sentence in the text to be recognized, a category label of the text to be recognized is determined.

18. The method according to claim 17, wherein Determining a category label of the text to be recognized according to the category of each sentence in the text to be recognized includes: Performing weighted calculation on the category of each sentence in the text to be recognized to determine the category label of the text to be recognized; Alternatively, the category with the highest number of sentence categories is determined as the category label of the text to be recognized.

19. A model training device for text recognition, comprising: A first acquisition unit, configured to acquire a first text set to be trained; A second acquisition unit is configured to obtain a second set of texts to be trained, wherein each first text to be trained in the first set of texts to be trained is a text without a category label, and the second set of texts to be trained includes a plurality of text subsets, and each second text to be trained in the text subsets is a text with a preset category label; A first training unit is configured to train an initial model based on the first set of texts to be trained to obtain a first model; The second training unit is used to repeat the following steps until a preset condition is met, wherein the initial value of i is 1 and i is a positive integer greater than or equal to 1: inputting the i-th text subset into the first model to obtain a trained first model; determining the trained first model as a new first model, and determining the value of i to be i+1; wherein the number of texts in different text subsets is different; and the data sources of different text subsets are different; wherein the preset category label of the second text to be trained in the first text subset in the second text set to be trained is a pseudo label; and the preset category label of the second text to be trained in the second text subset in the second text set to be trained is a category label obtained by identification; The first text subset and the second text subset are respectively used to train the first model at different stages; the first text subset is used to enable the trained first model to recognize semantic information of the text and recognize the category label of the text; the second text subset is used to enable the trained first model to recognize the category label of the text; The new first model obtained when it is determined that the preset condition is met is a text recognition model for identifying the preset category label.

20. The device according to claim 19, wherein The pseudo-label representation is a category label obtained by processing the same text on multiple online systems; The initial model is a model obtained by using other texts to be trained to enhance the representation model through knowledge integration; wherein, the other texts to be trained are texts without category labels, and the other texts to be trained are texts that are not in the first set of texts to be trained, and the other texts to be trained are texts that are not in the second set of texts to be trained.

21. The device according to claim 20, wherein When the second acquisition unit is used to acquire the first text subset in the second to-be-trained text set, the second acquisition unit includes: A first sending module is configured to send a first acquisition instruction to the online system, wherein the first acquisition instruction is used to instruct to acquire texts under multiple category labels; a second receiving module, configured to receive a text to be processed sent by the online system, wherein the text to be processed has at least one category label; A generating module is used to generate a first text subset in the second text set to be trained based on each of the texts to be processed.

22. The device according to claim 21, wherein The generation module includes: The first determination submodule is used to determine, for the same text to be processed, the category labels determined by different offline systems for the text to be processed; The second determination submodule is used to determine the category label with the highest number for the same text to be processed, which is the pseudo label of the text to be processed; The third determining submodule is configured to determine that the text to be processed, whose pseudo label is the preset category label, is the second text to be trained in the first text subset.

23. The device according to any one of claims 20 to 22, wherein: The preset category labels of the second texts to be trained in the second text subset in the second text set to be trained are category labels identified manually.

24. The device according to any one of claims 19 to 22, wherein: The number of texts in the first text subset in the second text set to be trained is greater than the number of texts in the second text subset in the second text set to be trained.

25. The device according to any one of claims 19 to 22, wherein: When the i-th text subset is the first text subset, the second training unit, when used to input the i-th text subset into the first model to obtain the trained first model, is specifically used to: The first text subset is input into a first model to obtain a trained first model, wherein the trained first module is used to identify semantic information of the text, and the trained first model is used to identify category labels of the text.

26. The device according to claim 25, wherein The second training unit includes: A first input module is configured to input the second text to be trained in the first text subset into the first model for masking to obtain the masked second text to be trained; The first training module is used to perform word prediction and category label prediction on the masked second text to be trained based on the first model to obtain a trained first model.

27. The device according to claim 26, wherein The first training module includes: an output submodule, configured to predict the masked words in the second to-be-trained text after masking based on the first model and output a first loss function; and to predict the category label of the second to-be-trained text after masking based on the first model and obtain a second loss function; The adjustment submodule is used to adjust the parameters of the first model based on the first loss function and the second loss function to obtain the trained first model.

28. The apparatus according to claim 27, wherein The regulating submodule is specifically used to: Adding the first loss function and the second loss function to obtain a processed loss function; Based on the processed loss function, the parameters of the first model are adjusted to obtain a trained first model.

29. The apparatus according to claim 28, wherein The regulating submodule is specifically used to: Performing weighted processing on the first loss function and the second loss function to obtain a processed loss function; Based on the processed loss function, the parameters of the first model are adjusted to obtain a trained first model.

30. The device according to any one of claims 19 to 22 or 26 to 29, wherein: When the i-th text subset is the second text subset, the second training unit, when used to input the i-th text subset into the first model to obtain the trained first model, is specifically used to: The second text subset is input into the first model to obtain a trained first model, wherein the trained first model is used to identify the category label of the text.

31. The device according to claim 30, wherein The second training unit includes: a second input module, configured to input the second text to be trained in the second text subset into the first model for feature processing to obtain vector information; wherein the vector information represents text information of the second text to be trained in the second text subset and information about category labels of the second text to be trained in the second text subset; The second training module is used to train the first model according to the vector information to obtain a trained first model.

32. The apparatus according to claim 31, wherein The first model corresponding to the second text subset has a fully connected layer; the second training module is specifically used to: The vector information is processed based on the fully connected layer to train the first model to obtain a trained first model; wherein the last layer of the trained first model is a fully connected layer.

33. The device according to any one of claims 19-22, 26-29 or 31-32, wherein: The first acquiring unit includes: A second sending module is used to send an acquisition instruction to the online system, wherein the acquisition instruction is used to instruct to acquire text without category labels; The second receiving module is used to receive the text without category labels sent by the online system to obtain the first text set to be trained; wherein the number of texts in the first text set to be trained is greater than the number of texts in the second text set to be trained.

34. A text recognition device comprising: An acquisition unit, used for acquiring the text to be recognized; A recognition unit, configured to input the text to be recognized into a text recognition model to obtain a category label of the text to be recognized; Wherein, the text recognition model is a model obtained by using the device described in any one of claims 19-33.

35. The apparatus of claim 34, wherein: The text to be recognized includes multiple sentences; The identification unit includes: An input module, configured to input the sentences of the text to be recognized into the text recognition model to obtain the categories of the sentences; The recognition module is used to determine the category label of the text to be recognized according to the category of each sentence in the text to be recognized.

36. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 18.

37. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-18.

38. A computer program product comprising a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 18.

Images