Text classification model training method and device, equipment and medium

Abstract

The invention provides a text classification model training method and device, equipment and a medium, relates to the technical field of text processing, and aims to solve the problem of low accuracy of a text classification model caused by low quality of a training text for training the text classification model in a general technology. The text classification model training method comprises the following steps: acquiring an initial training text for training a text classification model; and performing coding processing on the initial training text to obtain coding features of the initial training text. And carrying out merging processing on the redundant field features to obtain a target training text. And training the text classification model based on the target training text. Wherein the coding feature comprises a redundant field feature. The redundant field features are used for representing redundant fields in the initial training text.

Description

Technical Field

[0001] The present application relates to the field of text processing technology, and in particular to a text classification model training method, device, equipment and medium. Background Art

[0002] With the rapid development of information technology, all kinds of data have grown explosively, providing rich resources for innovation and development in various fields. At the same time, model technology is also constantly developing, especially in the field of text processing, the importance of text classification models is becoming increasingly prominent. The efficient training of text classification models depends on a large amount of high-quality text.

[0003] When training text classification models, general technology collects text from multiple sources and processes and annotates it with the help of deep learning algorithms. However, the texts collected by general technology usually have redundant fields, which interferes with deep learning's understanding of the meaning of text data. In this way, it is difficult for deep learning algorithms to accurately extract key information from the text, resulting in frequent omissions or annotation errors. This makes the training text used to train text classification models lack accuracy and low quality.

[0004] Therefore, the quality of the training text used to train the text classification model in the general technology is low, resulting in low accuracy of the text classification model. Summary of the invention

[0005] The embodiments of the present disclosure provide a text classification model training method, apparatus, device and medium, aiming to solve the problem in general technology that the quality of training text used to train text classification models is low, thereby reducing the accuracy of the text classification models.

[0006] In order to achieve the above objectives, this application adopts the following technical solutions:

[0007] In a first aspect, a text classification model training method is provided, comprising: obtaining an initial training text for training a text classification model; encoding the initial training text to obtain encoding features of the initial training text; the encoding features include redundant field features; the redundant field features are used to represent redundant fields in the initial training text; merging the redundant field features to obtain a target training text; and training a text classification model based on the target training text.

[0008] Optionally, the encoding features also include field position features of redundant fields; the field position features are used to indicate the field positions of redundant fields in the initial training text; the redundant field features are merged to obtain the target training text, including: determining continuous redundant fields according to the field position features; continuous redundant fields are redundant fields with continuous field positions in the initial training text; the redundant field features of continuous redundant fields are merged to obtain the target training text.

[0009] Optionally, the encoding features also include sentence features of the sentence to which the redundant field belongs; the redundant field features of consecutive redundant fields are merged to obtain the target training text, including: determining consecutive redundant fields in the same sentence according to the sentence features; and merging the redundant field features of consecutive redundant fields in the same sentence to obtain the target training text.

[0010] Optionally, redundant field features of consecutive redundant fields in the same sentence are merged to obtain target training text, including: determining the redundant field features to be merged based on the redundant field features of consecutive redundant fields in the same sentence and the number of consecutive redundant fields in the same sentence; and performing feature fusion on the merged redundant field features, field position features and sentence features to obtain the target training text.

[0011] Optionally, the feature content of redundant field features is the same.

[0012] Optionally, before encoding the initial training text to obtain the encoding features of the initial training text, it also includes: determining the application scenario of the training text classification model; and determining the redundant fields in the initial training text according to the redundant field library corresponding to the application scenario of the training text classification model.

[0013] In a second aspect, a text classification model training device is provided, which includes: a communication unit and a processing unit; the communication unit is used to obtain an initial training text for training a text classification model; the processing unit is used to encode the initial training text to obtain encoding features of the initial training text; the encoding features include redundant field features; the redundant field features are used to represent redundant fields in the initial training text; the processing unit is also used to merge the redundant field features to obtain a target training text; the processing unit is also used to train a text classification model based on the target training text.

[0014] Optionally, the encoding features also include field position features of redundant fields; the field position features are used to indicate the field positions of redundant fields in the initial training text; the processing unit is specifically used to: determine continuous redundant fields based on the field position features; continuous redundant fields are redundant fields with continuous field positions in the initial training text; and merge the redundant field features of continuous redundant fields to obtain the target training text.

[0015] Optionally, the encoding features also include sentence features of the sentence to which the redundant field belongs; the processing unit is specifically used to: determine consecutive redundant fields in the same sentence based on the sentence features; and merge the redundant field features of consecutive redundant fields in the same sentence to obtain the target training text.

[0016] Optionally, the processing unit is specifically used to: determine the redundant field features for merged processing based on the redundant field features of consecutive redundant fields in the same sentence and the number of consecutive redundant fields in the same sentence; and perform feature fusion on the merged redundant field features, field position features and sentence features to obtain the target training text.

[0017] Optionally, the feature content of redundant field features is the same.

[0018] Optionally, before encoding the initial training text to obtain the encoding features of the initial training text, the processing unit is also used to determine the application scenario of the training text classification model; the processing unit is also used to determine the redundant fields in the initial training text based on the redundant field library corresponding to the application scenario of the training text classification model.

[0019] In a third aspect, a text classification model training device is provided, comprising a memory and a processor; the memory is used to store computer execution instructions, and the processor is connected to the memory through a bus; when the text classification model training device is running, the processor executes the computer execution instructions stored in the memory, so that the text classification model training device performs the text classification model training method of the first aspect.

[0020] The text classification model training device can be an electronic device, or a part of an electronic device, such as a chip system in an electronic device. The chip system is used to support the electronic device to implement the functions involved in the first aspect and any possible implementation thereof, for example, to obtain and determine the data and / or information involved in the above-mentioned text classification model training method. The chip system includes a chip, and may also include other discrete devices or circuit structures.

[0021] In a fourth aspect, a computer-readable storage medium is provided, the computer-readable storage medium comprising computer execution instructions, and when the computer execution instructions are executed on a computer, the computer executes the text classification model training method described in the first aspect.

[0022] In a fifth aspect, a computer program product is also provided, which includes a computer program or instructions. When the computer instructions are run on a text classification model training device, the text classification model training device executes the text classification model training method described in the first aspect above.

[0023] It should be noted that the above-mentioned computer instructions may be stored in whole or in part on a computer-readable storage medium. The computer-readable storage medium may be packaged together with the processor of the text classification model training device, or may be packaged separately from the processor of the text classification model training device, and this embodiment of the application does not limit this.

[0024] The description of the second, third, fourth and fifth aspects of the present application can refer to the detailed description of the first aspect.

[0025] In the embodiments of the present application, the name of the above-mentioned text classification model training device does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear with other names. For example, the receiving unit may also be called a receiving module, a receiver, etc. As long as the functions of each device or functional module are similar to those of the present application, they fall within the scope of the claims of the present application and their equivalent technologies.

[0026] The technical solution provided by this application brings at least the following beneficial effects:

[0027] Based on any of the above aspects, an embodiment of the present application provides a text classification model training method. First, an initial training text for training a text classification model is obtained. Then, the initial training text is encoded to obtain encoding features of the initial training text. The encoding features include redundant field features, which are used to represent redundant fields in the initial training text. Then, the redundant field features are merged to obtain a target training text. Then, a text classification model is trained based on the target training text.

[0028] As can be seen from the above, the present application can encode the initial training text used to train the text classification model to obtain the encoding features of the initial training text. And the redundant field features corresponding to the redundant fields are merged to obtain the target training text. The target training text obtained by the present application with redundant fields removed can improve the quality of the training text. Moreover, the present application trains a text classification model based on the target training text, which can reduce the computational cost of the text classification model and improve the accuracy of the text classification model.

[0029] The beneficial effects of the first, second, third, fourth and fifth aspects of the present application can all be referred to the analysis of the above beneficial effects, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A structural diagram of a text classification model training system provided in an embodiment of the present application;

[0031] Figure 2 A schematic diagram of a hardware structure of a communication device provided in an embodiment of the present application;

[0032] Figure 3 A flowchart of a text classification model training method provided in an embodiment of the present application;

[0033] Figure 4 A schematic diagram of an encoding result provided in an embodiment of the present application;

[0034] Figure 5 A flowchart of another text classification model training method provided in an embodiment of the present application;

[0035] Figure 6 A schematic diagram of another encoding result provided in an embodiment of the present application;

[0036] Figure 7 A flowchart of another text classification model training method provided in an embodiment of the present application;

[0037] Figure 8 A flowchart of another text classification model training method provided in an embodiment of the present application;

[0038] Fig. 9 A flowchart of another text classification model training method provided in an embodiment of the present application;

[0039] Fig.10 A schematic diagram of another encoding result provided in an embodiment of the present application;

[0040] Fig.11 A structural schematic diagram of a text classification model training device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0042] It should be noted that, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0043] In order to facilitate a clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and order of execution.

[0044] Before introducing in detail the text classification model training method provided by this application, a brief introduction to the application scenarios and implementation environment involved in this application is first given.

[0045] First, a brief introduction to the application scenarios involved in this application is given.

[0046] As mentioned in the background technology, with the rapid development of information technology, various types of data have grown explosively, providing rich resources for innovation and development in various fields. At the same time, model technology is also constantly developing, especially in the field of text processing, the importance of text classification models is becoming increasingly prominent. The efficient training of text classification models depends on high-quality structured text.

[0047] When training text classification models, general technology collects text from a variety of sources (e.g., web pages, databases, social media), and processes and annotates it with the help of deep learning algorithms. However, the text data collected by general technology usually has redundant fields, which interferes with deep learning's understanding of the meaning of text data. In this way, it is difficult for deep learning algorithms to accurately extract key information from the text, resulting in frequent omissions or annotation errors. This makes the training text used to train text classification models lack accuracy and low quality. In order to improve the quality of training texts, manual verification and annotation are usually required. However, manual annotation verification is time-consuming, labor-intensive, and inefficient.

[0048] From the above, we can see that the training texts used to train text classification models lack accuracy and completeness, and there is a shortage of high-quality training texts and data imbalance.

[0049] Therefore, the quality of the training text used to train the text classification model in the general technology is low, resulting in low accuracy of the text classification model.

[0050] In view of the above problems, an embodiment of the present application provides a text classification model training method. First, an initial training text for training a text classification model is obtained. Then, the initial training text is encoded to obtain encoding features of the initial training text. The encoding features include redundant field features, which are used to represent redundant fields in the initial training text. Then, the redundant field features are merged to obtain a target training text. Then, a text classification model is trained based on the target training text.

[0051] As can be seen from the above, the present application can encode the initial training text used to train the text classification model to obtain the encoding features of the initial training text. And the redundant field features corresponding to the redundant fields are merged to obtain the target training text. The target training text obtained by the present application with redundant fields removed can improve the quality of the training text. Moreover, the present application trains a text classification model based on the target training text, which can reduce the computational cost of the text classification model and improve the accuracy of the text classification model.

[0052] The implementation environment of the above-mentioned text classification model training method can be the text classification model training system provided in the embodiment of the present application.

[0053] Figure 1 FIG. 1 is a schematic diagram showing the structure of a text classification model training system provided by an embodiment of the present application. Figure 1 As shown, the text classification model training system includes: a text classification model training device 101 and a data storage device 102.

[0054] The text classification model training device 101 and the data storage device 102 are communicatively connected.

[0055] In practical applications, the text classification model training device 101 can be connected to any number of data storage devices 102. For ease of understanding, Figure 1 An example is given for describing the connection between a text classification model training device 101 and a data storage device 102 .

[0056] In an embodiment of the present application, the data storage device 102 is used to provide data for text classification model training (for example, initial training text, etc.) to the text classification model training device 101, so that the text classification model training device 101 implements text classification model training according to the data sent by the data storage device 102.

[0057] Optionally, the physical device of the text classification model training device 101 may be a server, a terminal, or other types of electronic devices, which is not limited in the embodiments of the present application.

[0058] Optionally, the terminal may be a device that provides voice and / or data connectivity to a user, a handheld device with wireless connection function, or other processing device connected to a wireless modem. The wireless terminal may communicate with one or more core networks via a radio access network (RAN). The wireless terminal may be a mobile terminal, such as a mobile phone (or "cellular" phone) and a computer with a mobile terminal, or a portable, pocket-sized, handheld, computer-built-in or vehicle-mounted mobile device that exchanges language and / or data with a radio access network, such as a mobile phone, tablet computer, laptop computer, netbook, personal digital assistant (PDA).

[0059] Optionally, the above-mentioned server can be a server in a server cluster (consisting of multiple servers), or a chip in the server, or a system on a chip in the server, or can be implemented by a virtual machine (VM) deployed on a physical machine, which is not limited in the embodiments of the present application.

[0060] Optionally, the text classification model training device 101 and the data storage device 102 may be two devices independently set up, or may be integrated into the same device. When the text classification model training device 101 and the data storage device 102 are integrated into the same device, the data storage device 102 may be a storage module (such as a database, etc.) of the text classification model training device 101.

[0061] It is easy to understand that when the text classification model training device 101 and the data storage device 102 are integrated into the same device, the communication mode between the text classification model training device 101 and the data storage device 102 is the communication between the internal modules of the device. In this case, the communication process between the two is the same as the communication process between the text classification model training device 101 and the data storage device 102 when they are independent of each other.

[0062] For ease of understanding, the present application is described by taking the example that the text classification model training device 101 and the data storage device 102 are independent of each other.

[0063] The text classification model training device in the text classification model training system includes: Figure 2 The following are the components included. Figure 2 Taking the communication device shown as an example, the hardware structure of the text classification model training device is introduced.

[0064] Figure 2, which is a schematic diagram of a hardware structure of a communication device provided in an embodiment of the present application. The communication device includes a processor 21, a memory 22, a communication interface 23, and a bus 24. The processor 21, the memory 22, and the communication interface 23 can be connected via a bus 24.

[0065] The processor 21 is the control center of the communication device, which can be a processor or a general term for multiple processing elements. For example, the processor 21 can be a general-purpose central processing unit (CPU) or other general-purpose processors. Among them, the general-purpose processor can be a microprocessor or any conventional processor.

[0066] As an embodiment, the processor 21 may include one or more CPUs, such as Figure 2 CPU0 and CPU1 are shown in the figure.

[0067] The memory 22 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0068] In a possible implementation, the memory 22 may exist independently of the processor 21, and the memory 22 may be connected to the processor 21 via a bus 24 to store instructions or program codes. When the processor 21 calls and executes the instructions or program codes stored in the memory 22, the text classification model training method provided in the following embodiments of the present application can be implemented.

[0069] In the embodiment of the present application, for the communication device, the software programs stored in the memory 22 are different, so the functions implemented by the communication device are different. The functions performed by each device will be described in conjunction with the following flowchart.

[0070] In another possible implementation, the memory 22 may also be integrated with the processor 21 .

[0071] The communication interface 23 is used for the communication device to connect with other devices through a communication network, and the communication network may be Ethernet, wireless access network, wireless local area network (WLAN), etc. The communication interface 23 may include a receiving unit for receiving data and a sending unit for sending data.

[0072] The bus 24 may be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 2 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0073] It should be pointed out that Figure 2 The structure shown in the figure does not constitute a limitation on the communication device, except Figure 2 In addition to the components shown, the communication device may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0074] The text classification model training method provided in the embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0075] The text classification model training method provided in the embodiment of the present application is applied to Figure 1 The text classification model training device 101 in the text classification model training system shown in FIG. Figure 3 As shown, the text classification model training method provided in the embodiment of the present application includes:

[0076] S301: Obtain initial training text for training a text classification model.

[0077] The text classification model training device can obtain an initial training text for training the text classification model, and process the initial training text to obtain a target training text. Further, the text classification model training device can train the text classification model based on the target training text. In this way, the text classification model training device can improve the accuracy of the text classification model.

[0078] Text classification models can be applied to a variety of scenarios to classify text in different scenarios.

[0079] Optionally, in the intent recognition scenario, the text classification model can classify the text to identify the intent of the text. The intent categories include: business consultation, business handling, complaints and suggestions, and fee inquiry and payment.

[0080] Optionally, in the context of sentiment recognition, the text classification model can classify the text to identify the sentiment tendency of the text. The sentiment categories include: negative sentiment class (also called negative class) and positive sentiment class (also called positive class).

[0081] Combination Figure 1 , the text classification model training device can obtain the initial training text for training the text classification model from the data storage device.

[0082] In one practicable manner, the text classification model training device may obtain initial training text from a public Chinese data set, and remove abnormal text, invalid text, or repeated text from the initial training text to clean the initial training text.

[0083] S302: Encode the initial training text to obtain encoding features of the initial training text.

[0084] The encoding features include redundant field features, which are used to represent redundant fields in the initial training text.

[0085] The feature content of redundant field features is the same.

[0086] The text classification model training device can encode the initial training text to obtain the encoding features of the initial training text. The text classification model training device can merge the redundant field features to obtain the target training text merged with the redundant fields. In this way, the text classification model training device can improve the text quality of the target training text. Further, the text classification model training device can train the text classification model based on the target training text. In this way, the text classification model training device can improve the accuracy of the text classification model.

[0087] Coding features include: redundant field features, field position features, and sentence features.

[0088] Optionally, the encoding features also include: key field features.

[0089] Optionally, the coding feature also includes: symbol feature.

[0090] Among them, the redundant field feature is used to represent the redundant fields in the initial training text. The key field feature is used to represent the key fields in the initial training text. The field position feature is used to represent the field position of the field in the initial training text. The sentence feature is used to represent the sentence to which the field belongs. The symbol feature is used to represent the symbol feature of the symbol.

[0091] In some embodiments, before encoding the initial training text, the text classification model training device may perform word segmentation on the initial training text.

[0092] Optionally, the text classification model training device can perform word segmentation processing on the initial training text based on the word segmentation function lcut in the jieba library.

[0093] For example, when the initial training text is [“How can I help you? I want to apply for broadband”], the initial training text after word segmentation processing is [“Excuse me”, “What is there”, “Can”, “Help you”, “I”, “Want”, “Apply”, “Broadband”].

[0094] The text classification model training device encodes the initial training text to obtain redundant field features, key field features, symbol features, field position features and sentence features of the initial training text.

[0095] The text classification model training device encodes the redundant fields, key fields, and symbols in the initial training text to obtain the redundant field features, key field features, and symbol features in the initial text training, respectively.

[0096] Among them, the redundant field features have the same feature content and are all represented by E [word] The characteristic contents of symbolic features are the same and are all represented by [SEP].

[0097] In some embodiments, the text classification model training device can encode key fields in the initial training text based on the Chinese word vector library (which can also be called encoding of the text entity embedding dimension (Token Embeddings)).

[0098] The text classification model training device encodes the field positions in the initial training text (which may also be referred to as encoding of the text entity position dimension (Position Embeddings)) to obtain field position features.

[0099] The text classification model training device can encode the field positions in the initial training text based on the one-hot method to obtain the field position features.

[0100] The text classification model training device encodes the sentences to which the fields in the initial training text belong (also referred to as encoding of the text entity separation dimension (Segment Embeddings)) to obtain sentence features.

[0101] For example, Figure 4 A schematic diagram of an encoding result provided by an embodiment of the application is shown.

[0102] Among them, [Class (CLS)] is used to indicate the category to which the initial training text belongs, [word (word)] is used to indicate the redundant fields of the initial training text, [field 1] and [field 2] are used to indicate the key fields in the initial training text, and [separator (SEP)] is used to indicate symbols in the initial training text, such as commas, periods, and exclamation marks.

[0103] Among them, the field characteristics of [CLS] are "E [CLS] ", the field feature of [word] is "E [word] ", [SEP] field features are "E [SEP] ", the key field [field 1] has the field characteristic "E [字段1] ", the field characteristics of [Field 2] are "E [字段2] ”.

[0104] In some embodiments, "E [CLS] " can be set to 101, "E [word] " can be set to 102, "E [SEP] " can be set to 103.

[0105] The text classification model training device can determine the “E [字段1] "E [字段2] "The values ​​corresponding to key field features such as "(can also be called vectors).

[0106] Among them, “[CLS][word][word][word][word][SEP]” belongs to the same clause, and its clause features are the same, which can be marked as “E A "[word][word][word][field1][field2][SEP]" belong to the same clause and have the same clause features, so they can be marked as "E B ”.

[0107] In some embodiments, "E A " can be set to 0, "E B " can be set to 1.

[0108] Among them, the field position features of “[CLS][word][word][word][word][SEP][word][word][word][field1][field2][SEP]” are “E0, E1, E2, E3, E4, E5, E6, E7, E8, E9, E10, E11, E12, E13, E14, E15, E16, E17, E18, E19, E20, E21, E22, E23, E24, E25, E26, E27, E28, E29, E30, E31, E32, E33, E34, E35, E36, E37, E38, E39, E40, E41, E42, E43, E44, E45, E46, E47, E48, E49, E50, E51, E52, E53, E54, E55 10 、E 11 ”.

[0109] In some embodiments, "E0, E1, E2, E3, E4, E5, E6, E7, E8, E9, E 10 、E 11 " can be set to 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 respectively.

[0110] S303: Merge redundant field features to obtain target training text.

[0111] The text classification model training device can merge the redundant field features to obtain the target training text merged with the redundant fields. In this way, the text classification model training device can improve the text quality of the target training text. Further, the text classification model training device can train the text classification model based on the target training text. In this way, the text classification model training device can improve the accuracy of the text classification model.

[0112] S304: Train a text classification model based on the target training text.

[0113] The text classification model training device can train a text classification model based on a target training text. The target training text is a training text that has been processed by merging redundant fields. In this way, the text classification model training device can ignore redundant fields to better understand the target training text and improve the accuracy of the text classification model.

[0114] Optionally, the text classification model training device can also test the text to be classified based on the trained text classification model to evaluate the classification effect of the text classification model.

[0115] The text classification models include: Bidirectional Encoder Representations from Transformers (Bert) model and Global Vectors for Word Representation (GloVe) model.

[0116] In some embodiments, the encoding feature also includes a field position feature of the redundant field, the field position feature is used to indicate the field position of the redundant field in the initial training text, combined with Figure 3 ,like Figure 5 As shown, in S303, the redundant field features are merged to obtain the target training text, which specifically includes:

[0117] S501. Determine consecutive redundant fields according to field position characteristics.

[0118] The continuous redundant fields are redundant fields whose field positions are continuous in the initial training text.

[0119] In one achievable manner, the text classification model training device can determine the continuous redundant fields according to the field position feature. In this way, the text classification model training device can merge the redundant fields with continuous field positions in the initial training text to obtain the target training text. In this way, the target training text with the continuous redundant fields merged can be obtained. The text classification model training device can improve the text quality of the target training text.

[0120] like Figure 4 The field position characteristics shown in the figure, where the field corresponding to the field position characteristics of "E1, E2, E3, E4" is a redundant field, therefore, the redundant field corresponding to the field position characteristics of "E1, E2, E3, E4" is a continuous redundant field. Wherein, the redundant field corresponding to the field position characteristics of "E6, E7, E8" is a redundant field, therefore, the redundant field corresponding to the field position characteristics of "E1, E2, E3, E4" is a continuous redundant field. Therefore, the redundant field corresponding to the field position characteristics of "E6, E7, E8" is a continuous redundant field.

[0121] S502: Merge redundant field features of consecutive redundant fields to obtain target training text.

[0122] The text classification model training device can merge the redundant field features of the continuous redundant fields to obtain the target training text with the merged redundant fields. In this way, the text classification model training device can improve the text quality of the target training text.

[0123] Exemplary, combined Figure 4 ,like Figure 6 As shown, Figure 6 Another schematic diagram of encoding results provided by an embodiment of the application is shown.

[0124] Among them, the redundant fields corresponding to the field position features of "E1, E2, E3, E4" are continuous redundant fields, and the redundant fields corresponding to the field position features of "E6, E7, E8" are continuous redundant fields. The text classification model training device merges the redundant field features of the continuous redundant fields to obtain the target training text.

[0125] The target training text is “[CLS][word][SEP][word][field1][field2][SEP]”.

[0126] Among them, the redundant field feature after merging is E [ ' word] .

[0127] Among them, "[CLS][word][SEP]" belongs to the same clause, and its clause features are the same, so it can be marked as E A "[word][field 1][field 2][SEP]" belong to the same clause, and their clause features are the same, so they can be marked as E. B .

[0128] The text classification model training device re-encodes the field positions in the initial training text after merging the redundant fields to obtain the field position features in the target training text.

[0129] Among them, the field position features of “[CLS][word][SEP][word][field 1][field 2][SEP]” are “E0, E1, E2, E3, E4, E5, E6” respectively.

[0130] The vector dimension of the above initial training text is (n=1, m=1), where n is the number of fields and m is the field feature dimension. The value of m is based on the dimension of the word vector in the Chinese word vector library. If the word vector in the Chinese word vector library used is 512-dimensional, then m=512.

[0131] From the above, we can see that the field length (also called vector length) of the initial training text is 12. After the merging process, the field length of the target training text is 7, and the target training text retains the text segmentation information and position information. Under the premise of simplifying the content of the training text, the core semantics of the training text is retained.

[0132] In some embodiments, the encoding feature also includes the sentence feature of the sentence to which the redundant field belongs, combined with Figure 5 ,like Figure 7 As shown, in S402, the redundant field features of the continuous redundant fields are merged to obtain the target training text, which specifically includes:

[0133] S701. Determine consecutive redundant fields in the same sentence according to sentence features.

[0134] In one achievable manner, the text classification model training device can determine the continuous redundant fields in the same sentence according to the sentence features. In this way, the text classification model training device can merge the redundant fields with continuous field positions in the initial training text to obtain the target training text. In this way, the target training text with the continuous redundant fields merged can be obtained. The text classification model training device can improve the text quality of the target training text.

[0135] The text classification model training device can determine the sentence to which a field (including: key field, redundant field) belongs according to the sentence features, and determine the continuous redundant fields in the same sentence.

[0136] Combination Figure 4 , the text classification model training device can determine the sentence feature as E A The text classification model training device can determine the sentence feature as E B and determine the consecutive redundant fields in clause B.

[0137] S702: Merge redundant field features of consecutive redundant fields in the same sentence to obtain target training text.

[0138] The text classification model training device can merge the redundant field features of consecutive redundant fields in the same sentence to obtain a target training text with merged redundant fields. In this way, the text classification model training device can improve the text quality of the target training text.

[0139] In some embodiments, in combination Figure 7 ,like Figure 8 As shown, in S702, the redundant field features of consecutive redundant fields in the same sentence are merged to obtain the target training text, specifically including:

[0140] S801. Determine redundant field features for merging processing according to redundant field features of consecutive redundant fields in the same sentence and the number of consecutive redundant fields in the same sentence.

[0141] The text classification model training device can determine the redundant field features for merging processing based on the redundant field features of the consecutive redundant fields in the same sentence and the number of the consecutive redundant fields in the same sentence.

[0142] Among them, the redundant field characteristics after merging satisfy the following formula:

[0143]

[0144] Among them, E [word] It is used to represent the redundant field characteristics before merging. [ ' word] It is used to represent the characteristics of redundant fields after merging. decay is a custom constant term, and x is used to represent the number of consecutive redundant fields.

[0145] The decay value range is [0.0, 1.0].

[0146] It can be understood that when the number of consecutive redundant fields is 0, E [ ' word] is 0. When the number of consecutive redundant fields tends to infinity, E [ ' word] Infinitely close to E [word] .

[0147] For example, the initial training text is "I want to apply for a broadband", and after the merging process, the field features of the target training text are "[Apply][E [ ' word] ][Broadband]". The initial training text is "This traffic package has been processed. Thank you for your patience. There are discounts on broadband packages recently". After merging, the field features of the target training text are "[Traffic Package][Processing][E [ ' word] ][Broadband][Package][Preferential Activities]". In the above training text, "[Handling][E [ ' word] "E" in "[broadband]" [ ' word] The values ​​of are different to distinguish the different meanings of the above texts.

[0148] For example, when the initial training text 1 is “Customer Service: Hello, I’m happy to serve you. Customer: Hey, hello, I’d like to inquire about that broadband service. Customer Service: Well, you say. Customer: We’ll install a broadband,” the text classification model training device determines that the field features corresponding to the initial training text 1 after the merge processing are “[CLS][word][SEP][word][“Inquiry”][word][“Broadband Service”][SEP][word][SEP][word][“Installation”][word][“Broadband”][word][SEP]”. When the initial training text 2 is “Customer Service: Hello, I’m happy to serve you. How can I help you? Customer: Hello. Customer Service: Hello, how can I help you? Customer: Ah, I want to ask you this. Customer: Why is the network speed so slow? Sometimes there is no network at all,” the classification model training device determines that the field features corresponding to the initial training text 2 after the merge processing are “[CLS][word][SEP][word][SEP][word][SEP][word][“Network Speed”][word][“Slow”][word][“No Network”][SEP]”.

[0149] In the Chinese word vector library, "E[CLS]", "E[word]", "E[SEP]", "E[consulting]", "E[security]", "E[broadband]", "E[speed]", and "E[slow]" are 101, 102, 103, 869, 5578, 3345, 5362, 7458, 562, and 7835, respectively. Therefore, the text classification model training device determines that the field features of the initial training text 1 are [101, 102*Dx, 103, 102*Dx, 869, 102*Dx, 5578, 103, 102*Dx, 103, 102*Dx, 3345, 102*Dx, 5362, 102*Dx, 103]. Therefore, the text classification model training device determines that the field features of the initial training text 2 are [101, 102*Dx, 103, 102*Dx, 103, 102*Dx, 103, 102*Dx, 7458, 102*Dx, 562, 102*Dx, 7835, 103].

[0150] S802: performing feature fusion on the merged redundant field features, field position features and sentence features to obtain a target training text.

[0151] The text classification model training device can perform feature fusion on the merged redundant field features, field position features and sentence features to obtain the target training text.

[0152] In one achievable manner, the text classification model training device may train the text classification model based on the sum of features of the merged redundant field features, field position features, and sentence features.

[0153] In some embodiments, in combination Figure 8 ,like Fig. 9 As shown, before encoding the initial training text to obtain the encoding features of the initial training text, the method further includes:

[0154] S901: Determine application scenarios for training text classification models.

[0155] S902: Determine redundant fields in the initial training text according to a redundant field library corresponding to the application scenario of the training text classification model.

[0156] In the embodiment of the present application, the text classification model training device can determine the application scenario of the training text classification model, and determine the redundant fields in the initial training text according to the redundant field library corresponding to the application scenario of the training text classification model. In this way, the target training text obtained by the text classification model training device is more targeted, which can better train the text classification model and improve the accuracy of the text classification model.

[0157] In one practicable manner, the text classification model training device can obtain the frequency of various words and determine redundant fields (also called non-essential words and sentences), such as fixed-language sentences, adverbs with higher frequency, adjectives, and modal particles.

[0158] In another achievable manner, the text classification model training device may obtain manually counted redundant fields.

[0159] Optionally, the text classification model training device can build different redundant field libraries according to different application scenarios.

[0160] In the scenario where the text classification model is applied to intent recognition in the operator's customer service hotline, the redundant field library of the intent recognition model includes: "Hello", "Hello", "Glad to serve you", "How can I help you", "I want", "Um", "Ah", "Wait a minute", "I want to ask", "Please", "Thank you", "Thank you for your call", "There are currently a large number of consultations", "Manual customer service", "This phone number", "No need to reply", "This conversation is about to end", "Since your conversation has been unresponsive", "Dear user", "My best efforts", and "I hope you are 100% satisfied".

[0161] For example, the initial training text includes: "How can I help you? Hello, I want to apply for broadband", "Why is the network speed so slow? Wait a minute, let me check", and when the text classification model is an intent recognition model, the redundant fields include: "How can I help you?", "Hello, I want", "This", "Why so", "Wait a minute, let me check". When the text classification model is a sentiment recognition model, the redundant fields include: "Broadband", "This network speed".

[0162] For example, Fig.10 To illustrate another schematic diagram of encoding results provided by an embodiment of the present application. The initial training text is "How can I help you? Hello, I want to apply for broadband.", wherein the redundant fields include: "Excuse me", "What is there", "Can", "Help you", "Hello", "I", "Want". The text classification model training device can replace the redundant fields with "[word]". Among them, the field features corresponding to the initial training text are "[CLS][word][word][word][word][SEP][word][word][word][Apply][Broadband][SEP]".

[0163] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to realize the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0164] The embodiment of the present application can divide the text classification model training device into functional modules according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. Optionally, the division of modules in the embodiment of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0165] Fig.11 FIG. 1 is a schematic diagram showing the structure of a text classification model training device provided in an embodiment of the present application. Fig.11 As shown, the text classification model training device includes: a communication unit 1101 and a processing unit 1102;

[0166] The communication unit 1101 is used to obtain the initial training text for training the text classification model; the processing unit 1102 is used to encode the initial training text to obtain the encoding features of the initial training text; the encoding features include redundant field features; the redundant field features are used to represent the redundant fields in the initial training text; the processing unit 1102 is also used to merge the redundant field features to obtain the target training text; the processing unit 1102 is also used to train the text classification model based on the target training text.

[0167] Optionally, the encoding features also include field position features of redundant fields; the field position features are used to indicate the field positions of redundant fields in the initial training text; the processing unit 1102 is specifically used to: determine continuous redundant fields according to the field position features; the continuous redundant fields are redundant fields with continuous field positions in the initial training text; and merge the redundant field features of the continuous redundant fields to obtain the target training text.

[0168] Optionally, the encoding features also include sentence features of the sentence to which the redundant field belongs; the processing unit 1102 is specifically used to: determine consecutive redundant fields in the same sentence based on the sentence features; and merge the redundant field features of consecutive redundant fields in the same sentence to obtain the target training text.

[0169] Optionally, the processing unit 1102 is specifically used to: determine the redundant field features for merging based on the redundant field features of consecutive redundant fields in the same sentence and the number of consecutive redundant fields in the same sentence; and perform feature fusion on the redundant field features, field position features and sentence features for merging to obtain the target training text.

[0170] Optionally, the feature content of redundant field features is the same.

[0171] Optionally, before encoding the initial training text to obtain the encoding features of the initial training text, the processing unit 1102 is also used to determine the application scenario of the training text classification model; the processing unit 1102 is also used to determine the redundant fields in the initial training text based on the redundant field library corresponding to the application scenario of the training text classification model.

[0172] An embodiment of the present application also provides a computer-readable storage medium, which includes computer execution instructions. When the computer execution instructions are executed on a computer, the computer executes the text classification model training method provided in the above embodiment.

[0173] An embodiment of the present application also provides a computer program, which can be directly loaded into a memory and contains software code. After being loaded and executed by a computer, the computer program can implement the text classification model training method provided in the above embodiment.

[0174] Those skilled in the art should be aware that in one or more of the above examples, the functions described in this application can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer-readable storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. Storage media can be any available media that can be accessed by general or special-purpose computers.

[0175] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0176] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The unit described as a separate component may or may not be physically separated, and the component displayed as a unit may be a physical unit or multiple physical units, that is, it may be located in one place, or it may be distributed in multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0177] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the general technology or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a ROM, a RAM, a disk, or an optical disk.

[0178] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A text classification model training method, characterized in that: include: Obtain initial training text for training a text classification model; Performing encoding processing on the initial training text to obtain encoding features of the initial training text; the encoding features include redundant field features; The redundant field feature is used to represent the redundant fields in the initial training text; Merging the redundant field features to obtain target training text; The text classification model is trained based on the target training text.

2. The method according to claim 1, characterized in that The encoding feature also includes a field position feature of the redundant field; The field position feature is used to indicate the field position of the redundant field in the initial training text; and the merging process of the redundant field features to obtain the target training text includes: Determine continuous redundant fields according to the field position feature; the continuous redundant fields are redundant fields with continuous field positions in the initial training text; The redundant field features of the continuous redundant fields are merged to obtain the target training text.

3. The method according to claim 2, characterized in that The encoding feature also includes a sentence feature of the sentence to which the redundant field belongs; the merging process of the redundant field features of the continuous redundant fields to obtain the target training text includes: Determine continuous redundant fields in the same sentence according to the sentence characteristics; The redundant field features of the consecutive redundant fields in the same sentence are merged to obtain the target training text.

4. The method according to claim 3, characterized in that: The merging of redundant field features of consecutive redundant fields in the same sentence to obtain the target training text includes: Determining redundant field features for merging processing according to redundant field features of consecutive redundant fields in the same sentence and the number of consecutive redundant fields in the same sentence; The redundant field features, the field position features and the sentence features processed by the merging process are subjected to feature fusion to obtain the target training text.

5. The method according to claim 1, characterized in that The feature contents of the redundant field features are the same.

6. The method according to claim 1, characterized in that Before encoding the initial training text to obtain the encoding features of the initial training text, the method further includes: Determining an application scenario of the training text classification model; The redundant fields in the initial training text are determined according to the redundant field library corresponding to the application scenario of the training text classification model.

7. A text classification model training device, characterized in that: The device comprises: a communication unit and a processing unit; The communication unit is used to obtain initial training text for training the text classification model; The processing unit is used to encode the initial training text to obtain the encoding features of the initial training text; the encoding features include redundant field features; the redundant field features are used to represent redundant fields in the initial training text; The processing unit is further used to merge the redundant field features to obtain a target training text; The processing unit is further used to train the text classification model based on the target training text.

8. A text classification model training device, characterized in that: include: A processor and a memory; wherein the memory is used to store one or more programs, and the one or more programs include computer-executable instructions. When the device is running, the processor executes the computer-executable instructions stored in the memory to enable the device to perform the method described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: When the computer-executable instructions stored in the computer-readable storage medium are executed by a processor of a text classification model training device, the text classification model training device can execute the method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product comprises: a computer program or instructions, and when the computer program or instructions are run on a computer, the computer is caused to perform the method according to any one of claims 1 to 6.

Images