Natural language processing method, system, model training method, medium and device

Abstract

The application provides a natural language processing model training method, comprising inputting a text to be processed into a natural language processing model; the natural language processing model comprises a basic branch and a natural language processing branch; text features and text mapping features are obtained based on the basic branch; the text mapping features and the text features are processed based on the natural language processing branch to obtain a multi-task natural language processing result of the text to be processed; and the natural language processing model is trained according to the text to be processed and the multi-task natural language processing result to obtain a trained natural language processing model. The natural language processing method, system, model training method, medium and equipment provided by the application can simultaneously perform different natural language processing tasks, including text classification, text translation, named entity recognition and text correction tasks, thereby effectively reducing hardware resource consumption and labor cost.

Description

TECHNICAL FIELD

[0001] The present application relates to a natural language processing method, in particular to a natural language processing method, system, model training method, medium and device. BACKGROUND

[0002] In today's society, with the improvement of computing power and the wide application of deep learning technologies such as convolutional neural networks in the field of natural language, the performance of many natural language processing tasks has made significant progress, such as semantic segmentation, classification, translation, etc. At present, many researches in the field of natural language are limited to solving a single processing task. This makes it necessary to design a model separately for different natural language processing tasks. If you want to realize multiple functions at the same time, you need to deploy multiple models to process different natural language processing tasks. Such an approach not only increases the consumption of hardware resources, but also increases the cost of manual design. SUMMARY

[0003] In view of the above-mentioned shortcomings of the prior art, the purpose of the present application is to provide a natural language processing method, system, model training method, medium and device, which solves the problem that a single model cannot process multiple natural language tasks at the same time in the prior art.

[0004] To achieve the above-mentioned purpose and other related purposes, the first aspect of the present application provides a training method of a natural language processing model, comprising inputting a text to be processed into a natural language processing model; the natural language processing model comprises a basic branch and a natural language processing branch; obtaining text features and text mapping features based on the basic branch; processing the text mapping features and the text features based on the natural language processing branch to obtain a multi-task natural language processing result of the text to be processed; training the natural language processing model according to the text to be processed and the multi-task natural language processing result to obtain a trained natural language processing model.

[0005] In an embodiment of the first aspect, the processing of the text mapping features and the text features based on the natural language processing branch to obtain the multi-task natural language processing result of the text to be processed comprises obtaining a correction result of the text to be processed based on a correction branch; obtaining a recognition result of the text to be processed based on a named entity recognition branch; obtaining a classification result of the text to be processed based on the classification branch; obtaining a translation result of the text to be processed based on the translation branch.

[0006] In an embodiment of the first aspect, the obtaining the error correction result of the text to be processed based on the error correction branch comprises: obtaining an error correction feature based on the text mapping feature; inputting the text feature and the error correction feature into a cross-attention mechanism and a self-attention mechanism to obtain an error correction fusion result; and obtaining the error correction result based on the error correction fusion result.

[0007] In an embodiment of the first aspect, the obtaining the recognition result of the text to be processed based on the named entity recognition branch comprises: obtaining a recognition feature based on the text mapping feature; inputting the text feature and the recognition feature into a cross-attention mechanism, a long short-term memory network and a conditional random field to obtain the recognition result.

[0008] In an embodiment of the first aspect, the obtaining the classification result of the text to be processed based on the classification branch comprises: obtaining a classification feature based on the text mapping feature; and inputting the text feature and the classification feature into a cross-attention mechanism to obtain the classification result.

[0009] In an embodiment of the first aspect, the obtaining the translation result of the text to be processed based on the translation branch comprises: obtaining a translation feature based on the text mapping feature; inputting the text feature and the classification feature into a cross-attention mechanism to obtain a translation fusion feature; and obtaining the translation result based on the translation fusion feature.

[0010] A second aspect of the present application provides a natural language processing method, comprising: obtaining a text to be processed; and processing the text to be processed based on a natural language processing model to obtain a multi-task natural language processing result; the natural language processing model is trained by the training method of the first aspect.

[0011] A third aspect of the present application provides a natural language processing system, comprising: an obtaining module configured to obtain a text to be processed; and a processing module configured to process the text to be processed based on a natural language processing model to obtain a multi-task natural language processing result; the natural language processing model is trained by the training method of the first aspect.

[0012] A fourth aspect of the present application provides a computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the training method of the natural language processing model of the first aspect and / or the natural language processing method of the second aspect.

[0013] In a fifth aspect, the present application provides an electronic device, comprising a memory storing a computer program, and a processor connected to the memory and configured to execute the training method of the natural language processing model and / or the natural language processing method when the computer program is invoked.

[0014] As described above, the natural language processing method, system, model training method, medium and device provided by the embodiments of the present application have the following beneficial effects: by training the natural language processing model, different natural language processing tasks can be performed simultaneously, including text classification, text translation, named entity recognition and text correction tasks, thereby effectively reducing hardware resource consumption and labor costs. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 A flowchart of the training method of the natural language processing model is shown.

[0016] Figure 2 A flowchart of the training method of the natural language processing model is shown.

[0017] Figure 3 A flowchart of the training method of the natural language processing model is shown.

[0018] Figure 4 A flowchart of the natural language processing method is shown.

[0019] Figure 5 A structural diagram of the natural language processing system is shown.

[0020] Figure 6 A structural diagram of the electronic device is shown.

[0021] ELEMENT NUMBER EXPLANATION

[0022] 20 acquisition module

[0023] 30 processing module

[0024] 50 electronic device

[0025] 501 memory

[0026] 502 processor

[0027] 503 display

[0028] S1-S4 steps

[0029] S31-S34 steps

[0030] S6~S7 steps DETAILED DESCRIPTION

[0031] Other advantages and benefits of the present application will become apparent to those skilled in the art upon consideration of the disclosure or can be learned by practice of the application. The application can be realized and achieved by means of the structures and combinations of structures described in this specification and it is therefore to be understood that various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the present application. It is intended that the following examples and features thereof be considered as exemplary only and that various modifications, equivalents, and alternatives thereto are possible and contemplated within the scope of the present application. Where there is no conflict, the following examples and features thereof can be combined with each other.

[0032] It is to be understood that the drawings shown below are only schematic and that the actual implementation of the application can vary as a consequence of, for example, manufacturing techniques, or tolerances, and are therefore not to be considered limiting the application. The following terms are used in the description: the singular forms "a", "an", and "the" include plural referents unless the context clearly dictates otherwise. The terms "comprises", "comprising", "includes", "including" and the like can be used herein. Such terms are inclusive and allow for items, components, elements, or steps not expressly mentioned or subsequent actions, limitations, or possibilities beyond those listed. The terms "coupled" and "connected" and variations thereof describe a relationship between or among two or more entities or components, as defined by the context.

[0033] The present application provides a natural language processing method, system, model training method, medium and device, which can simultaneously perform different natural language processing tasks, including text classification, text translation, named entity recognition and text correction tasks, thereby effectively reducing hardware resource consumption and labor costs.

[0034] Next, the natural language processing method, system, model training method, medium and device provided by the present application will be described by specific examples in conjunction with the drawings.

[0035] As shown in FIG. 1, in an embodiment, the training method of the natural language processing model of the present application includes steps S1-S4. Figure 1

[0036] S1: input the text to be processed into the natural language processing model; the natural language processing model includes a basic branch and a natural language processing branch.

[0037] As shown in FIG. 2, the structure of the natural language processing model and the natural language processing process will be described in detail below. Figure 2

[0038] The basic branch (Base branch) includes the encoder (Encoder) part of the transformation network model (Transformer).

[0039] ​​It should be noted that the encoding output result of the basic branch still needs to be input to the natural language processing branch after passing through the gate recurrent unit (GRU) to obtain the processing result of the multi-class natural language task.

[0040] The natural language processing branch includes a correction branch, a named entity recognition branch, a classification branch, and a translation branch.

[0041] Specifically, the correction branch includes a fully connected layer (FC layer) + cross-attention layer + self-attention layer + FC layer.

[0042] Specifically, the named entity recognition branch includes an FC layer + cross-attention layer + long short term memory (LSTM) + conditional random field (CRF layer).

[0043] Specifically, the classification branch includes an FC layer + cross-attention layer + FC layer.

[0044] Specifically, the translation branch FC layer + cross-attention layer + decoder (transformer) layer.

[0045] S2: Obtain text features and text mapping features based on the basic branch.

[0046] Specifically, the text to be processed is input into the basic branch, and the basic branch obtains text features feature_base through the encoder (transformer).

[0047] Specifically, the feature_base passes through the GRU to obtain the text mapping feature feature.

[0048] S3: Process the text mapping features and the text features based on the natural language processing branch to obtain the multi-task natural language processing result of the text to be processed.

[0049] As shown in Figure 3 Step S3 includes steps S31-S34.

[0050] S31: Obtain the correction result of the text to be processed.

[0051] Specifically, the text mapping feature feature obtained in step S2 is input into the error correction branch, an error correction feature feature_correct is obtained through an FC layer, and the error correction feature feature_correct is taken as Q and K, the text feature feature_base is taken as V, cross-attention operation is performed, and then a preliminary result is obtained through a self-attention layer and an FC layer.

[0052] Specifically, the preliminary result is input into an FC + sigmoid function to obtain an error position position of a character in the to-be-processed text.

[0053] Specifically, the preliminary result is input into an FC + softmax function to obtain a correct result text of an error character in the to-be-processed text.

[0054] S32: Obtain a recognition result of the to-be-processed text.

[0055] Specifically, the text mapping feature feature obtained in step S2 is input into the named entity recognition branch, a recognition feature feature_ner is obtained through an FC layer, and the recognition feature feature_ner is taken as Q and K, the text feature feature_base is taken as V, cross-attention operation is performed, and then an entity name, that is, a recognition result, is obtained through an LSTM and a CRF layer.

[0056] S33: Obtain a classification result of the to-be-processed text.

[0057] Specifically, the text mapping feature feature obtained in step S2 is input into the classification branch, a classification feature feature_class is obtained through an FC layer, and the classification feature feature_class is taken as Q and K, the text feature feature_base is taken as V, cross-attention operation is performed, and then a class class, that is, a classification result, is obtained through an FC layer.

[0058] S34: Obtain a translation result of the to-be-processed text.

[0059] Specifically, the text mapping feature feature obtained in step S2 is input into the translation branch, a translation feature feature_translate is obtained through an FC layer, and the translation feature feature_translate is taken as Q and K, the text feature feature_base is taken as V, cross-attention operation is performed, and then a translation fusion feature is obtained.

[0060] Specifically, the character SOS is input as a start special character of a translation result into a natural language processing model, and is passed through a decoder (transformer) layer together with a translation fusion feature to generate a translation result translate text word by word. When an EOS special character is encountered, translation is stopped.

[0061] S4: training the natural language processing model according to the to-be-processed text and the multi-task natural language processing result to obtain a trained natural language processing model.

[0062] As shown in Figure 4 , the present application also provides a natural language processing method, comprising steps S6-S7.

[0063] S6: obtaining a to-be-processed text.

[0064] S7: processing the to-be-processed text based on a natural language processing model to obtain a multi-task natural language processing result.

[0065] Specifically, the natural language processing model is a natural language processing model trained in steps S1-S4. The specific process of step S7 can be seen from Figure 2 .

[0066] As shown in Figure 5 , the present application also provides a natural language processing system, comprising an obtaining module 20 and a processing module 30.

[0067] The obtaining module 20 is configured to obtain a to-be-processed text.

[0068] The processing module 30 is configured to process the to-be-processed text based on a natural language processing model to obtain a multi-task natural language processing result; the natural language processing model is a natural language processing model trained in steps S1-S4.

[0069] Specifically, the obtaining module 20 inputs the to-be-processed text into the natural language processing model. The model obtains text features and text mapping features through a basic branch, and then processes the input text based on the text features and the text mapping features by an error correction branch, a named entity recognition branch, a classification branch, and a translation branch to obtain an error correction result, an identification result, a classification result, and a translation result, respectively. The specific processing process is shown in Figure 2 .

[0070] The present application also provides a computer-readable storage medium having a computer program stored thereon. The computer program is executed by a processor to implement the training method of the natural language processing model provided in the embodiments of the present application, and / or to implement the natural language processing method provided in the embodiments of the present application.

[0071] In the present application, any combination of one or more storage media can be employed. The storage media can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a RAM, a ROM, an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by an instruction execution system, apparatus, or device to execute the program.

[0072] The present application also provides an electronic device. Figure 6 Fig. 1 shows a schematic diagram of an electronic device 50 according to an embodiment of the present application. As shown in Fig. 1, the electronic device 50 comprises a memory 501 and a processor 502. Figure 6 The memory 501 is configured to store a computer program; preferably, the memory 501 comprises a ROM, a RAM, a disk, a U disk, a memory card, an optical disk, or other media capable of storing program codes.

[0073] In particular, the memory 501 can comprise a computer system readable medium in the form of volatile memory, such as a random access memory (RAM) and / or a cache memory. The electronic device 50 can further include other removable / non-removable, volatile / non-volatile computer system storage media. The memory 501 can include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of the embodiments of the present application.

[0074] The processor 502 is connected to the memory 501 and is configured to execute the computer program stored in the memory 501, so that the electronic device 50 executes the training method of the natural language processing model provided in the embodiments of the present application, and / or executes the natural language processing method provided in the embodiments of the present application.

[0075] The processor 502 is connected to the memory 501 and is configured to execute the computer program stored in the memory 501, so that the electronic device 50 executes the training method of the natural language processing model provided in the embodiments of the present application, and / or executes the natural language processing method provided in the embodiments of the present application.

[0076] Preferably, the processor 502 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0077] Preferably, in this embodiment, the electronic device 50 may further include a display 503. The display 503 is communicatively connected to the memory 501 and the processor 502, and is used to display the training method of the natural language processing model and / or the related GUI interactive interface of the natural language processing method.

[0078] The training method and / or the scope of protection of the natural language processing model described in this invention are not limited to the execution order of the steps listed in this embodiment. Any solution implemented by adding, subtracting, or replacing steps in the prior art based on the principles of this invention is included within the scope of protection of this invention.

[0079] In summary, the embodiments of the present invention provide a natural language processing method, system, model training method, medium, and device. This natural language processing model can simultaneously perform different natural language processing tasks, including text classification, text translation, named entity recognition, and text error correction, thereby effectively reducing hardware resource consumption and labor costs. Therefore, the present invention effectively overcomes the various shortcomings of the prior art and has high industrial application value.

[0080] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A training method for a natural language processing model, characterized in that, include: The text to be processed is input into the natural language processing model; the natural language processing model includes a basic branch and a natural language processing branch. Based on the aforementioned basic branches, text features and text mapping features are obtained; The basic branch obtains the text features through the encoder part of the transformation network model, and the text features are then passed through a gated recurrent unit to obtain the text mapping features. The text mapping features and text features are processed based on the natural language processing branch to obtain the multi-task natural language processing result of the text to be processed, including obtaining the error correction result of the text to be processed based on the error correction branch; The natural language processing model is trained based on the text to be processed and the results of the multi-task natural language processing to obtain a trained natural language processing model. The error correction result obtained from the text to be processed based on the error correction branch includes: Error correction features are obtained based on the text mapping features; The text features and the error correction features are input into a cross-attention mechanism and a self-attention mechanism to obtain the error correction fusion result; The error correction result is obtained based on the error correction fusion result.

2. The training method for the natural language processing model according to claim 1, characterized in that, The multi-task natural language processing results for obtaining the text to be processed by processing the text mapping features and the text features based on the natural language processing branch also include: The recognition result of the text to be processed is obtained based on the named entity recognition branch; The classification result of the text to be processed is obtained based on the classification branch; The translation result of the text to be processed is obtained based on the translation branch.

3. The training method for the natural language processing model according to claim 2, characterized in that, The recognition results obtained from the named entity recognition branch of the text to be processed include: Recognition features are obtained based on the text mapping features; The text features and the recognition features are input into a cross-attention mechanism, a long short-term memory network, and a conditional random field to obtain the recognition results.

4. The training method for the natural language processing model according to claim 2, characterized in that, Obtaining the classification result of the text to be processed based on the classification branch includes: Based on the text mapping features, classification features are obtained; The text features and the classification features are input into a cross-attention mechanism to obtain the classification result.

5. The training method for the natural language processing model according to claim 2, characterized in that, The translation results obtained based on the translation branch include: Translation features are obtained based on the text mapping features; The text features and the translation features are input into a cross-attention mechanism to obtain translation fusion features; The translation result is obtained based on the translation fusion features.

6. A natural language processing method, characterized in that, include: Get the text to be processed; The text to be processed is processed based on a natural language processing model to obtain multi-task natural language processing results; The natural language processing model is a natural language processing model trained by the training method described in any one of claims 1 to 5.

7. A natural language processing system, characterized in that, include: The acquisition module is used to acquire the text to be processed; The processing module is used to process the text to be processed based on a natural language processing model to obtain multi-task natural language processing results; The natural language processing model is a natural language processing model trained by the training method described in any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When executed by a processor, the computer program implements the training method of the natural language processing model according to any one of claims 1 to 5 and / or the natural language processing method according to claim 6.

9. A device, characterized in that, The device includes: A memory that stores a computer program; The processor, which is communicatively connected to the memory, executes the training method of the natural language processing model according to any one of claims 1 to 5 and / or the natural language processing method according to claim 6 when calling the computer program.

Images