Named entity recognition method and device and electronic equipment

Abstract

The invention discloses a named entity recognition method and device and electronic equipment. The method comprises the steps of obtaining to-be-recognized text data; a named entity recognition model is adopted to recognize to-be-recognized text data, a named entity recognition result is obtained, the named entity recognition model uses an embedded text vector layer as a bottom layer neural network, and the embedded text vector layer is used for enhancing a semantic representation result of the named entity recognition model; the parameters of the named entity recognition model are determined through the adversarial gradient and a combinatorial optimizer, the combinatorial optimizer comprises a first optimizer and a second optimizer, the first optimizer is used for updating the parameters of the named entity recognition model through a first learning rate, and the second optimizer is used for updating the parameters of the named entity recognition model through a second learning rate; the first learning rate is smaller than the second learning rate. The technical problem that a named entity recognition model in the prior art is poor in recognition ability and generalization ability is solved.

Description

Technical Field

[0001] The present application relates to the field of natural language processing, and in particular to a method, device and electronic device for named entity recognition. Background Art

[0002] With the rapid development of artificial intelligence technology, especially the continuous updating of natural language processing, named entity recognition has become an important technology in natural language processing. Its main purpose is to identify entity types with specific meanings from unstructured texts, including names of people, places, disease names, various proper nouns, etc. It is a prerequisite task for natural language processing technologies such as relationship extraction and question-answering systems. Therefore, as a relatively core task and research hotspot in the field of natural language processing, the quality of Chinese named entity recognition methods will seriously affect the healthy development of related corporate businesses.

[0003] In related technologies, for example, the hybrid learning BiLSTM-CRF method proposed by the Baidu team and later optimized by the NLP (Natural Language Processing) research team of Carnegie Mellon University mixes the statistical-based CRF (Conditional Random Field) and the deep learning-based BiLSTM (Bi-directional Long Short-Term Memory) model. However, this hybrid learning method does not make further improvements to the optimizer and training mechanism of the deep learning core, nor does it make improvements to the underlying representation model to enhance the generalization ability of the model. Its recognition ability of Chinese named entities is limited, and it is difficult to use in scenarios.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a method, apparatus and electronic device for named entity recognition, so as to at least solve the technical problem that the named entity recognition model in the related art has poor recognition ability and generalization ability.

[0006] According to one aspect of an embodiment of the present application, a method for named entity recognition is provided, comprising: obtaining text data to be recognized; using a named entity recognition model to recognize the text data to be recognized, and obtaining a named entity recognition result, wherein the named entity recognition model uses an embedded text vector layer as an underlying neural network, and the embedded text vector layer is used to strengthen the semantic representation result of the named entity recognition model, and the parameters of the named entity recognition model are determined by adversarial gradient and a combination optimizer, and the combination optimizer includes a first optimizer and a second optimizer, the first optimizer is used to update the parameters of the named entity recognition model through a first learning rate, and the second optimizer is used to update the parameters of the named entity recognition model through a second learning rate, and the first learning rate is less than the second learning rate.

[0007] Optionally, the named entity recognition model is determined in the following manner: obtaining historical text data and label data corresponding to the historical text data; constructing an initial named entity recognition model, wherein the initial named entity recognition model includes an embedded text vector layer; iteratively training the initial named entity recognition model based on the historical text data and the label data, stopping the iteration after a preset number of iterations is met, and determining the adversarial gradient of the initial named entity recognition model during each iteration, wherein the adversarial gradient is used to represent the stability of the initial named entity recognition model in the face of potential adversarial disturbances; determining the optimal parameters of the initial named entity recognition model based on the combined optimizer to obtain the named entity recognition model.

[0008] Optionally, after obtaining the historical text data and the label data corresponding to the historical text data, the method further includes: obtaining a mapping dictionary containing the label data and integer indexes, wherein the label data and the integer indexes in the mapping dictionary correspond one to one; and converting the label data into corresponding integer indexes according to the mapping dictionary.

[0009] Optionally, constructing an initial named entity recognition model includes: determining parameters of the initial named entity recognition model; determining a multi-layer neural network structure of the initial named entity recognition model, wherein the multi-layer neural network structure includes: an embedded text vector layer, a bidirectional long short-term memory network layer, a random dropout regularization layer, a fully connected layer, and a conditional random field layer.

[0010] Optionally, the iterative training includes initialization training and adversarial training, wherein the initialization training includes: determining the target loss function of the initial named entity recognition model during the iterative training process, wherein the target loss function is used to represent the difference between the recognition result of the initial named entity recognition model and the label data; determining the initial sample from the historical text data and the label data, determining the loss value of the initial sample through the target loss function, and determining the initial gradient of the initial named entity recognition model, wherein the initial gradient is used to represent the gradient of the parameters of the initial named entity recognition model relative to the loss value of the initial sample.

[0011] Optionally, adversarial training includes: obtaining an adversarial sample of the initial sample; determining the loss value of the adversarial sample through a target loss function, and determining the adversarial gradient of the initial named entity recognition model, wherein the adversarial gradient is used to represent the gradient of the parameters of the initial named entity recognition model relative to the loss value of the adversarial sample; obtaining the sum of all adversarial gradients of the initial named entity recognition model during each iterative training process to obtain a total adversarial gradient; performing gradient descent on the total adversarial gradient to update the parameters of the initial named entity recognition model.

[0012] Optionally, the adversarial sample is updated in the following manner: obtaining the loss value and adversarial gradient of the adversarial sample at each iterative training; determining the norm ball radius and single-step perturbation size when performing a projection operation on the adversarial sample; and updating the adversarial sample based on the loss value, adversarial gradient, norm ball radius, and single-step perturbation size of the adversarial sample.

[0013] Optionally, the parameters of the second optimizer are updated in the following manner: determining a first weight corresponding to the first learning rate, and determining a second weight corresponding to the second learning rate; determining a weight difference between the first weight and the second weight; and updating the parameters of the second optimizer based on the weight difference and a preset scaling factor, wherein the preset scaling factor is used to adjust the update amplitude of the initial named entity recognition model.

[0014] According to another aspect of an embodiment of the present application, a training method for a named entity recognition model is also provided, including: obtaining historical text data and label data corresponding to the historical text data; constructing an initial named entity recognition model, wherein the initial named entity recognition model includes an embedded text vector layer; iteratively training the initial named entity recognition model based on the historical text data and the label data, stopping the iteration after a preset number of iterations is met, and determining the adversarial gradient of the initial named entity recognition model during each iteration, wherein the adversarial gradient is used to represent the stability of the initial named entity recognition model in the face of potential adversarial disturbances; determining the optimal parameters of the initial named entity recognition model based on a combined optimizer to obtain the named entity recognition model, wherein the combined optimizer includes a first optimizer and a second optimizer, the first optimizer is used to update the parameters of the named entity recognition model through a first learning rate, and the second optimizer is used to update the parameters of the named entity recognition model through a second learning rate, and the first learning rate is less than the second learning rate.

[0015] According to another aspect of the embodiment of the present application, a named entity recognition device is also provided, including: a first acquisition module, used to acquire text data to be recognized; a recognition module, used to use a named entity recognition model to recognize the text data to be recognized, and obtain a named entity recognition result, wherein the named entity recognition model uses an embedded text vector layer as the underlying neural network, and the embedded text vector layer is used to strengthen the semantic representation result of the named entity recognition model. The parameters of the named entity recognition model are determined by adversarial gradient and a combination optimizer, and the combination optimizer includes a first optimizer and a second optimizer, the first optimizer is used to update the parameters of the named entity recognition model through a first learning rate, and the second optimizer is used to update the parameters of the named entity recognition model through a second learning rate, and the first learning rate is less than the second learning rate.

[0016] According to another aspect of the embodiment of the present application, a training device for a named entity recognition model is also provided, including: a second acquisition module, used to acquire historical text data and label data corresponding to the historical text data; a construction module, used to construct an initial named entity recognition model, wherein the initial named entity recognition model includes an embedded text vector layer; an iteration module, used to iteratively train the initial named entity recognition model based on the historical text data and the label data, stop the iteration after a preset number of iterations is met, and determine the adversarial gradient of the initial named entity recognition model during each iteration, wherein the adversarial gradient is used to represent the stability of the initial named entity recognition model in the face of potential adversarial disturbances; a determination module, used to determine the optimal parameters of the initial named entity recognition model based on a combined optimizer to obtain the named entity recognition model, wherein the combined optimizer includes a first optimizer and a second optimizer, the first optimizer is used to update the parameters of the named entity recognition model through a first learning rate, and the second optimizer is used to update the parameters of the named entity recognition model through a second learning rate, and the first learning rate is less than the second learning rate.

[0017] According to another aspect of an embodiment of the present application, there is also provided an electronic device, comprising: a memory and a processor, wherein the memory is used to store program instructions; the processor is connected to the memory and is used to execute the above-mentioned named entity recognition method or the above-mentioned named entity recognition model training method.

[0018] According to another aspect of an embodiment of the present application, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned named entity recognition method or the above-mentioned named entity recognition model training method by running the computer program.

[0019] According to another aspect of the embodiments of the present application, a computer program product is also provided, including computer instructions, which, when executed by a processor, implement the above-mentioned named entity recognition method or the above-mentioned named entity recognition model training method.

[0020] In an embodiment of the present application, by obtaining text data to be recognized; using a named entity recognition model to recognize the text data to be recognized, a named entity recognition result is obtained, wherein the named entity recognition model uses an embedded text vector layer as the underlying neural network, the embedded text vector layer is used to strengthen the semantic representation result of the named entity recognition model, the parameters of the named entity recognition model are determined by adversarial gradient and a combinatorial optimizer, the combinatorial optimizer includes a first optimizer and a second optimizer, the first optimizer is used to update the parameters of the named entity recognition model by a first learning rate, the second optimizer is used to update the parameters of the named entity recognition model by a second learning rate, the first learning rate is less than the second learning rate, and the purpose of improving the convergence speed and stability of the named entity recognition model through iterative training of the embedded text vector layer, the adversarial gradient and the combinatorial optimizer is achieved, thereby achieving a technical effect of more accurate named entity recognition, and then solving the technical problem that the named entity recognition model in the related art has poor recognition ability and generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0022] Figure 1 is a hardware structure diagram of a computer terminal for implementing a method for named entity recognition according to an embodiment of the present application;

[0023] Figure 2 is a flowchart of a method for named entity recognition according to an embodiment of the present application;

[0024] Figure 3 is a schematic diagram of a named entity recognition process according to an embodiment of the present application;

[0025] Figure 4 is a flowchart of a method for training a named entity recognition model according to an embodiment of the present application;

[0026] Figure 5 is a structural diagram of a named entity recognition device according to an embodiment of the present application;

[0027] Figure 6 It is a structural diagram of a training device for a named entity recognition model according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] First, some nouns or terms that appear in the process of explaining the embodiments of the present application are subject to the following explanations:

[0031] NLP (Natural Language Processing): An important direction in the fields of computer science and artificial intelligence, which studies various theories and methods that can enable effective communication between humans and computers using natural language.

[0032] NER (Named Entity Recognition): It is a subtask of information extraction in natural language processing, which aims to identify entities of interest in text, such as location, organization, and time. The identified entities can be used in various downstream applications, such as de-identification and information extraction systems based on patient records, and can also be used as features of machine learning systems for other natural language processing tasks. It is an important research area in natural language processing.

[0033] BiLSTM (Bi-directional Long Short-Term Memory) is composed of a forward LSTM (Long Short-Term Memory, a type of RNN (Recurrent Neural Network)) and a backward LSTM. Due to its design characteristics, it is very suitable for modeling time series data and is suitable for sequence labeling tasks with upper and lower relationships. Therefore, it is often used to model contextual information in NLP.

[0034] CRF (Conditional Random Field): It combines the characteristics of the maximum entropy model and the hidden Markov model. It is an undirected graph model that is suitable for discriminant models of prediction tasks, in which adjacent contextual information or states will affect the current prediction. It is often used to label or analyze sequence data, such as word segmentation, part-of-speech tagging, and named entity recognition, in sequence labeling tasks in natural language processing.

[0035] In the related technologies, named entity recognition methods based on manual sorting, preset rules, statistics or deep learning have the problems of poor recognition ability and poor model generalization ability. Taking e-commerce companies as an example, the product information of competing companies crawled on the Internet is often a narrative text. It is necessary to intelligently extract information from these Chinese texts and organize them into standardized documents that can be efficiently read and subsequently analyzed, so as to help company decision makers quickly obtain valuable information. If the manual sorting method is used, the time and labor costs required are too high. If rule-based named entity recognition is used (a system that relies on manual rules, combined with a named entity library, each rule is weighted, and then the type is judged by the match between the entity and the rule), most of the time, the rules often rely on specific language fields and text styles, and it is difficult to cover all language phenomena, and the effect is difficult to guarantee. In addition to rule-based named entity recognition, there are also statistical methods and deep learning-based methods. A typical statistical method is the conditional random field (CRF)-based named entity recognition method proposed by McCallum et al. of the University of Massachusetts. The conditional random field is actually a model of probability graphs. Its essence is to adaptively calculate feature weights to find the conditional probability distribution of the labeled entity. However, this method does not deeply learn the relationship between the front and back positions of the text sequence, and does not undergo deep learning of neural networks. In addition, a typical deep learning-based named entity recognition method is the bidirectional long short-term memory network (BiLSTM)-based method proposed by the University of London at the NLP International Summit. By using multiple independent BiLSTM units at the same input, the model regularization is used to promote the diversity between LSTM units, thereby reducing the parameters of the model. However, due to the bidirectional processing of this method, each output node contains both forward and reverse information, and it is easy to form overfitting when used alone.

[0036] In order to solve the above problems, the present invention provides a method for named entity recognition and a method for training a named entity recognition model. The method for training a named entity recognition model and the method for training a named entity recognition model can be run on Figure 1 In the computer terminal shown, the computer terminal is described below by taking a named entity recognition method as an example.

[0037] The named entity recognition method embodiment provided in the embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal for implementing a method for named entity recognition. Figure 1As shown, the computer terminal 10 may include one or more (102a, 102b, ..., 102n are used to illustrate) processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions connected via a wired and / or wireless network. In addition, it may also include: a display, a keyboard, a cursor control device, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, and a BUS bus. Those skilled in the art can understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0038] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10. As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0039] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the named entity recognition method in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the above-mentioned named entity recognition method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0040] The transmission module 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission module 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission module 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0041] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 .

[0042] It should be noted that, in some optional embodiments, the above Figure 1 The computer terminal shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. It should be noted that Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computer terminal described above.

[0043] In the above-mentioned operating environment, an embodiment of the present application provides an embodiment of a named entity recognition method. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0044] Figure 2 is a flowchart of a method for named entity recognition according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0045] Step S202, obtaining text data to be recognized.

[0046] Step S204, using a named entity recognition model to recognize the text data to be recognized to obtain a named entity recognition result, wherein the named entity recognition model uses an embedded text vector layer as the underlying neural network, the embedded text vector layer is used to strengthen the semantic representation result of the named entity recognition model, and the parameters of the named entity recognition model are determined by adversarial gradient and a combination optimizer, the combination optimizer includes a first optimizer and a second optimizer, the first optimizer is used to update the parameters of the named entity recognition model through a first learning rate, and the second optimizer is used to update the parameters of the named entity recognition model through a second learning rate, and the first learning rate is less than the second learning rate.

[0047] In the above steps S202 to S204, a pre-trained named entity recognition model can be used to identify the text data that needs to be extracted in step S202, so as to obtain the named entity recognition result corresponding to the text data. Among them, the named entity recognition model uses an embedded text vector layer as the underlying neural network to convert the text data into a vector form to enhance the named entity recognition model's ability to understand the text data. In the training process of the named entity recognition model, the adversarial gradient is introduced so that the named entity recognition model can learn how to remain stable in the face of potential adversarial perturbations, which helps to improve the robustness of the named entity recognition model. Finally, a combined optimizer updates the parameters of the named entity recognition model, and the combined optimizer includes: a LookAhead optimizer (i.e., the first optimizer mentioned above) and a RAdam optimizer (i.e., the second optimizer mentioned above); wherein the LookAhead optimizer uses a smaller learning rate for parameter update, while the RAdam optimizer uses a larger learning rate for update, thereby balancing the training speed of the named entity recognition model.

[0048] Through the above steps S202 to S204, the purpose of improving the convergence speed and stability of the named entity recognition model through iterative training of the embedded text vector layer, adversarial gradient and combined optimizer is achieved, thereby achieving the technical effect of more accurate named entity recognition, and further solving the technical problem that the named entity recognition model in the related art has poor recognition ability and generalization ability. The following is a detailed description.

[0049] Optionally, the named entity recognition model is determined in the following manner: obtaining historical text data and label data corresponding to the historical text data; constructing an initial named entity recognition model, wherein the initial named entity recognition model includes an embedded text vector layer; iteratively training the initial named entity recognition model based on the historical text data and the label data, stopping the iteration after a preset number of iterations is met, and determining the adversarial gradient of the initial named entity recognition model during each iteration, wherein the adversarial gradient is used to represent the stability of the initial named entity recognition model in the face of potential adversarial disturbances; determining the optimal parameters of the initial named entity recognition model based on the combined optimizer to obtain the named entity recognition model.

[0050] Figure 3 is a schematic diagram of a named entity recognition process provided according to an embodiment of the present application. Figure 3 The training process of the above named entity recognition model is analyzed. Figure 3As shown, first, the historical text data is preprocessed, for example, it may include setting the vocabulary corresponding to the text data, mapping dictionary, setting the maximum sequence length of the model to truncate or fill the text data. Secondly, based on the pre-trained model of Hugging Face, a multi-layer neural network learning model (i.e., the above-mentioned initial named entity recognition model) is built, which is specifically reflected in setting the model parameters of the initial named entity recognition model, and sequentially building the embedded text vector layer, the bidirectional long short-term memory network BiLSTM layer, the random inactivation regularization Dropout layer, the fully connected layer, and the conditional random field CRF layer. Then, the initial named entity recognition model is trained, the PGD (Projected Gradient Descent) adversarial sample is added, and the combined optimizer of the LookAhead optimizer and the RAdam optimizer is used to obtain the optimal model of hybrid learning. Specifically: the historical text data and the historical label data are divided into multiple batches for processing. In each batch, the target loss function of the initial named entity recognition model is set. After multiple iterative training, the strongest first-order adversarial sample perturbation attack within a limited range is used to obtain the adversarial gradient. The LookAhead optimizer in the combined optimizer is used to achieve the combined effect of fast average moving and the slower exponential moving average is obtained through the RAdam optimizer to train the globally optimal model, that is, the named entity recognition model mentioned above.

[0051] Furthermore, the trained named entity recognition model is used to perform named entity recognition on the text data that needs to be recognized, and the obtained named entity recognition results are sorted to meet the requirements of related natural language processing tasks such as information extraction to ensure the availability of the recognition results. Through the above process, after the recognition is completed, a hybrid learning Chinese text named entity recognition result with higher accuracy and better effect can be obtained.

[0052] Optionally, after obtaining the historical text data and the label data corresponding to the historical text data, the method further includes: obtaining a mapping dictionary containing the label data and integer indexes, wherein the label data and the integer indexes in the mapping dictionary correspond one to one; and converting the label data into corresponding integer indexes according to the mapping dictionary.

[0053] In the embodiments of the present application, Figure 3 The process of preprocessing historical text data can be as follows:

[0054] Define the vocabulary VOCAB of the historical text data that needs to be annotated, create a mapping dictionary that corresponds one-to-one between label data and integer index, and store the historical text data and label data in the corresponding list, so that in the training process of the initial named entity recognition model, the label data can be converted into the corresponding integer index through the mapping dictionary. Subsequently, set the maximum sequence length processed by the initial named entity recognition model. If the length of the historical text data exceeds the set maximum length, truncate the sentence and retain the front part; if the length of the historical text data does not exceed the maximum length, retain the entire sentence, and at the same time fill the current batch of historical text data according to the maximum sequence length, fill the historical text data and label data to the same length to ensure the subsequent processing of the initial named entity recognition model.

[0055] Optionally, constructing an initial named entity recognition model includes: determining parameters of the initial named entity recognition model; determining a multi-layer neural network structure of the initial named entity recognition model, wherein the multi-layer neural network structure includes: an embedded text vector layer, a bidirectional long short-term memory network layer, a random dropout regularization layer, a fully connected layer, and a conditional random field layer.

[0056] In the embodiments of the present application, Figure 3 Based on the pre-trained model of Hugging Face, build a multi-layer neural network learning model (that is, the initial named entity recognition model mentioned above). The specific steps can be as follows:

[0057] 1. Set parameters: Set the relevant parameters of the entire deep learning model, including the hidden layer dimension, embedded text vector dimension, etc.

[0058] 2. Build an embedded text vector layer: For example, use Hugging Face's Transformer-based pre-trained model "chinese-macbert-base" as the underlying embedded text vector layer for deep language representation learning to achieve encoding of Chinese input text sequences.

[0059] 3. Construct a bidirectional long short-term memory network BiLSTM layer: define a two-layer bidirectional long short-term memory neural network BiLSTM layer to capture the long-term dependencies (i.e. long-distance semantic information) in historical text data. Each LSTM layer contains hidden states in two directions: forward propagation and backward propagation, which together constitute the final output of each time step of the embedded text vector of the previous step, thereby obtaining a more accurate encoding of the input historical text data based on the pre-trained model.

[0060] 4. Construct a random inactivation regularized Dropout layer: Set the probability of Dropout for the initial named entity recognition model, and use this regularization technology to randomly discard some neurons of the embedded text vector to reduce the risk of overfitting and improve the generalization ability of the initial named entity recognition model.

[0061] 5. Construct a fully connected layer: that is, a linear layer, which maps the hidden state of the sample data output by the Dropout layer after the BiLSTM layer to a dimensional space with the same size as the label set, so as to generate a probability distribution of possible label data at each position and predict the annotation of historical text data.

[0062] 6. Build the conditional random field layer: The conditional random field (CRF) layer is a statistically based probabilistic graph model that is used to predict label data for the embedded text vectors of the historical text data output by the fully connected layer.

[0063] Optionally, the iterative training includes initialization training and adversarial training, wherein the initialization training includes: determining the target loss function of the initial named entity recognition model during the iterative training process, wherein the target loss function is used to represent the difference between the recognition result of the initial named entity recognition model and the label data; determining the initial sample from the historical text data and the label data, determining the loss value of the initial sample through the target loss function, and determining the initial gradient of the initial named entity recognition model, wherein the initial gradient is used to represent the gradient of the parameters of the initial named entity recognition model relative to the loss value of the initial sample.

[0064] In the embodiments of the present application, Figure 3 The learning and training of the initial named entity recognition model can be divided into two processes: initialization training and iterative training. The specific steps of initialization training can be as follows:

[0065] The historical text data is divided into multiple batches for processing. In each batch, the target loss function of the initial named entity recognition model is set, wherein a batch of historical text data can be represented as an initial sample. Subsequently, the loss value of the initial sample is calculated by the target loss function to reflect the difference between the prediction result of the initial named entity recognition model (i.e., the above-mentioned recognition result) and the real data (such as the above-mentioned label data); then the gradient of the loss value of the initial named entity recognition model relative to the initial sample is calculated, for example, the partial derivative of the loss value of the initial sample with respect to the model parameters is calculated by automatic differentiation technology to obtain the initial gradient as the basis for adjusting the parameters of the initial named entity recognition model. Among them, the target loss function can be a negative log-likelihood loss function, as follows:

[0066]

[0067] Among them, N represents the size of a batch, x i represents the input sequence of the i-th initial sample, y i represents the corresponding true label data sequence, θ represents the parameters of the named entity recognition model, and P(y|x;θ) represents the conditional probability of the label data y given the input sequence x and the parameters θ of the named entity recognition model.

[0068] Optionally, adversarial training includes: obtaining an adversarial sample of the initial sample; determining the loss value of the adversarial sample through a target loss function, and determining the adversarial gradient of the initial named entity recognition model, wherein the adversarial gradient is used to represent the gradient of the parameters of the initial named entity recognition model relative to the loss value of the adversarial sample; obtaining the sum of all adversarial gradients of the initial named entity recognition model during each iterative training process to obtain a total adversarial gradient; performing gradient descent on the total adversarial gradient to update the parameters of the initial named entity recognition model.

[0069] Optionally, the adversarial sample is updated in the following manner: obtaining the loss value and adversarial gradient of the adversarial sample at each iterative training; determining the norm ball radius and single-step perturbation size when performing a projection operation on the adversarial sample; and updating the adversarial sample based on the loss value, adversarial gradient, norm ball radius, and single-step perturbation size of the adversarial sample.

[0070] In the embodiment of the present application, the specific steps of adversarial training may be as follows:

[0071] 1. Generate adversarial samples corresponding to the initial samples, such as using the PGD algorithm to generate adversarial samples that are visually similar to the initial samples but misleading in the decision of the initial named entity recognition model.

[0072] 2. Set the perturbation accumulation step number of the adversarial sample to K steps, and then perform K-step iterative training. For the first step, save the word embedding weights and initial gradients of the initial sample that has not been attacked. For other steps (such as step t, t<=K-1), calculate the loss value of the adversarial sample through the target loss function, and then calculate the gradient of the initial named entity recognition model relative to the loss value of the adversarial sample. For example, the partial derivative of the loss value of the adversarial sample with respect to the model parameters is calculated through automatic differentiation technology to obtain the adversarial gradient, which is also saved as a backup as a basis for adjusting the parameters of the initial named entity recognition model.

[0073] It should be noted that, in the non-final iteration step, that is, non-K-1 step, it is necessary to clearly know the gradient of the current initial named entity recognition model; in the final iteration step, that is, K-1 step, the backup gradient of the initial named entity recognition model is restored.

[0074] 3. After completing the adversarial sample attack, restore the original word embedding weights to maintain the original semantic representation ability of the initial named entity recognition model.

[0075] 4. By accumulating all adversarial gradients during K iterative training processes, the total adversarial gradient is obtained, and then gradient descent is performed on it to update the parameters of the original named entity recognition model, so that the original named entity recognition model has stronger stability when facing potential adversarial perturbations.

[0076] At the same time, in each iterative training process, the adversarial sample can be updated by the following formula:

[0077]

[0078] Among them, x t+1 represents the adversarial sample at step t+1, x t represents the adversarial sample of step t. For the first step, x t Represents the initial sample, Clip x,ε Represents the projection operation. If the input is outside the norm ball, the input is projected onto the norm ball with x as the center and ∈ as the radius to ensure that the perturbation range of the adversarial sample during the iteration does not exceed the limit ∈. α represents the single-step perturbation size, and sign represents the sign function with a value of 1 or -1. Represents taking partial derivatives, L represents the target loss function, θ represents the relevant parameters of the target loss function, x represents the embedded vector of the adversarial sample input sequence, and y represents the text vector of the predicted label data sequence.

[0079] Optionally, the parameters of the second optimizer are updated in the following manner: determining a first weight corresponding to the first learning rate, and determining a second weight corresponding to the second learning rate; determining a weight difference between the first weight and the second weight; and updating the parameters of the second optimizer based on the weight difference and a preset scaling factor, wherein the preset scaling factor is used to adjust the update amplitude of the initial named entity recognition model.

[0080] In an embodiment of the present application, when searching for the optimal parameters of the initial named entity recognition model based on the adversarial gradient, a combined optimizer of LookAhead and RAdam can be used for auxiliary exploration. For example, a smaller learning rate (i.e., the first learning rate mentioned above) is used by the LookAhead optimizer to perform "slow exploration", and then a larger learning rate (i.e., the second learning rate mentioned above) is used by the internal RAdam optimizer to perform multiple batches of "fast exploration", so as to achieve the combined effect of the fast moving average of the internal RAdam optimizer and the slower exponential moving average of the LookAhead optimizer, so as to train the initial named entity recognition model and obtain the globally optimal named entity recognition model.

[0081] Among them, the parameter update method of the RAdam optimizer can be as follows: determine the copy of the parameter weights that the LookAhead optimizer retains additionally during the "slow exploration" process, that is, the above-mentioned first weight; then determine the parameter weights of the RAdam optimizer during the "fast exploration" process, that is, the above-mentioned second weight; by calculating the weight difference between the first weight and the second weight, and then multiplying the weight difference with a preset scaling factor, the updated parameters of the RAdam optimizer can be obtained, so as to carry out the next round of fast and slow combination exploration.

[0082] The following is an explanation of the above named entity recognition method with specific examples:

[0083] 1. Preprocess the original product information text data.

[0084] First, analyze the original text data (such as any e-commerce product information text data), define the annotated Chinese named entities as product category, product brand, product model, and other product descriptions, set the label data to HCCX, HPPX, XH, MISC, and define the vocabulary as follows:

[0085] VOCAB=(' <pad> ’,‘ <sep>','O','B-MISC','B-HPPX','B-XH','B-HCCX','I-MISC','I-HPPX','I-XH','I-HCCX')

[0086] According to this vocabulary, the label data is mapped to integer indices to form a mapping dictionary for this information extraction and annotation task, and the maximum sequence length processed by the initial named entity recognition model is set to 254.

[0087] 2. Build the model and construct a hybrid deep learning model, which is the initial named entity recognition model mentioned above.

[0088] Set the embedded text vector dimension to 768, the hidden layer dimension of the initial named entity recognition model to 256, set the underlying representation model to "chinese-macbert-base", set the input data dimension of the BiLSTM layer to the embedded text vector dimension of 768, the Dropout probability to 0.1, the dimension of the fully connected layer to the hidden layer dimension of 256, and use the CRF layer as the output layer of the initial named entity recognition model of the overall hybrid learning.

[0089] 3. Initial named entity recognition model training.

[0090] TESLA V100 GPU is used for training. The training generation is set to 30, the size of each batch is 64, 10% of the training data is used as warm-up, the number of steps K for iterative search for the strongest first-order adversarial sample is 3, the perturbation range limit parameter ∈ is 1 during the iteration, the single-step perturbation size is 0.3, and the combined optimizer of the "slow exploration" LookAhead optimizer and the "fast exploration" RAdam optimizer is used as a convergence tool for finding the optimal model. The optimal model is found according to the loss function and accuracy of the verification data. The model with the smallest loss function and the largest accuracy on the verification data is used as the final named entity recognition model training.

[0091] 4. Named entity inference and form information extraction results.

[0092] By loading the named entity recognition model obtained through the above training, inputting the original e-commerce product information and performing Chinese named entity labeling, key information such as product type, product brand, product model, and other product descriptions can be identified, and the inferred and labeled results can be automatically and standardized according to the labels to form intelligent and standardized product sorting data.

[0093] In the embodiment of the present application, when statistics and neural networks are mixed for learning, multiple iterations within a limited range are performed to find the strongest adversarial samples of the first order, increase disturbances, and improve the generalization ability of the initial named entity recognition model. When looking for the optimal parameters of the model according to the adversarial gradient, the LookAhead optimizer retains an additional copy of the parameter weights for "slow exploration", while the internal RAdam optimizer performs "fast exploration", achieving the combined effect of the fast moving average of the internal optimizer RAdam and the slower exponential moving average of the LookAhead optimizer, thereby obtaining the global optimal improved optimal named entity recognition model based on hybrid learning. When the input Chinese text sequence is semantically represented, the pre-trained model "chinese-macbert-base" based on the Transformer architecture of Hugging Face is used. Compared with the traditional text vector representation method (such as one-hot, TF-IDF, etc.), the underlying representation also has the performance of neural network deep learning, and the upper-level BiLSTM-based neural network and CRF-based network together constitute a hybrid learning model, which strengthens the effect of semantic representation, thereby improving the overall performance of annotation recognition.

[0094] Figure 4 is a flowchart of a method for training a named entity recognition model according to an embodiment of the present application, such as Figure 4 As shown, the method comprises the following steps:

[0095] Step S402: Acquire historical text data and label data corresponding to the historical text data.

[0096] Step S404, constructing an initial named entity recognition model, wherein the initial named entity recognition model includes an embedded text vector layer.

[0097] Step S406, iteratively train the initial named entity recognition model based on the historical text data and label data, stop the iteration after meeting the preset number of iterations, and determine the adversarial gradient of the initial named entity recognition model during each iteration, wherein the adversarial gradient is used to represent the stability of the initial named entity recognition model in the face of potential adversarial disturbances.

[0098] Step S408, determine the optimal parameters of the initial named entity recognition model based on the combined optimizer to obtain the named entity recognition model, wherein the combined optimizer includes a first optimizer and a second optimizer, the first optimizer is used to update the parameters of the named entity recognition model through a first learning rate, and the second optimizer is used to update the parameters of the named entity recognition model through a second learning rate, and the first learning rate is less than the second learning rate.

[0099] It should be noted that the training method of the named entity recognition model is similar to the above-mentioned named entity recognition method, so the relevant explanations in the above-mentioned named entity recognition method are also applicable to the training method of the named entity recognition model, which will not be repeated here.

[0100] According to an embodiment of the present application, a named entity recognition device is provided. It should be noted that the named entity recognition device of the embodiment of the present application can be used to execute the method for named entity recognition provided by the embodiment of the present application. The following introduces the named entity recognition device provided by the embodiment of the present application.

[0101] Figure 5 is a structural diagram of a named entity recognition device provided according to an embodiment of the present application. Figure 5 As shown, the device comprises:

[0102] A first acquisition module 50 is used to acquire text data to be recognized;

[0103] The recognition module 52 is used to use a named entity recognition model to recognize text data to be recognized and obtain a named entity recognition result, wherein the named entity recognition model uses an embedded text vector layer as an underlying neural network, and the embedded text vector layer is used to strengthen the semantic representation result of the named entity recognition model. The parameters of the named entity recognition model are determined by adversarial gradient and a combination optimizer. The combination optimizer includes a first optimizer and a second optimizer. The first optimizer is used to update the parameters of the named entity recognition model through a first learning rate, and the second optimizer is used to update the parameters of the named entity recognition model through a second learning rate, and the first learning rate is less than the second learning rate.

[0104] In the named entity recognition device provided in the embodiment of the present application, a training module 54 is also included. The training module includes a second acquisition module, a construction module, an iteration module and a determination module, that is, each module in the training device of the named entity recognition model. Figure 6 The training device of the named entity recognition model is used as an example to illustrate.

[0105] Figure 6 is a structural diagram of a training device for a named entity recognition model provided in an embodiment of the present application. Figure 6 As shown, the device comprises:

[0106] A second acquisition module 60, used to acquire historical text data and label data corresponding to the historical text data;

[0107] A construction module 62 is used to construct an initial named entity recognition model, wherein the initial named entity recognition model includes an embedded text vector layer;

[0108] An iteration module 64 is used to iteratively train the initial named entity recognition model based on the historical text data and the label data, stop the iteration after a preset number of iterations are met, and determine the adversarial gradient of the initial named entity recognition model in each iteration process, wherein the adversarial gradient is used to represent the stability of the initial named entity recognition model in the face of potential adversarial disturbances;

[0109] Determination module 66 is used to determine the optimal parameters of the initial named entity recognition model based on the combined optimizer to obtain the named entity recognition model, wherein the combined optimizer includes a first optimizer and a second optimizer, the first optimizer is used to update the parameters of the named entity recognition model through a first learning rate, and the second optimizer is used to update the parameters of the named entity recognition model through a second learning rate, and the first learning rate is less than the second learning rate.

[0110] Through the second acquisition module 60, construction module 62, iteration module 64 and determination module 66 in the training device of the above-mentioned named entity recognition model, the purpose of improving the convergence speed and stability of the named entity recognition model through iterative training of the embedded text vector layer, adversarial gradient and combination optimizer is achieved, thereby achieving the technical effect of more accurate named entity recognition, and further solving the technical problem that the named entity recognition model in the related art has poor recognition ability and generalization ability.

[0111] In the training device of the named entity recognition model provided in the embodiment of the present application, the second acquisition module is also used to obtain a mapping dictionary containing label data and integer indexes, wherein the label data and integer indexes in the mapping dictionary correspond one to one; and the label data is converted into a corresponding integer index according to the mapping dictionary.

[0112] In the training device of the named entity recognition model provided in the embodiment of the present application, the construction module is also used to determine the parameters of the initial named entity recognition model; determine the multi-layer neural network structure of the initial named entity recognition model, wherein the multi-layer neural network structure includes: an embedded text vector layer, a bidirectional long short-term memory network layer, a random inactivation regularization layer, a fully connected layer, and a conditional random field layer.

[0113] In the training device of the named entity recognition model provided in the embodiment of the present application, the iteration module is also used to determine the target loss function of the initial named entity recognition model in the iterative training process, wherein the target loss function is used to represent the difference between the recognition result of the initial named entity recognition model and the label data; determine the initial sample from the historical text data and the label data, determine the loss value of the initial sample through the target loss function, and determine the initial gradient of the initial named entity recognition model, wherein the initial gradient is used to represent the gradient of the parameters of the initial named entity recognition model relative to the loss value of the initial sample.

[0114] In the training device of the named entity recognition model provided in the embodiment of the present application, the iteration module is also used to obtain an adversarial sample of the initial sample; determine the loss value of the adversarial sample through the target loss function, and determine the adversarial gradient of the initial named entity recognition model, wherein the adversarial gradient is used to represent the gradient of the parameters of the initial named entity recognition model relative to the loss value of the adversarial sample; obtain the sum of all adversarial gradients of the initial named entity recognition model in each iterative training process to obtain the total adversarial gradient; perform gradient descent on the total adversarial gradient to update the parameters of the initial named entity recognition model.

[0115] In the training device of the named entity recognition model provided in the embodiment of the present application, the iteration module is also used to obtain the loss value and adversarial gradient of the adversarial sample in each iterative training; determine the norm ball radius and single-step perturbation size when performing the projection operation on the adversarial sample; and update the adversarial sample according to the loss value, adversarial gradient, norm ball radius and single-step perturbation size of the adversarial sample.

[0116] In the training device of the named entity recognition model provided in the embodiment of the present application, the determination module is also used to determine a first weight corresponding to a first learning rate, and to determine a second weight corresponding to a second learning rate; determine the weight difference between the first weight and the second weight; and update the parameters of the second optimizer based on the weight difference and a preset scaling factor, wherein the preset scaling factor is used to adjust the update amplitude of the initial named entity recognition model.

[0117] An embodiment of the present application also provides an electronic device, including: a memory and a processor, wherein the memory is used to store program instructions; the processor is connected to the memory and is used to execute the above-mentioned named entity recognition method or the above-mentioned named entity recognition model training method.

[0118] It should be noted that the above electronic equipment is used to execute Figure 2 The named entity recognition method shown, therefore the relevant explanations and instructions in the above-mentioned named entity recognition method are also applicable to the electronic device and will not be repeated here.

[0119] An embodiment of the present application also provides a non-volatile storage medium, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned named entity recognition method or the above-mentioned named entity recognition model training method by running the computer program.

[0120] It should be noted that the above non-volatile storage medium is used to execute Figure 2 The named entity recognition method shown, therefore the relevant explanations in the above-mentioned named entity recognition method are also applicable to the non-volatile storage medium, and will not be repeated here.

[0121] An embodiment of the present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the above-mentioned named entity recognition method or the above-mentioned named entity recognition model training method.

[0122] It should be noted that the above-mentioned computer program product is used to execute Figure 2 The named entity recognition method shown, therefore the relevant explanations and instructions in the above-mentioned named entity recognition method are also applicable to the computer program product and will not be repeated here.

[0123] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0124] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0125] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0126] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0127] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0128] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0129] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.< / sep> < / pad>

Claims

1. A method for named entity recognition, characterized in that: include: Obtain text data to be recognized; A named entity recognition model is used to recognize the text data to be recognized to obtain a named entity recognition result, wherein the named entity recognition model uses an embedded text vector layer as an underlying neural network, the embedded text vector layer is used to strengthen the semantic representation result of the named entity recognition model, and the parameters of the named entity recognition model are determined by adversarial gradient and a combination optimizer, the combination optimizer includes a first optimizer and a second optimizer, the first optimizer is used to update the parameters of the named entity recognition model through a first learning rate, and the second optimizer is used to update the parameters of the named entity recognition model through a second learning rate, and the first learning rate is less than the second learning rate.

2. The method according to claim 1, characterized in that The named entity recognition model is determined in the following way: Acquire historical text data and label data corresponding to the historical text data; Constructing an initial named entity recognition model, wherein the initial named entity recognition model includes the embedded text vector layer; Iteratively training the initial named entity recognition model according to the historical text data and the label data, stopping the iteration after a preset number of iterations is met, and determining the adversarial gradient of the initial named entity recognition model in each iteration process, wherein the adversarial gradient is used to represent the stability of the initial named entity recognition model in the face of potential adversarial disturbances; The optimal parameters of the initial named entity recognition model are determined according to the combined optimizer to obtain the named entity recognition model.

3. The method according to claim 2, characterized in that After acquiring the historical text data and the label data corresponding to the historical text data, the method further includes: Obtain a mapping dictionary containing the label data and the integer index, wherein the label data in the mapping dictionary and the integer index correspond one to one; The label data is converted into corresponding integer indices according to the mapping dictionary.

4. The method according to claim 2, characterized in that: Build an initial named entity recognition model, including: Determining parameters of the initial named entity recognition model; Determine a multi-layer neural network structure of the initial named entity recognition model, wherein the multi-layer neural network structure includes: an embedded text vector layer, a bidirectional long short-term memory network layer, a random dropout regularization layer, a fully connected layer, and a conditional random field layer.

5. The method according to claim 2, characterized in that: The iterative training includes initialization training and adversarial training, wherein the initialization training includes: Determining a target loss function of the initial named entity recognition model during an iterative training process, wherein the target loss function is used to represent a difference between a recognition result of the initial named entity recognition model and the label data; An initial sample is determined from the historical text data and the label data, a loss value of the initial sample is determined by the target loss function, and an initial gradient of the initial named entity recognition model is determined, wherein the initial gradient is used to represent the gradient of the parameters of the initial named entity recognition model relative to the loss value of the initial sample.

6. The method according to claim 5, characterized in that The adversarial training comprises: Obtaining an adversarial sample of the initial sample; Determine the loss value of the adversarial sample by using the target loss function, and determine the adversarial gradient of the initial named entity recognition model, wherein the adversarial gradient is used to represent the gradient of the parameters of the initial named entity recognition model relative to the loss value of the adversarial sample; Obtaining the sum of all adversarial gradients of the initial named entity recognition model during each iterative training process to obtain a total adversarial gradient; Performing gradient descent on the total adversarial gradient to update parameters of the initial named entity recognition model.

7. The method according to claim 6, characterized in that The adversarial sample is updated in the following way: Obtaining the loss value and adversarial gradient of the adversarial sample during each iterative training; Determining a norm ball radius and a single-step perturbation size when performing a projection operation on the adversarial sample; The adversarial sample is updated according to the loss value of the adversarial sample, the adversarial gradient, the norm ball radius, and the single-step perturbation size.

8. The method according to claim 2, characterized in that: The parameters of the second optimizer are updated in the following way: Determine a first weight corresponding to the first learning rate, and determine a second weight corresponding to the second learning rate; determining a weight difference between the first weight and the second weight; The parameters of the second optimizer are updated according to the weight difference and a preset scaling factor, wherein the preset scaling factor is used to adjust the update amplitude of the initial named entity recognition model.

9. A method for training a named entity recognition model, characterized in that: include: Acquire historical text data and label data corresponding to the historical text data; Constructing an initial named entity recognition model, wherein the initial named entity recognition model includes an embedded text vector layer; Iteratively training the initial named entity recognition model according to the historical text data and the label data, stopping the iteration after a preset number of iterations is met, and determining the adversarial gradient of the initial named entity recognition model in each iteration process, wherein the adversarial gradient is used to represent the stability of the initial named entity recognition model in the face of potential adversarial disturbances; The optimal parameters of the initial named entity recognition model are determined according to the combined optimizer to obtain the named entity recognition model, wherein the combined optimizer includes a first optimizer and a second optimizer, the first optimizer is used to update the parameters of the named entity recognition model through a first learning rate, and the second optimizer is used to update the parameters of the named entity recognition model through a second learning rate, and the first learning rate is less than the second learning rate.

10. A named entity recognition device, characterized in that: include: A first acquisition module, used to acquire text data to be recognized; A recognition module is used to use a named entity recognition model to recognize the text data to be recognized and obtain a named entity recognition result, wherein the named entity recognition model uses an embedded text vector layer as an underlying neural network, and the embedded text vector layer is used to strengthen the semantic representation result of the named entity recognition model. The parameters of the named entity recognition model are determined by adversarial gradient and a combination optimizer, and the combination optimizer includes a first optimizer and a second optimizer, the first optimizer is used to update the parameters of the named entity recognition model through a first learning rate, and the second optimizer is used to update the parameters of the named entity recognition model through a second learning rate, and the first learning rate is less than the second learning rate.

11. A training device for a named entity recognition model, characterized in that: include: A second acquisition module, used to acquire historical text data and label data corresponding to the historical text data; A construction module, used to construct an initial named entity recognition model, wherein the initial named entity recognition model includes an embedded text vector layer; An iteration module, used for iteratively training the initial named entity recognition model according to the historical text data and the label data, stopping iteration after a preset number of iterations is met, and determining an adversarial gradient of the initial named entity recognition model in each iteration process, wherein the adversarial gradient is used to represent the stability of the initial named entity recognition model in the face of potential adversarial disturbances; A determination module is used to determine the optimal parameters of the initial named entity recognition model based on a combination optimizer to obtain a named entity recognition model, wherein the combination optimizer includes a first optimizer and a second optimizer, the first optimizer is used to update the parameters of the named entity recognition model through a first learning rate, and the second optimizer is used to update the parameters of the named entity recognition model through a second learning rate, and the first learning rate is less than the second learning rate.

12. An electronic device, characterized in that: include: A memory and a processor, wherein The memory is used to store program instructions; The processor is connected to the memory and is used to execute the named entity recognition method described in any one of claims 1 to 8 or the training method of the named entity recognition model in claim 9.

13. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the named entity recognition method described in any one of claims 1 to 8 or the named entity recognition model training method in claim 9 by running the computer program.

14. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the method for named entity recognition described in any one of claims 1 to 8 or the method for training a named entity recognition model in claim 9 is implemented.

Images