Named entity recognition method, device, equipment and storage medium

Abstract

The present application belongs to the field of computer technology, and specifically relates to a named entity recognition method, apparatus, device and storage medium for improving the efficiency of named entity recognition. The named entity recognition method includes: obtaining a target text to be subjected to named entity recognition, obtaining a word embedding vector sequence corresponding to the target text; inputting the word embedding vector sequence into a named entity recognition model, and obtaining the output of the named entity recognition model as a target category label sequence corresponding to the named entity in the target text; the named entity recognition model is used to extract target feature vectors corresponding to each word embedding vector contained in the word embedding vector sequence in parallel, and to classify the target feature vectors in parallel; according to the target category label sequence, a named entity recognition result of the target text is obtained. The present application can greatly improve the efficiency of named entity recognition.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and specifically relates to a method, apparatus, device and storage medium for named entity recognition. Background Art

[0002] Natural language processing technology can better enable interaction between humans and computers. Named entity recognition (NER) is a key component of NLP. Named entity recognition refers to the process of identifying meaningful names or symbols within text. Named entities primarily include names of people, places, organizations, dates, proper nouns, and so on. More accurate recognition of named entities within text helps computers better understand the semantics of language and perform tasks, thereby enhancing the human-computer interaction experience.

[0003] At present, the commonly used named entity recognition scheme is: input the target text to be recognized by the named entity into the Bert model to obtain the corresponding word embedding vector sequence, input the word embedding vector sequence into the bidirectional long short-term memory network (BiLSTM) to extract the context features of the target text, and then input the context features of the target text into the conditional random field (CRF) to obtain the category labels corresponding to the named entities in the target text, integrate the category labels, and obtain the named entity recognition results of the target text.

[0004] The inventors have found that although the named entity recognition method described above can effectively extract the named entities in the text, it also has the problem of low efficiency. Summary of the Invention

[0005] In order to solve the above-mentioned problems in the prior art, that is, to improve the efficiency of named entity recognition, the present application provides a method, apparatus, device and storage medium for named entity recognition.

[0006] In a first aspect, the present application provides a method for named entity recognition, comprising:

[0007] Obtain the target text for named entity recognition;

[0008] Get the word embedding vector sequence corresponding to the target text;

[0009] The word embedding vector sequence is input into the named entity recognition model. The output of the named entity recognition model is a target category label sequence corresponding to the named entities in the target text. The named entity recognition model is used to extract the target feature vector corresponding to each word embedding vector contained in the word embedding vector sequence in parallel, and classify the target feature vectors in parallel.

[0010] According to the target category label sequence, the named entity recognition result of the target text is obtained.

[0011] In one possible implementation, the named entity recognition model includes: a fully connected layer and a classification processing layer, and the word embedding vector sequence is input into the named entity recognition model, and the output of the named entity recognition model is a target category label sequence corresponding to the named entities in the target text, including: passing the word embedding vector sequence through the fully connected layer, and extracting the target feature vectors corresponding to each word embedding vector contained in the word embedding vector sequence in parallel; passing the target feature vector through the classification processing layer, and classifying the target feature vector in parallel to obtain the target category label sequence corresponding to the named entities in the target text.

[0012] In one possible implementation, obtaining a word embedding vector sequence corresponding to the target text includes: obtaining a word embedding vector sequence corresponding to the target text through a RoBERTa model including a preset number of attention mechanism layers, the RoBERTa model is used to convert the text contained in the target text into a corresponding numerical sequence through a preset dictionary, and obtaining a word embedding vector sequence corresponding to the target text based on the numerical sequence and the pre-trained word vector.

[0013] In one possible implementation, a named entity recognition result of a target text is obtained based on a target category label sequence, including: integrating the target category label sequence according to a preset category label format to obtain each target category label after integration; extracting text in the target text corresponding to the position of each target category label after integration to obtain a named entity recognition result of the target text.

[0014] In one possible implementation, a named entity recognition model is obtained in the following manner: obtaining a training set, the training set containing multiple sample pairs, the sample pairs containing sample text and a sequence of annotated category labels corresponding to the sample text, the category labels being used to identify the named entities contained in the text; training the named entity recognition model using the training set, obtaining a sequence of word embedding vectors corresponding to the sample text, extracting target feature vectors corresponding to each word embedding vector contained in the word embedding vector sequence in parallel, and classifying the target feature vectors in parallel to obtain a sequence of target category labels corresponding to the named entities in the sample text, and obtaining a loss function value of the target category label sequence relative to the annotated category label sequence; adjusting the parameters of the named entity recognition model based on the loss function value to obtain a trained named entity recognition model.

[0015] In one possible implementation, the target feature vector corresponding to each word embedding vector contained in the word embedding vector sequence is extracted in parallel, including: for each word embedding vector, the following steps are performed in parallel: the word embedding vector is input into a fully connected layer containing a first preset number of neurons for fully connected processing to obtain a corresponding first feature vector; the first feature vector is input into a fully connected layer containing a second preset number of neurons for fully connected processing to obtain a corresponding second feature vector; the second feature vector is input into a fully connected layer containing a third preset number of neurons for fully connected processing to obtain a target feature vector corresponding to the word embedding vector, and the third preset number of neurons is the same as the number of preset category labels.

[0016] In one possible implementation, the target feature vectors are classified in parallel to obtain a target category label sequence corresponding to the named entities in the sample text, including: for each target feature vector, the following steps are performed in parallel: the target feature vector is classified to obtain the probability that the target feature vector corresponds to each preset category label; and the preset category label corresponding to the maximum probability among the probabilities is determined as the target category label corresponding to the named entity in the sample text.

[0017] In a second aspect, the present application provides a method for training a named entity recognition model, comprising:

[0018] Obtain a training set, which contains multiple sample pairs. Each sample pair contains a sample text and a sequence of annotated category labels corresponding to the sample text. The category labels are used to identify named entities contained in the text.

[0019] The named entity recognition model is trained using the training set to obtain a sequence of word embedding vectors corresponding to the sample text. The target feature vectors corresponding to each word embedding vector in the word embedding vector sequence are extracted in parallel, and the target feature vectors are classified in parallel to obtain a sequence of target category labels corresponding to the named entities in the sample text. The loss function value of the target category label sequence relative to the annotated category label sequence is obtained.

[0020] According to the loss function value, the parameters of the named entity recognition model are adjusted to obtain the trained named entity recognition model.

[0021] In one possible implementation, the target feature vector corresponding to each word embedding vector contained in the word embedding vector sequence is extracted in parallel, including: for each word embedding vector, the following steps are performed in parallel: the word embedding vector is input into a fully connected layer containing a first preset number of neurons for fully connected processing to obtain a corresponding first feature vector; the first feature vector is input into a fully connected layer containing a second preset number of neurons for fully connected processing to obtain a corresponding second feature vector; the second feature vector is input into a fully connected layer containing a third preset number of neurons for fully connected processing to obtain a target feature vector corresponding to the word embedding vector, and the third preset number of neurons is the same as the number of preset category labels.

[0022] In one possible implementation, the target feature vectors are classified in parallel to obtain a target category label sequence corresponding to the named entities in the sample text, including: for each target feature vector, the following steps are performed in parallel: the target feature vector is classified to obtain the probability that the target feature vector corresponds to each preset category label; and the preset category label corresponding to the maximum probability among the probabilities is determined as the target category label corresponding to the named entity in the sample text.

[0023] In a third aspect, the present application provides a named entity recognition device, comprising:

[0024] A first acquisition module is used to acquire a target text to be subjected to named entity recognition;

[0025] The second acquisition module is used to obtain the word embedding vector sequence corresponding to the target text;

[0026] A processing module is used to input the word embedding vector sequence into the named entity recognition model, and the output of the named entity recognition model is a target category label sequence corresponding to the named entities in the target text. The named entity recognition model is used to extract target feature vectors corresponding to each word embedding vector contained in the word embedding vector sequence in parallel, and classify the target feature vectors in parallel;

[0027] The third acquisition module is used to obtain the named entity recognition result of the target text according to the target category label sequence.

[0028] In one possible implementation, the named entity recognition model includes: a fully connected layer and a classification processing layer. The processing module is specifically used to: pass the word embedding vector sequence through the fully connected layer, and in parallel extract the target feature vectors corresponding to each word embedding vector contained in the word embedding vector sequence; pass the target feature vector through the classification processing layer, and in parallel classify the target feature vector to obtain the target category label sequence corresponding to the named entity in the target text.

[0029] In one possible implementation, the second acquisition module is specifically used to: obtain a word embedding vector sequence corresponding to the target text through a RoBERTa model including a preset number of attention mechanism layers, the RoBERTa model is used to convert the text contained in the target text into a corresponding numerical sequence through a preset dictionary, and obtain a word embedding vector sequence corresponding to the target text based on the numerical sequence and the pre-trained word vector.

[0030] In one possible implementation, the third acquisition module is specifically used to: integrate the target category label sequence according to a preset category label format to obtain the integrated target category labels; extract the text in the target text corresponding to the position of the integrated target category labels to obtain the named entity recognition result of the target text.

[0031] In one possible implementation, the named entity recognition device also includes a fourth acquisition module, which is used to: obtain a training set, the training set includes multiple sample pairs, the sample pairs include sample text and annotated category label sequence corresponding to the sample text, and the category label is used to identify the named entities contained in the text; train the named entity recognition model through the training set, obtain the word embedding vector sequence corresponding to the sample text, extract the target feature vectors corresponding to each word embedding vector contained in the word embedding vector sequence in parallel, and classify the target feature vectors in parallel to obtain the target category label sequence corresponding to the named entities in the sample text, and obtain the loss function value of the target category label sequence relative to the annotated category label sequence; adjust the parameters of the named entity recognition model according to the loss function value to obtain a trained named entity recognition model.

[0032] In one possible implementation, when the fourth acquisition module is used to parallel extract the target feature vector corresponding to each word embedding vector contained in the word embedding vector sequence, it is specifically used to: for each word embedding vector, perform the following steps in parallel: input the word embedding vector to a fully connected layer containing a first preset number of neurons for full connection processing to obtain the corresponding first feature vector; input the first feature vector to a fully connected layer containing a second preset number of neurons for full connection processing to obtain the corresponding second feature vector; input the second feature vector to a fully connected layer containing a third preset number of neurons for full connection processing to obtain the target feature vector corresponding to the word embedding vector, and the third preset number of neurons is the same as the number of preset category labels.

[0033] In one possible implementation, when the fourth acquisition module is used to classify the target feature vectors in parallel to obtain the target category label sequence corresponding to the named entities in the sample text, it is specifically used to: for each target feature vector, perform the following steps in parallel: classify the target feature vector to obtain the probability that the target feature vector corresponds to each preset category label; determine the preset category label corresponding to the maximum probability in the probabilities, which is the target category label corresponding to the named entity in the sample text.

[0034] In a fourth aspect, the present application provides a training device for a named entity recognition model, comprising:

[0035] An acquisition module is used to acquire a training set, which includes multiple sample pairs. Each sample pair includes a sample text and a sequence of annotated category labels corresponding to the sample text. The category labels are used to identify named entities contained in the text.

[0036] A training module is used to train the named entity recognition model using the training set, obtain the word embedding vector sequence corresponding to the sample text, extract the target feature vector corresponding to each word embedding vector contained in the word embedding vector sequence in parallel, and classify the target feature vectors in parallel to obtain the target category label sequence corresponding to the named entities in the sample text, and obtain the loss function value of the target category label sequence relative to the annotated category label sequence;

[0037] The processing module is used to adjust the parameters of the named entity recognition model according to the loss function value to obtain a trained named entity recognition model.

[0038] In one possible implementation, the training module is specifically used to: for each word embedding vector, perform the following steps in parallel: input the word embedding vector into a fully connected layer containing a first preset number of neurons for fully connected processing to obtain a corresponding first eigenvector; input the first eigenvector into a fully connected layer containing a second preset number of neurons for fully connected processing to obtain a corresponding second eigenvector; input the second eigenvector into a fully connected layer containing a third preset number of neurons for fully connected processing to obtain a target eigenvector corresponding to the word embedding vector, and the third preset number of neurons is the same as the number of preset category labels.

[0039] In one possible implementation, the training module is specifically used to: for each target feature vector, perform the following steps in parallel: classify the target feature vector to obtain the probability that the target feature vector corresponds to each preset category label; determine the preset category label corresponding to the maximum probability among the probabilities, which is the target category label corresponding to the named entity in the sample text.

[0040] In a fifth aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0041] Memory stores computer-executable instructions;

[0042] The processor executes the computer-executable instructions stored in the memory to implement the method as described in the first aspect of the present application.

[0043] In a sixth aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0044] Memory stores computer-executable instructions;

[0045] The processor executes the computer-executable instructions stored in the memory to implement the method as described in the second aspect of the present application.

[0046] In a seventh aspect, the present application provides a computer-readable storage medium, in which computer program instructions are stored. When the computer program instructions are executed, the method described in the first aspect of the present application is implemented.

[0047] In an eighth aspect, the present application provides a computer-readable storage medium, in which computer program instructions are stored. When the computer program instructions are executed, the method described in the second aspect of the present application is implemented.

[0048] In a ninth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method described in the first aspect of the present application.

[0049] In a tenth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method described in the second aspect of the present application.

[0050] Those skilled in the art will understand that, in the present application, by obtaining the target text to be subjected to named entity recognition, a word embedding vector sequence corresponding to the target text is obtained; the word embedding vector sequence is input into the named entity recognition model, and the output of the named entity recognition model is a target category label sequence corresponding to the named entities in the target text, and the named entity recognition model is used to extract the target feature vectors corresponding to each word embedding vector contained in the word embedding vector sequence in parallel, and to classify the target feature vectors in parallel; according to the target category label sequence, the named entity recognition result of the target text is obtained. Since the present application uses the named entity recognition model to extract the target feature vectors corresponding to each word embedding vector contained in the word embedding vector sequence in parallel, and to classify the target feature vectors in parallel, the efficiency of named entity recognition can be greatly improved, and the accuracy of named entity recognition can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The preferred embodiments of the washing machine interaction method, device, equipment and storage medium of the present application are described below with reference to the accompanying drawings.

[0052] Figure 1 A schematic diagram of an application scenario provided in an embodiment of the present application;

[0053] Figure 2 A flowchart of a method for named entity recognition provided in one embodiment of the present application;

[0054] Figure 3 A flowchart of a method for named entity recognition provided in another embodiment of the present application;

[0055] Figure 4 A flowchart of a method for obtaining a named entity recognition model provided in one embodiment of the present application;

[0056] Figure 5 A flowchart of a method for training a named entity recognition model provided in one embodiment of the present application;

[0057] Figure 6 A schematic diagram of a named entity recognition model processing flow provided in one embodiment of the present application;

[0058] Figure 7 A schematic diagram of the structure of a named entity recognition device provided in one embodiment of the present application;

[0059] Figure 8 A schematic diagram of the structure of a training device for a named entity recognition model provided in one embodiment of the present application;

[0060] Figure 9 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0061] First, those skilled in the art should understand that these embodiments are merely used to explain the technical principles of this application and are not intended to limit the scope of protection of this application. Those skilled in the art may adjust them as needed to suit specific applications.

[0062] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "an" and "the" used in the embodiments of the present application are also intended to include plural forms, unless the context clearly indicates otherwise.

[0063] It should be understood that the term "and / or" as used herein is merely a description of the relationship between associated objects, indicating that three possible relationships exist. For example, A and / or B can be expressed as: A alone, A and B simultaneously, or B alone. Furthermore, the character " / " in this document generally indicates that the associated objects are in an "or" relationship.

[0064] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to the determination" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)," depending on the context.

[0065] It should also be noted that the terms "include," "comprises," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a product or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such product or system. In the absence of further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the product or system comprising the element.

[0066] First, some technical terms involved in this application are explained:

[0067] BIO is a category label format, where B stands for Begin, I stands for Intermediate, and O stands for Other. It is used to mark characters that are not named entities. BIO can split a real named entity category label, such as Name, into two category labels representing named entities, B-Name and I-Name, and a category label representing a non-named entity, O. After obtaining B-Name, BIO can be understood as obtaining the named entity Name. The BIO format has a high recall rate.

[0068] BIES is a category label format in which E stands for End and S stands for Single. BIES can split real named entity category labels, such as Name, into B-Name, I-Name, E-Name, S-Name, and O. BIES requires B-Name and E-Name or S-Name to be considered as the named entity Name. Compared with the BIO format, the BIES format has higher accuracy.

[0069] CRF is a discriminant method whose calculation mainly involves two matrices: the transfer matrix and the emission matrix. The transfer matrix is ​​responsible for ensuring the order of the output category label sequence. For example, the next category label of B-Name can only correspond to I-Name or E-Name, and cannot be other named entity category labels such as I-Location. The emission matrix is ​​responsible for the probability of generating category labels, such as the probability that a character corresponds to B-Name or I-Location.

[0070] Currently, in the field of intelligent voice control, named entity recognition (NER) of short texts is often involved, extracting named entities from the text. The commonly used NER solution (i.e., Bert+BiLSTM+CRF) has the following specific processing steps:

[0071] (1) Data preprocessing: Convert the labeled data into the Bert input format and select an appropriate batch size (batch_size);

[0072] (2) Embedding layer: The output sequence of Bert is used as the embedding result, that is, the word embedding vector sequence is obtained;

[0073] (3) BiLSTM layer: BiLSTM refers to a two-layer network structure with forward and reverse directions in sentence order. The two layers are independent when extracting text sequence features. Finally, the outputs of the two layers are connected into a vector, and then the output is converted to the dimension of the hidden layer size (hidden_size) through a fully connected (dense) operation.

[0074] (4) CRF layer: The CRF layer mainly iteratively optimizes the transfer matrix and the emission matrix, and uses the transfer matrix and the emission matrix to decode the output of the BiLSTM layer. The prediction process only requires decoding calculations;

[0075] (5) Label integration: The predicted continuous category labels with BIES are mapped to the real category labels, and the text at the corresponding position in the original text is extracted.

[0076] The above named entity recognition scheme can effectively extract named entities in text, but it has the following disadvantages:

[0077] (1) Native BERT uses a 768-dimensional hidden_size and a 12-layer attention mechanism. This results in a large number of model parameters, slow training speed, and a high resource consumption during the prediction process.

[0078] (2) Native BERT uses the Next Sentence Prediction (NSP) task. The NSP task is used to predict whether a sentence pair is a contextual relationship. However, the training set data are all single sentences, and there are no sentence pairs. In addition, the NSP task in BERT is generally believed to have no obvious effect on extracting sentence features.

[0079] (3) BiLSTM is an improvement on the Recurrent Neural Network (RNN). It is a temporal network in which the calculation of the next word depends on the calculation result of the previous word. It can only be calculated serially, not in parallel.

[0080] (4) The CRF decoding process uses the Viterbi method, which selects the path with the highest probability among all possible paths, that is, generates the most likely category label sequence. CRF decoding is also performed sequentially and cannot be calculated in parallel.

[0081] In related technologies, named entity recognition can also be performed using the following two approaches: the first uses a fixed word embedding vector sequence, such as word2vec or glove, combined with an LSTM and a CRF. The second approach uses a pre-trained word embedding vector sequence from a pre-trained model, such as BERT or GPT, combined with an LSTM and a CRF. The second approach also does not require an LSTM network. The second approach achieves higher accuracy with the pre-trained model because it learns a large number of open-domain text features during pre-training, resulting in a more refined partitioning of the feature space of the word embedding vector sequence. The word embedding vector of the same word can express multiple meanings when combined with different words, thus creating a polysemy problem. Although the methods for obtaining the word embedding vector sequence differ, both employ an LSTM and a CRF. However, LSTM can only be computed serially, not in parallel. CRF decoding is also performed sequentially and cannot be performed in parallel. This leads to a problem: when online services have a high concurrency rate, this serial computation can significantly slow service response times.

[0082] Because both LSTM and CRF perform serial computations based on the order of text, they cannot achieve parallel computation at the word level (including individual characters). When the volume of online service requests is high, serial computation becomes a bottleneck, resulting in underutilized computing resources and a backlog of requests. Based on the second solution described above, experiments have improved upon it by removing the LSTM component and using only the Bert+CRF solution. While the query rate per second (QPS) is higher than that of the solution with LSTM, GPU utilization still fluctuates. This is because the CRF decoding process is also a serial computation, causing a sudden drop in GPU utilization. After decoding is complete, GPU utilization returns to 99%. Since CRF's serial computation is inherently structured, increasing CRF concurrency requires adding more machines, which does not improve the utilization of individual machines. Therefore, other methods are needed to achieve the same results as the CRF output.

[0083] From an output perspective, the total number of category labels is known and small. Once the BERT model has fully learned the semantic features of the training set, it can directly classify the word embedding vector sequence using a normalized exponential function (softmax) across all category labels, effectively transforming the named entity recognition problem into a classification problem. However, this classification result does not explicitly verify the validity of the category label sequence. Therefore, it is possible that the next category label after a B-Name is an I-Location. In this case, using BIO labels will result in a large number of false calls. Therefore, choosing the BIES format can reduce false calls. After adopting the Bert+softmax solution, through comparison, after fine-tuning the named entity recognition model using a 140,000 training set, the final classification results of softmax on the 30,000 test set were basically the same as those of CRF, both reaching above 99.5%. Most of the mispredicted data on the test set was caused by errors in the labeled data itself, indicating that the Bert+softmax solution is feasible and effective in the named entity recognition task of short texts. The solution was then stress tested, and the QPS increased from the original 100+ to the current 500+. The GPU utilization rate remained stable at 99%. This solution can well cope with high-concurrency online requests and save computing resources.

[0084] To address the above issues, this application provides a method, apparatus, device, and storage medium for named entity recognition. These methods obtain a sequence of word embedding vectors corresponding to a target text and then perform full-connection processing and classification processing on the word embedding vectors in parallel to obtain named entity recognition results for the target text. This significantly improves the efficiency and accuracy of named entity recognition.

[0085] Below, the application scenarios of the solution provided in this application are first illustrated.

[0086] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of the present application. Figure 1 As shown, in this application scenario, the client 101 obtains the text to be recognized by named entities and sends the text to the server 102. The server 102 recognizes the named entities in the text and sends the named entities to the client 101, which then displays the named entities. The specific implementation process of the server 102 recognizing the named entities in the text can be found in the solutions of the following embodiments.

[0087] It should be noted that Figure 1 This is only a schematic diagram of an application scenario provided by the embodiment of the present application. Figure 1 The equipment included in the Figure 1 The positional relationship between the devices in the Figure 1 The application scenario shown may also include a data storage device, which may be an external memory relative to the client 101 or the server 102 , or an internal memory integrated in the client 101 or the server 102 .

[0088] Next, a named entity recognition method is introduced through a specific embodiment.

[0089] Figure 2 The method of the embodiment of the present application can be applied to an electronic device, which can be Figure 1 The client 101 or server 102, or server cluster etc. Figure 2 As shown, the method of the embodiment of the present application includes:

[0090] S201: Obtain target text for named entity recognition.

[0091] In an embodiment of the present application, for example, the target text contains named entities to be recognized. For example, the target text is: "Open the refrigerator after one hour", where "one hour" and "refrigerator" are both named entities to be recognized. For example, the target text can be input by a user to the electronic device executing the embodiment of the present application, or sent by another device to the electronic device executing the embodiment of the present application, so that the target text for named entity recognition can be obtained.

[0092] S202: Obtain a word embedding vector sequence corresponding to the target text.

[0093] For example, a pre-trained model such as the RoBERTa model can be used to obtain a sequence of word embedding vectors corresponding to the target text. Specifically, for each word (a single character can also be understood as a word) contained in the target text, a corresponding word embedding vector can be obtained. The word embedding vector is, for example, a 768-dimensional vector. For how to obtain the word embedding vector sequence corresponding to the target text, please refer to the relevant technology or subsequent embodiments, which will not be repeated here.

[0094] S203. Input the word embedding vector sequence into the named entity recognition model, and obtain the output of the named entity recognition model as a target category label sequence corresponding to the named entities in the target text. The named entity recognition model is used to extract the target feature vectors corresponding to each word embedding vector contained in the word embedding vector sequence in parallel, and to classify the target feature vectors in parallel.

[0095] In this step, after obtaining the word embedding vector sequence corresponding to the target text, the word embedding vector sequence can be input into the named entity recognition model, and the output of the named entity recognition model is the target category label sequence corresponding to the named entities in the target text. For example, the target text is: open the refrigerator after one hour, and the preset category labels include time (time) and device name (deviceName). Using the above-mentioned BIES category label format, the word embedding vector sequence corresponding to the target text is input into the named entity recognition model, and the target category label sequence corresponding to the named entities in the target text can be obtained as follows: B-time I-time E-time OOO B-deviceName E-deviceName. It should be noted that the preset category labels can be set as needed, and the preset category labels include, for example: deviceName, time, mode (mode), quantity (number) and location (location).

[0096] S204: Obtain a named entity recognition result of the target text according to the target category label sequence.

[0097] After obtaining the target category label sequence corresponding to the named entity in the target text, the named entity recognition result of the target text can be obtained according to the target category label sequence. For example, if the target text is: open the refrigerator after one hour, then according to the target category label sequence corresponding to the named entity in the target text B-time I-time E-time OO O B-deviceName E-deviceName, the named entity recognition result of the target text is: {"deviceName":"refrigerator","time":"one hour"}. As for how to obtain the named entity recognition result of the target text according to the target category label sequence, please refer to the relevant technology or subsequent embodiments, which will not be repeated here.

[0098] After obtaining the named entity recognition result of the target text, the named entity recognition result can be displayed.

[0099] The named entity recognition method provided in the embodiment of the present application obtains a target text to be subjected to named entity recognition, obtains a word embedding vector sequence corresponding to the target text; inputs the word embedding vector sequence into a named entity recognition model, and obtains the output of the named entity recognition model as a target category label sequence corresponding to the named entities in the target text; the named entity recognition model is used to extract target feature vectors corresponding to each word embedding vector contained in the word embedding vector sequence in parallel, and classify the target feature vectors in parallel; and obtains the named entity recognition result of the target text according to the target category label sequence. Since the embodiment of the present application extracts target feature vectors corresponding to each word embedding vector contained in the word embedding vector sequence in parallel through the named entity recognition model, and classifies the target feature vectors in parallel, it can greatly improve the efficiency of named entity recognition and can improve the accuracy of named entity recognition.

[0100] Figure 3 A flowchart of a method for identifying named entities is provided in accordance with another embodiment of the present application. Figure 3 As shown, the method of the embodiment of the present application may include:

[0101] S301: Obtain target text for named entity recognition.

[0102] The detailed description of this step can be found in Figure 2 The relevant description of S201 in the illustrated embodiment will not be repeated here.

[0103] In the embodiment of this application, Figure 2 The step S202 may further include the following step S302:

[0104] S302. Obtain the word embedding vector sequence corresponding to the target text through the RoBERTa model including a preset number of attention mechanism layers.

[0105] Among them, the RoBERTa model is used to convert the text contained in the target text into a corresponding numerical sequence through a preset dictionary, and to obtain the word embedding vector sequence corresponding to the target text based on the numerical sequence and the pre-trained word vector.

[0106] Exemplarily, the number of preset attention mechanism layers is, for example, 3 layers, which can greatly reduce the amount of network parameters compared to the 12-layer attention mechanism of the Bert model. For the characters contained in the target text, the preset dictionary of the RoBERTa model has a serial number corresponding to each character. Therefore, the characters contained in the target text can be converted into a corresponding numerical sequence through the preset dictionary of the RoBERTa model, that is, a numerical sequence composed of serial numbers corresponding to the characters contained in the target text. Then, the numerical sequence can be calculated through the pre-trained word vector of the RoBERTa model to obtain a word embedding vector sequence corresponding to the target text. It can be understood that for a numerical sequence, the pre-trained word vector of the RoBERTa model has a word embedding vector corresponding to each numerical value, and the word embedding vector is, for example, a 768-dimensional vector. If the target text contains 8 characters, 8 corresponding 768-dimensional word embedding vectors can be obtained through the RoBERTa model.

[0107] Exemplarily, if there are multiple target texts to be recognized as named entities, the number of target texts to be input into the RoBERTa model at the same time, that is, batch_size, can be set, and batch_size is, for example, 16. The characters contained in each target text are converted into corresponding numerical sequences through the preset dictionary of the RoBERTa model. In order to keep the length of each batch consistent, the numerical sequence is padded (padding) according to the maximum length (for example, 32, that is, 32 characters), and the dimension of the numerical sequence is: 16*32. Then, the numerical sequence can be calculated through the pre-trained word vector of the RoBERTa model to obtain the word embedding vector sequence corresponding to the target text. The dimension of the word embedding vector is, for example, 768 dimensions, and the dimension of the word embedding vector sequence is 16*32*768.

[0108] In the embodiment of this application, Figure 2 Step S203 may further include the following two steps S303 and S304:

[0109] S303. The named entity recognition model includes: a fully connected layer and a classification processing layer. The word embedding vector sequence passes through the fully connected layer, and the target feature vector corresponding to each word embedding vector contained in the word embedding vector sequence is extracted in parallel.

[0110] Exemplarily, the fully connected layer is, for example, three fully connected layers, the first fully connected layer contains, for example, 1024 neurons, the second fully connected layer contains, for example, 2048 neurons, and the third fully connected layer contains, for example, 21 neurons, wherein the number of neurons contained in the third fully connected layer is the same as the number of preset category labels, that is, the number of preset category labels is also 21. After obtaining the word embedding vector sequence corresponding to the target text, the word embedding vector sequence can be fully connected in the word embedding vector dimension through the first fully connected layer, the second fully connected layer and the third fully connected layer in sequence, and the target feature vector corresponding to each word embedding vector contained in the word embedding vector sequence can be extracted in parallel. Exemplarily, for the word embedding vector sequence of 16*32*768 dimensions in the above embodiment, the corresponding target feature vector of 16*32*21 dimensions can be obtained.

[0111] S304 , passing the target feature vector through a classification processing layer, and performing classification processing on the target feature vector in parallel to obtain a target category label sequence corresponding to the named entity in the target text.

[0112] For example, if the classification processing layer includes a softmax function, the target feature vector is processed by softmax and then classified in parallel to obtain a target category label sequence corresponding to the named entities in the target text. Specifically, the probability of the target feature vector corresponding to each preset category label can be obtained by the following formula:

[0113]

[0114] Wherein, e represents a natural number, N represents the total number of values ​​contained in a vector, i represents the i-th value of the N values ​​whose probability is to be calculated, and j represents each value in the N values.

[0115] Then, the preset category label corresponding to the maximum probability in the probabilities can be determined as the target category label, and then the target category label sequence corresponding to the named entity in the target text can be obtained.

[0116] In the embodiment of this application, Figure 2 The step S204 may further include the following two steps S305 and S306:

[0117] S305 : Integrate the target category label sequence according to a preset category label format to obtain integrated target category labels.

[0118] For example, if the preset category label format is the BIES format, the target category labels that conform to the BIES format are merged and sorted, and the target category labels that do not conform to the BIES format are discarded to obtain a target category label sequence after integration. For example, if the target text is: Open the refrigerator after one hour, the corresponding target category label sequence before integration is: B-time I-time E-time OOO B-deviceName E-deviceName, then B-time, I-time, and E-time are extracted and synthesized into the target category label time, and the positions of B-time, I-time, and E-time are recorded. B-deviceName and E-deviceName are synthesized into the target category label deviceName, and the positions of B-deviceName and E-deviceName are recorded.

[0119] S306: extract the text corresponding to the position of each target category label after the integration process in the target text to obtain the named entity recognition result of the target text.

[0120] Exemplarily, referring to the target text example in step S305, for the target text: open the refrigerator after one hour, the target category labels after integration processing are time and deviceName respectively; among them, for time, the text corresponding to the positions of B-time, I-time and E-time in the target text can be extracted, that is, one hour; for deviceName, the text corresponding to the positions of B-deviceName and E-deviceName in the target text can be extracted, that is, refrigerator, and the named entity recognition result of the target text can be obtained as: {"deviceName":"refrigerator","time":"one hour"}.

[0121] The named entity recognition method provided in the embodiment of the present application obtains a target text for named entity recognition, obtains a word embedding vector sequence corresponding to the target text through a RoBERTa model including a preset number of attention mechanism layers, the named entity recognition model includes a fully connected layer and a classification processing layer, passes the word embedding vector sequence through the fully connected layer, extracts the target feature vector corresponding to each word embedding vector contained in the word embedding vector sequence in parallel, passes the target feature vector through the classification processing layer, and classifies the target feature vector in parallel to obtain a target category label sequence corresponding to the named entity in the target text, integrates the target category label sequence according to a preset category label format, obtains each target category label after integration, extracts the text corresponding to the position of each target category label after integration in the target text, and obtains a named entity recognition result for the target text. Since the embodiment of the present application extracts the target feature vector corresponding to each word embedding vector contained in the word embedding vector sequence in parallel through the fully connected layer of the named entity recognition model, and classifies the target feature vector in parallel through the classification processing layer of the named entity recognition model, and integrates the classification processing results using a preset category label format with higher accuracy, the efficiency of named entity recognition can be greatly improved, and the accuracy of named entity recognition can be improved.

[0122] Figure 4 This is a flowchart of a method for obtaining a named entity recognition model provided by an embodiment of the present application. The method of the embodiment of the present application can be applied to an electronic device, which can be a server or a server cluster. Figure 4 As shown, the method of the embodiment of the present application includes:

[0123] S401: Obtain a training set, where the training set includes multiple sample pairs. Each sample pair includes a sample text and a sequence of annotated category labels corresponding to the sample text.

[0124] Among them, the category label is used to identify the named entities contained in the text.

[0125] For example, business log data is collected, and after operations such as deduplication, removal of business-irrelevant sentences, and removal of sensitive words, the obtained sample text is manually annotated, and then the annotated sample text is sampled 7 / 2 / 1, that is, 70% as a training set, 20% as a validation set, and 10% as a test set. The number of preset category labels is, for example, 21, and the preset category label format is, for example, the BIES format, which can reduce false calls. The format of the annotated sample text is, for example:

[0126] Open the refrigerator after one hour

[0127] B-time I-time E-time OOO B-deviceName E-deviceName

[0128] Among them, "Open the refrigerator after one hour" is the sample text, and "B-time I-time E-time OOO B-deviceName E-deviceName" is the annotation category label sequence corresponding to the sample text.

[0129] S402. Train the named entity recognition model using the training set to obtain a word embedding vector sequence corresponding to the sample text, extract the target feature vectors corresponding to each word embedding vector contained in the word embedding vector sequence in parallel, and classify the target feature vectors in parallel to obtain a target category label sequence corresponding to the named entities in the sample text, and obtain a loss function value of the target category label sequence relative to the annotated category label sequence.

[0130] Exemplarily, the sample text in the training set is converted into a corresponding numerical sequence through the preset dictionary of the RoBERTa model, and then the numerical sequence is calculated through the pre-trained word vector of the RoBERTa model to obtain the word embedding vector sequence corresponding to the sample text. For the word embedding vector sequence, the target feature vectors corresponding to each word embedding vector contained in the word embedding vector sequence can be extracted in parallel through the multi-layer fully connected layer of the named entity recognition model, and then the target feature vectors are classified in parallel through the softmax of the named entity recognition model to obtain the target category label sequence corresponding to the named entity in the sample text. According to the target category label sequence and the corresponding annotated category label sequence, the loss function value of the target category label sequence relative to the annotated category label sequence is obtained.

[0131] S403. Adjust the parameters of the named entity recognition model according to the loss function value to obtain a trained named entity recognition model.

[0132] After obtaining the loss function value, the parameters of the named entity recognition model can be adjusted according to the loss function value through reverse calculation of the gradient, and the named entity recognition model can be iteratively trained until the loss function value is less than the set threshold, thereby obtaining a trained named entity recognition model.

[0133] The method for obtaining a named entity recognition model provided in an embodiment of the present application obtains a training set, wherein the training set includes multiple sample pairs, wherein the sample pairs include sample text and a sequence of annotated category labels corresponding to the sample text, the named entity recognition model is trained through the training set, a sequence of word embedding vectors corresponding to the sample text is obtained, target feature vectors corresponding to each word embedding vector included in the word embedding vector sequence are extracted in parallel, and the target feature vectors are classified and processed in parallel to obtain a sequence of target category labels corresponding to the named entities in the sample text, a loss function value of the target category label sequence relative to the annotated category label sequence is obtained, and according to the loss function value, the parameters of the named entity recognition model are adjusted to obtain a trained named entity recognition model. Since the named entity recognition model of the embodiment of the present application extracts the target feature vectors corresponding to each word embedding vector included in the word embedding vector sequence in parallel, and classifies and processes the target feature vectors in parallel, the efficiency of named entity recognition can be greatly improved, and the accuracy of named entity recognition can be improved.

[0134] On the basis of the above embodiment, further, parallel extraction of target feature vectors corresponding to each word embedding vector contained in the word embedding vector sequence may include: for each word embedding vector, performing the following steps in parallel: inputting the word embedding vector into a fully connected layer containing a first preset number of neurons for fully connected processing to obtain a corresponding first feature vector; inputting the first feature vector into a fully connected layer containing a second preset number of neurons for fully connected processing to obtain a corresponding second feature vector; inputting the second feature vector into a fully connected layer containing a third preset number of neurons for fully connected processing to obtain a target feature vector corresponding to the word embedding vector, where the third preset number of neurons is the same as the number of preset category labels.

[0135] Exemplarily, the first preset number of neurons is, for example, 1024, the second preset number of neurons is, for example, 2048, and the third preset number of neurons is the same as the number of preset category labels, for example, both are 21. After obtaining the word embedding vector sequence, for each word embedding vector contained in the word embedding vector sequence, the following steps are performed in parallel: the word embedding vector is input into a fully connected layer containing 1024 neurons for full connection processing to obtain the corresponding first eigenvector, the length of the first eigenvector is 1024; the first eigenvector is input into a fully connected layer containing 2048 neurons for full connection processing to obtain the corresponding second eigenvector, the length of the second eigenvector is 2048; the second eigenvector is input into a fully connected layer containing 21 neurons for full connection processing to obtain the target eigenvector corresponding to the word embedding vector, the length of the target eigenvector is 21.

[0136] On the basis of the above embodiment, further, the target feature vectors are classified in parallel to obtain a target category label sequence corresponding to the named entities in the sample text, which may include: for each target feature vector, performing the following steps in parallel: classifying the target feature vector to obtain the probability that the target feature vector corresponds to each preset category label; determining the preset category label corresponding to the maximum probability in the probabilities, which is the target category label corresponding to the named entity in the sample text.

[0137] For example, the number of preset category labels is 21. Softmax classification is performed on the target feature vector in parallel to obtain the probability that the target feature vector corresponds to each of the 21 preset category labels. The preset category label corresponding to the maximum probability among the 21 probabilities can then be determined as the target category label corresponding to the named entity in the sample text. Based on each target category label, a sequence of target category labels corresponding to the named entity in the sample text can be obtained.

[0138] Figure 5 This is a flowchart of a method for training a named entity recognition model provided by an embodiment of the present application. The method of the embodiment of the present application can be applied to an electronic device, which can be a server or a server cluster. Figure 5 As shown, the method of the embodiment of the present application includes:

[0139] S501: Obtain a training set, where the training set includes multiple sample pairs. Each sample pair includes a sample text and a sequence of annotated category labels corresponding to the sample text.

[0140] Among them, the category label is used to identify the named entities contained in the text.

[0141] For example, business log data is collected, and after deduplication, removal of irrelevant business statements, and removal of sensitive words, the obtained sample text is manually annotated. The annotated sample text is then sampled in a 7 / 2 / 1 manner, that is, 70% is used as a training set, 20% as a validation set, and 10% as a test set. The number of preset category labels is, for example, 21. The format of the annotated sample text is, for example:

[0142] Open the refrigerator after one hour

[0143] B-time I-time E-time OOO B-deviceName E-deviceName

[0144] Among them, "Open the refrigerator after one hour" is the sample text, and "B-time I-time E-time OOO B-deviceName E-deviceName" is the annotation category label sequence corresponding to the sample text.

[0145] S502. Train the named entity recognition model using the training set to obtain a word embedding vector sequence corresponding to the sample text, extract the target feature vectors corresponding to each word embedding vector contained in the word embedding vector sequence in parallel, and classify the target feature vectors in parallel to obtain a target category label sequence corresponding to the named entities in the sample text, and obtain a loss function value of the target category label sequence relative to the annotated category label sequence.

[0146] Exemplarily, the sample text in the training set is converted into a corresponding numerical sequence through the preset dictionary of the RoBERTa model, and then the numerical sequence is calculated through the pre-trained word vector of the RoBERTa model to obtain the word embedding vector sequence corresponding to the sample text. For the word embedding vector sequence, the target feature vectors corresponding to each word embedding vector contained in the word embedding vector sequence can be extracted in parallel through the fully connected layer of the named entity recognition model, and then the target feature vectors are classified in parallel through the classification processing layer of the named entity recognition model to obtain the target category label sequence corresponding to the named entity in the sample text. According to the target category label sequence and the corresponding annotated category label sequence, the loss function value of the target category label sequence relative to the annotated category label sequence is obtained.

[0147] S503. Adjust the parameters of the named entity recognition model according to the loss function value to obtain a trained named entity recognition model.

[0148] After obtaining the loss function value, the parameters of the named entity recognition model can be adjusted according to the loss function value through reverse calculation of the gradient, and the named entity recognition model can be iteratively trained until the loss function value is less than the set threshold, thereby obtaining a trained named entity recognition model.

[0149] The training method of the named entity recognition model provided in the embodiment of the present application can obtain a named entity recognition model with higher recognition efficiency and can improve the accuracy of named entity recognition.

[0150] Based on the above embodiments, illustratively, Figure 6 This is a schematic diagram of the named entity recognition model processing flow provided in one embodiment of the present application. Figure 6 As shown, the method of the embodiment of the present application includes:

[0151] S601: Obtain target text for named entity recognition.

[0152] S602. Obtain the word embedding vector sequence corresponding to the target text through the RoBERTa model including a 3-layer attention mechanism.

[0153] For each word embedding vector included in the word embedding vector sequence, steps S603 to S605 are performed in parallel:

[0154] S603: Input the word embedding vector into a fully connected layer containing 1024 neurons for full connection processing to obtain the corresponding first eigenvector.

[0155] S604: Input the first eigenvector into a fully connected layer including 2048 neurons for full connection processing to obtain a corresponding second eigenvector.

[0156] S605: Input the second feature vector into a fully connected layer containing 21 neurons for full connection processing to obtain a target feature vector corresponding to the word embedding vector.

[0157] Among them, the number of preset category labels is also 21.

[0158] For each target feature vector, execute step S606 in parallel:

[0159] S606. Classify the target feature vector using a normalized exponential function to obtain the probability that the target feature vector corresponds to each preset category label; determine the preset category label corresponding to the maximum probability among the probabilities, which is the target category label corresponding to the named entity in the sample text, and the preset category label format is the BIES format.

[0160] Among them, the normalized exponential function is softmax.

[0161] S607 : Obtain a target category label sequence in BIES format corresponding to the named entity in the sample text according to each target category label.

[0162] S608: Integrate the target category label sequence to obtain integrated target category labels.

[0163] S609: extracting text corresponding to the position of each target category label after the integration process in the target text, and obtaining a named entity recognition result of the target text.

[0164] The named entity recognition model provided by the embodiment of the present application uses a simpler RoBERTa model with a three-layer attention mechanism, namely the RoBERTa-L3 model, compared with the current related technologies, which greatly reduces the number of network parameters; cancels the NSP pre-training task, making the RoBERTa-L3 model more in line with the training set distribution of the business scenario; removes the LSTM and CRF network structures, directly uses softmax to classify the word embedding vector sequence of RoBERTa in parallel, and then integrates the classification results using the BIES category label format, which can further reduce false calls and ensure the accuracy of the final named entity category label. Since the steps that affect parallel computing are removed, the computing efficiency can be greatly improved. After actual measurement, the recognition efficiency of the named entity recognition model provided by the embodiment of the present application is 4 times higher than that of the existing related solutions, and the accuracy rate can reach more than 99.5%.

[0165] Figure 7 A schematic diagram of the structure of a named entity recognition device provided in one embodiment of the present application is shown in FIG. Figure 7 As shown, the named entity recognition device 700 of the embodiment of the present application includes: a first acquisition module 701, a second acquisition module 702, a processing module 703 and a third acquisition module 704. Among them:

[0166] The first acquisition module 701 is used to acquire a target text to be subjected to named entity recognition.

[0167] The second acquisition module 702 is used to obtain the word embedding vector sequence corresponding to the target text.

[0168] Processing module 703 is used to input the word embedding vector sequence into the named entity recognition model, and the output of the named entity recognition model is a target category label sequence corresponding to the named entities in the target text. The named entity recognition model is used to parallelly extract the target feature vectors corresponding to each word embedding vector contained in the word embedding vector sequence, and to parallelly classify the target feature vectors.

[0169] The third acquisition module 704 is used to obtain the named entity recognition result of the target text according to the target category label sequence.

[0170] In some embodiments, the named entity recognition model includes: a fully connected layer and a classification processing layer, and the processing module 703 can be specifically used to: pass the word embedding vector sequence through the fully connected layer, and extract the target feature vectors corresponding to each word embedding vector contained in the word embedding vector sequence in parallel; pass the target feature vector through the classification processing layer, and classify the target feature vector in parallel to obtain the target category label sequence corresponding to the named entity in the target text.

[0171] Optionally, the second acquisition module 702 can be specifically used to: obtain a word embedding vector sequence corresponding to the target text through a RoBERTa model including a preset number of attention mechanism layers, the RoBERTa model is used to convert the text contained in the target text into a corresponding numerical sequence through a preset dictionary, and obtain a word embedding vector sequence corresponding to the target text based on the numerical sequence and the pre-trained word vector.

[0172] Optionally, the third acquisition module 704 can be specifically used to: integrate the target category label sequence according to a preset category label format to obtain the integrated target category labels; extract the text in the target text corresponding to the position of the integrated target category labels to obtain the named entity recognition result of the target text.

[0173] In some embodiments, the named entity recognition device 700 may further include a fourth acquisition module 705, which is used to: obtain a training set, the training set includes multiple sample pairs, the sample pairs include sample text and annotated category label sequence corresponding to the sample text, and the category label is used to identify the named entities contained in the text; train the named entity recognition model through the training set, obtain the word embedding vector sequence corresponding to the sample text, extract the target feature vector corresponding to each word embedding vector contained in the word embedding vector sequence in parallel, and classify the target feature vector in parallel to obtain the target category label sequence corresponding to the named entity in the sample text, and obtain the loss function value of the target category label sequence relative to the annotated category label sequence; adjust the parameters of the named entity recognition model according to the loss function value to obtain a trained named entity recognition model.

[0174] Optionally, when the fourth acquisition module 705 is used to parallel extract the target feature vector corresponding to each word embedding vector contained in the word embedding vector sequence, it can be specifically used to: for each word embedding vector, perform the following steps in parallel: input the word embedding vector to a fully connected layer containing a first preset number of neurons for full connection processing to obtain the corresponding first feature vector; input the first feature vector to a fully connected layer containing a second preset number of neurons for full connection processing to obtain the corresponding second feature vector; input the second feature vector to a fully connected layer containing a third preset number of neurons for full connection processing to obtain the target feature vector corresponding to the word embedding vector, and the third preset number of neurons is the same as the number of preset category labels.

[0175] Optionally, when the fourth acquisition module 705 is used to classify the target feature vectors in parallel to obtain a target category label sequence corresponding to the named entities in the sample text, it can be specifically used to: for each target feature vector, perform the following steps in parallel: classify the target feature vector to obtain the probability that the target feature vector corresponds to each preset category label; determine the preset category label corresponding to the maximum probability in the probabilities, which is the target category label corresponding to the named entity in the sample text.

[0176] The device of the embodiment of the present application can be used to execute the solution of the named entity recognition method in any of the above method embodiments. Its implementation principle and technical effects are similar and will not be repeated here.

[0177] Figure 8 A structural diagram of a training device for a named entity recognition model provided in one embodiment of the present application is shown in FIG. Figure 8 As shown, the training device 800 for the named entity recognition model of the embodiment of the present application includes: an acquisition module 801, a training module 802 and a processing module 803. Among them:

[0178] The acquisition module 801 is used to acquire a training set. The training set includes multiple sample pairs. The sample pairs include sample texts and a sequence of annotated category labels corresponding to the sample texts. The category labels are used to identify named entities contained in the texts.

[0179] The training module 802 is used to train the named entity recognition model through the training set, obtain the word embedding vector sequence corresponding to the sample text, extract the target feature vector corresponding to each word embedding vector contained in the word embedding vector sequence in parallel, and classify the target feature vector in parallel to obtain the target category label sequence corresponding to the named entity in the sample text, and obtain the loss function value of the target category label sequence relative to the annotated category label sequence.

[0180] The processing module 803 is used to adjust the parameters of the named entity recognition model according to the loss function value to obtain a trained named entity recognition model.

[0181] In some embodiments, the training module 802 can be specifically used to: for each word embedding vector, perform the following steps in parallel: input the word embedding vector into a fully connected layer containing a first preset number of neurons for fully connected processing to obtain a corresponding first eigenvector; input the first eigenvector into a fully connected layer containing a second preset number of neurons for fully connected processing to obtain a corresponding second eigenvector; input the second eigenvector into a fully connected layer containing a third preset number of neurons for fully connected processing to obtain a target eigenvector corresponding to the word embedding vector, and the third preset number of neurons is the same as the number of preset category labels.

[0182] In some embodiments, the training module 802 can be specifically used to: for each target feature vector, perform the following steps in parallel: classify the target feature vector to obtain the probability that the target feature vector corresponds to each preset category label; determine the preset category label corresponding to the maximum probability in the probabilities, which is the target category label corresponding to the named entity in the sample text.

[0183] The device of the embodiment of the present application can be used to execute the training method of the named entity recognition model in any of the above method embodiments. Its implementation principle and technical effects are similar and will not be repeated here.

[0184] Figure 9 This is a schematic diagram of the structure of an electronic device provided in one embodiment of the present application. For example, the electronic device can be provided as a server or a computer. Figure 9 The electronic device 900 includes a processing component 901, which further includes one or more processors, and a memory resource represented by a memory 902 for storing instructions executable by the processing component 901, such as an application. The application stored in the memory 902 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 901 is configured to execute the instructions to perform any of the above-described method embodiments.

[0185] The electronic device 900 may further include a power supply component 903 configured to perform power management of the electronic device 900, a wired or wireless network interface 904 configured to connect the electronic device 900 to a network, and an input / output (I / O) interface 905. The electronic device 900 may operate based on an operating system stored in the memory 902, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or the like.

[0186] The present application also provides a computer-readable storage medium, which stores computer-executable instructions. When a processor executes the computer-executable instructions, the above-mentioned named entity recognition method and the training method of the named entity recognition model are implemented.

[0187] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the above-mentioned named entity recognition method and the training method of the named entity recognition model.

[0188] The computer-readable storage medium mentioned above can be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk. The computer-readable storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0189] An exemplary readable storage medium is coupled to the processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in the named entity recognition device or in the training device of the named entity recognition model.

[0190] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be implemented using hardware associated with program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0191] Thus far, the technical solutions of the present application have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is readily understood by those skilled in the art that the scope of protection of the present application is obviously not limited to these specific embodiments. Without departing from the principles of the present application, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present application.

Claims

1. A method for named entity recognition, characterized in that: include: Obtain the target text for named entity recognition; Obtain a word embedding vector sequence corresponding to the target text; Inputting the word embedding vector sequence into a named entity recognition model, the output of the named entity recognition model is a target category label sequence corresponding to the named entities in the target text, the named entity recognition model is used to extract target feature vectors corresponding to each word embedding vector included in the word embedding vector sequence in parallel, and classify the target feature vectors in parallel, the named entity recognition model includes: a fully connected layer and a classification processing layer; Obtaining a named entity recognition result of the target text according to the target category label sequence; The step of inputting the word embedding vector sequence into a named entity recognition model to obtain an output of the named entity recognition model as a target category label sequence corresponding to the named entities in the target text comprises: passing the word embedding vector sequence through the fully connected layer and extracting target feature vectors corresponding to each word embedding vector contained in the word embedding vector sequence in parallel; The target feature vector is passed through the classification processing layer and classified in parallel to obtain a target category label sequence corresponding to the named entity in the target text.

2. The named entity recognition method according to claim 1, wherein The obtaining of a word embedding vector sequence corresponding to the target text includes: The word embedding vector sequence corresponding to the target text is obtained through a RoBERTa model including a preset number of attention mechanism layers. The RoBERTa model is used to convert the characters contained in the target text into a corresponding numerical sequence through a preset dictionary, and to obtain the word embedding vector sequence corresponding to the target text based on the numerical sequence and the pre-trained word vector.

3. The named entity recognition method according to claim 1 or 2, characterized in that Obtaining a named entity recognition result of the target text according to the target category label sequence includes: Integrate the target category label sequence according to a preset category label format to obtain integrated target category labels; Extract the text corresponding to the position of each target category label after the integration process in the target text to obtain the named entity recognition result of the target text.

4. The named entity recognition method according to claim 1 or 2, characterized in that The named entity recognition model is obtained in the following way: Obtaining a training set, wherein the training set includes a plurality of sample pairs, each of which includes a sample text and a sequence of annotated category labels corresponding to the sample text, wherein the category labels are used to identify named entities contained in the text; Training a named entity recognition model using the training set to obtain a word embedding vector sequence corresponding to the sample text, extracting target feature vectors corresponding to each word embedding vector contained in the word embedding vector sequence in parallel, and classifying the target feature vectors in parallel to obtain a target category label sequence corresponding to the named entities in the sample text, and obtaining a loss function value of the target category label sequence relative to the annotated category label sequence; According to the loss function value, the parameters of the named entity recognition model are adjusted to obtain a trained named entity recognition model.

5. The method for named entity recognition according to claim 4, wherein: The parallel extraction of target feature vectors corresponding to the word embedding vectors included in the word embedding vector sequence includes: For each word embedding vector, perform the following steps in parallel: Inputting the word embedding vector into a fully connected layer including a first preset number of neurons for full connection processing to obtain a corresponding first eigenvector; Inputting the first eigenvector into a fully connected layer including a second preset number of neurons for full connection processing to obtain a corresponding second eigenvector; The second feature vector is input into a fully connected layer including a third preset number of neurons for full connection processing to obtain a target feature vector corresponding to the word embedding vector, where the third preset number of neurons is the same as the number of preset category labels.

6. The method for named entity recognition according to claim 4, wherein: The parallel classification processing of the target feature vector to obtain a target category label sequence corresponding to the named entity in the sample text includes: For each target feature vector, the following steps are performed in parallel: Classify the target feature vector to obtain the probability that the target feature vector corresponds to each preset category label; A preset category label corresponding to the maximum probability among the probabilities is determined as the target category label corresponding to the named entity in the sample text.

7. A method for training a named entity recognition model, characterized in that: include: Obtaining a training set, wherein the training set includes a plurality of sample pairs, each of which includes a sample text and a sequence of annotated category labels corresponding to the sample text, wherein the category labels are used to identify named entities contained in the text; Training a named entity recognition model using the training set to obtain a word embedding vector sequence corresponding to the sample text, extracting target feature vectors corresponding to each word embedding vector contained in the word embedding vector sequence in parallel, and classifying the target feature vectors in parallel to obtain a target category label sequence corresponding to the named entities in the sample text, and obtaining a loss function value of the target category label sequence relative to the annotated category label sequence; According to the loss function value, the parameters of the named entity recognition model are adjusted to obtain a trained named entity recognition model, and the named entity recognition model is used to implement the named entity recognition method described in any one of claims 1 to 6.

8. A named entity recognition device, characterized in that: include: A first acquisition module is used to acquire a target text to be subjected to named entity recognition; A second acquisition module is used to obtain a word embedding vector sequence corresponding to the target text; a processing module, configured to input the word embedding vector sequence into a named entity recognition model, obtaining an output of the named entity recognition model as a target category label sequence corresponding to the named entities in the target text, the named entity recognition model being configured to concurrently extract target feature vectors corresponding to each word embedding vector contained in the word embedding vector sequence, and concurrently perform classification processing on the target feature vectors, the named entity recognition model comprising: a fully connected layer and a classification processing layer; A third acquisition module is used to obtain a named entity recognition result of the target text according to the target category label sequence; The processing module is specifically configured to pass the word embedding vector sequence through the fully connected layer and extract target feature vectors corresponding to the word embedding vectors contained in the word embedding vector sequence in parallel; The target feature vector is passed through the classification processing layer and classified in parallel to obtain a target category label sequence corresponding to the named entity in the target text.

9. A training device for a named entity recognition model, characterized in that: include: An acquisition module is used to acquire a training set, wherein the training set includes a plurality of sample pairs, each of which includes a sample text and a sequence of annotated category labels corresponding to the sample text, wherein the category labels are used to identify named entities contained in the text; A training module is configured to train a named entity recognition model using the training set, obtain a word embedding vector sequence corresponding to the sample text, extract target feature vectors corresponding to each word embedding vector contained in the word embedding vector sequence in parallel, and classify the target feature vectors in parallel to obtain a target category label sequence corresponding to the named entities in the sample text, and obtain a loss function value of the target category label sequence relative to the annotated category label sequence; A processing module is used to adjust the parameters of the named entity recognition model according to the loss function value to obtain a trained named entity recognition model, and the named entity recognition model is used to implement the named entity recognition method according to any one of claims 1 to 6.

10. An electronic device, characterized in that: include: a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 7.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed, the method according to any one of claims 1 to 7 is implemented.

12. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images