Named entity recognition method, training method, device, electronic equipment and medium

Abstract

The present disclosure provides a named entity recognition method, a named entity recognition model training method, an apparatus, an electronic device, a storage medium and a program product, relates to the technical field of data processing, in particular to the technical field of artificial intelligence such as knowledge graph and deep learning, and specifically to the technical field of medical, military, financial, energy, power, chemical, scientific research, water conservancy, transportation, entertainment, education and the like. The specific implementation scheme is as follows: determining a training sample, the training sample comprising a sample text and a to-be-confirmed sample annotation result sequence; processing the sample text by using an initial named entity recognition model to obtain a sample label confidence set sequence; determining a target sample annotation result sequence of the sample text based on the sample label confidence set sequence and the to-be-confirmed sample annotation result sequence; and adjusting parameters of the initial named entity recognition model by using the sample label confidence set sequence and the target sample annotation result sequence to obtain a named entity recognition model.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of data processing, and particularly relates to the technical field of artificial intelligence such as knowledge graph and deep learning. Specifically, the present disclosure relates to a named entity recognition method, a named entity recognition model training method, an apparatus, an electronic device, a storage medium, and a program product. BACKGROUND

[0002] Named entity recognition is a basic task in natural language processing. The named entity recognition technology can be applied to relation extraction, event extraction, knowledge graph, machine translation, question and answer search, and many other fields. SUMMARY

[0003] The present disclosure provides a named entity recognition method, a named entity recognition model training method, an apparatus, an electronic device, a storage medium, and a program product.

[0004] According to an aspect of the present disclosure, a named entity recognition model training method is provided, including: determining a training sample, wherein the training sample includes a sample text and a to-be-confirmed sample annotation result sequence; processing the sample text by using an initial named entity recognition model to obtain a sample label confidence set sequence; determining a target sample annotation result sequence of the sample text based on the sample label confidence set sequence and the to-be-confirmed sample annotation result sequence; and adjusting parameters of the initial named entity recognition model by using the sample label confidence set sequence and the target sample annotation result sequence to obtain a named entity recognition model.

[0005] According to another aspect of the present disclosure, a named entity recognition method is provided, including: inputting a to-be-recognized text into a named entity recognition model to obtain a named entity label sequence of the to-be-recognized text, wherein the to-be-recognized text includes at least one character, the named entity label sequence includes at least one named entity label, the at least one character corresponds to the at least one named entity label one by one, and the named entity label is used to indicate whether the character is a named entity; wherein the named entity recognition model is trained by using the named entity recognition model training method of the present disclosure.

[0006] According to another aspect of the present disclosure, there is provided a training apparatus of a named entity recognition model, comprising: a first determining module configured to determine a training sample, wherein the training sample comprises a sample text and a to-be-confirmed sample annotation result sequence; a processing module configured to process the sample text by using an initial named entity recognition model to obtain a sample label confidence set sequence; a second determining module configured to determine a target sample annotation result sequence of the sample text based on the sample label confidence set sequence and the to-be-confirmed sample annotation result sequence; and an adjusting module configured to adjust parameters of the initial named entity recognition model by using the sample label confidence set sequence and the target sample annotation result sequence to obtain a named entity recognition model.

[0007] According to another aspect of the present disclosure, there is provided a named entity recognition apparatus, comprising: an input module configured to input a to-be-recognized text into a named entity recognition model to obtain a named entity label sequence of the to-be-recognized text, wherein the to-be-recognized text comprises at least one character, the named entity label sequence comprises at least one named entity label, the at least one character corresponds to the at least one named entity label one by one, and the named entity label is used to indicate whether the character is a named entity; wherein the named entity recognition model is trained by using the training apparatus of the named entity recognition model of the present disclosure.

[0008] According to another aspect of the present disclosure, there is provided an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of the present disclosure.

[0009] According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to perform the method of the present disclosure.

[0010] According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the method of the present disclosure.

[0011] It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0012] The accompanying drawings are used to better understand the present scheme, and do not constitute a limitation on the present disclosure. Among them:

[0013] Figure 1An exemplary system architecture to which the named entity recognition method and apparatus according to embodiments of the present disclosure can be applied is schematically shown;

[0014] Figure 2 A flowchart of a training method of a named entity recognition model according to embodiments of the present disclosure is schematically shown;

[0015] Figure 3 A model architecture diagram of an initial named entity recognition model according to embodiments of the present disclosure is schematically shown;

[0016] Figure 4 A flowchart of a named entity recognition method according to embodiments of the present disclosure is schematically shown;

[0017] Figure 5 A flowchart of a named entity recognition method according to another embodiment of the present disclosure is schematically shown;

[0018] Figure 6 A block diagram of a training apparatus of a named entity recognition model according to embodiments of the present disclosure is schematically shown;

[0019] Figure 7 A block diagram of a named entity recognition apparatus according to embodiments of the present disclosure is schematically shown; and

[0020] Figure 8 A block diagram of an electronic device suitable for implementing a named entity recognition method according to embodiments of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0021] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of embodiments of the present disclosure are set forth to help understand the present disclosure. It should be understood that the embodiments described herein are merely exemplary and that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0022] The present disclosure provides a named entity recognition method, a training method and apparatus of a named entity recognition model, an electronic device, a storage medium, and a program product.

[0023] According to an embodiment of the present disclosure, a training method of a named entity recognition model is provided, which can include: determining a training sample, wherein the training sample includes a sample text and a to-be-confirmed sample annotation result sequence; processing the sample text by using an initial named entity recognition model to obtain a sample label confidence set sequence; determining a target sample annotation result sequence of the sample text based on the sample label confidence set sequence and the to-be-confirmed sample annotation result sequence; and adjusting parameters of the initial named entity recognition model by using the sample label confidence set sequence and the target sample annotation result sequence to obtain the named entity recognition model.

[0024] According to an embodiment of the present disclosure, a named entity recognition method is provided, which can include: inputting a to-be-recognized text into a named entity recognition model to obtain a named entity label sequence of the to-be-recognized text, wherein the to-be-recognized text includes at least one character, the named entity label sequence includes at least one named entity label, the at least one character corresponds to the at least one named entity label one by one, and the named entity label is used to indicate whether the character is a named entity; wherein the named entity recognition model is trained by using the training method of the named entity recognition model of the present disclosure.

[0025] According to an embodiment of the present disclosure, the named entity recognition method can be applied to technical fields such as medical treatment, military, finance, but is not limited thereto, and can also be applied to technical fields such as energy, power, chemical industry, scientific research, water conservancy, transportation, entertainment, education, etc. The application range is wide and the adaptability is high.

[0026] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in the technical solution comply with the relevant legal regulations and do not violate public order and good customs.

[0027] Figure 1 An exemplary system architecture to which the named entity recognition method and device according to an embodiment of the present disclosure can be applied is schematically shown.

[0028] It should be noted that, Figure 1 The shown is only an example of a system architecture to which the embodiment of the present disclosure can be applied, to help those skilled in the art understand the technical content of the present disclosure, but does not mean that the embodiment of the present disclosure cannot be used in other devices, systems, environments or scenarios. For example, in another embodiment, the exemplary system architecture to which the named entity recognition method and device can be applied can include a terminal device, but the terminal device can not need to interact with the server to implement the named entity recognition method and device provided by the present disclosure.

[0029] As Figure 1As shown, the system architecture 100 according to this embodiment can include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is a medium for providing a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 can include various connection types, such as wired and / or wireless communication links, and the like.

[0030] A user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, and the like. Various communication client applications can be installed on the terminal devices 101, 102, 103, such as knowledge reading applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, and the like (only as examples).

[0031] The terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smartphones, tablet computers, laptop computers, desktop computers, and the like.

[0032] The server 105 can be a server providing various services, such as a background management server providing support for content browsed by a user using the terminal devices 101, 102, 103 (only as an example). The background management server can analyze and process received user requests and the like, and feed back the processing results (such as web pages, information, or data, and the like obtained or generated according to user requests) to the terminal devices.

[0033] It should be noted that the named entity recognition method provided by the embodiments of the present disclosure can generally be executed by the terminal devices 101, 102, or 103. Accordingly, the named entity recognition apparatus provided by the embodiments of the present disclosure can also be arranged in the terminal devices 101, 102, or 103.

[0034] Alternatively, the named entity recognition method provided by the embodiments of the present disclosure can also be generally executed by the server 105. Accordingly, the named entity recognition apparatus provided by the embodiments of the present disclosure can generally be arranged in the server 105. The named entity recognition method provided by the embodiments of the present disclosure can also be executed by a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the named entity recognition apparatus provided by the embodiments of the present disclosure can also be arranged in a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, 103 and / or the server 105.

[0035] For example, when a user performs an online search, the terminal device 101, 102, 103 can obtain search information input by the user, and then send the obtained search information to the server 105, and the server 105 performs semantic analysis on the search information to determine the named entity related to the search information. Or the server or server cluster capable of communicating with the terminal device 101, 102, 103 and / or the server 105 performs semantic analysis on the search information and determines the named entity related to the search information.

[0036] It should be understood that Figure 1 The number of terminal devices, networks, and servers in the above description is only illustrative. Any number of terminal devices, networks, and servers can be provided according to the needs of implementation.

[0037] Figure 2 A flowchart of a method for training a named entity recognition model according to an embodiment of the present disclosure is schematically shown.

[0038] As Figure 2 shown, the method includes operations S210-S240.

[0039] In operation S210, a training sample is determined, wherein the training sample includes a sample text and a to-be-confirmed sample annotation result sequence.

[0040] In operation S220, the sample text is processed using an initial named entity recognition model to obtain a sample label confidence set sequence.

[0041] In operation S230, based on the sample label confidence set sequence and the to-be-confirmed sample annotation result sequence, a target sample annotation result sequence of the sample text is determined.

[0042] In operation S240, the parameters of the initial named entity recognition model are adjusted using the sample label confidence set sequence and the target sample annotation result sequence to obtain a named entity recognition model.

[0043] According to an embodiment of the present disclosure, the training sample can include a sample text and an annotated to-be-confirmed sample annotation result sequence. The initial named entity recognition model can be trained using the training sample, so that the trained named entity recognition model can recognize the named entity of a predefined category from the to-be-recognized text. For example, the named entity includes a person name, a place name, an organization name, a proper noun, etc.

[0044] According to an embodiment of the present disclosure, the to-be-confirmed sample annotation result sequence can be annotation data obtained by a remote supervision method, such as annotation data obtained by a rule, dictionary matching method. It can also be annotation data obtained by manual annotation method. As long as it is a way to obtain a sample text for representing a named entity category result.

[0045] According to an embodiment of the present disclosure, an initial named entity recognition model can be used to process sample text to obtain a sample label confidence set sequence, which is then used to verify the sequence of sample annotation results to be confirmed, thereby obtaining a target sample annotation result sequence. The target sample annotation result sequence is used as the true annotation result of the sample text, and the sample label confidence set sequence is used as the predicted result. The true annotation results and the predicted results are used to adjust the parameters of the initial named entity recognition model, thereby obtaining a named entity recognition model.

[0046] Using the training method for the named entity recognition model provided by the embodiments of the present disclosure, the prediction results obtained by the initial named entity recognition model, namely the sample label confidence set sequence, can be used to correct the sequence of unconfirmed sample annotation results. The confirmed target sample annotation result sequence is used as the true annotation result, that is, the confidence comparison estimation mechanism is used to calibrate the unconfirmed sample annotation results, thereby improving the accuracy of the training samples. This solves the problem of a large amount of noise in the unconfirmed annotation result sequence obtained using remote supervision annotation data.

[0047] According to embodiments of the present disclosure, the presence of noise in training samples generally refers to errors in the sequence of annotation results for the unverified samples. For example, the sequence of annotation results for the unverified samples may contain one or more of entity boundary noise, entity nesting noise, and entity type noise. However, this is not limiting. Noise in training samples may also include missing label noise.

[0048] According to an embodiment of the present disclosure, with respect to operation S210 , the following operations may be performed to reduce the label omission noise of the training samples.

[0049] For example, an initial training sample is obtained. The initial training sample may include a sample text such as "Little A is going to BB Primary School to participate in a track and field sports meet," and an initial sample annotation result sequence such as "Little (B-PER) A (I-PER) is going (O) to (O) B (B-LOC) B (I-LOC) Primary (I-LOC) School (I-LOC) to (O) participate (O) in (O) track and field (O) sports (O) meet (O)." The sample text includes at least one sample character, and the initial sample annotation result sequence may be, for example, the initial sample annotation result in brackets that indicates whether it is a named entity. The at least one sample character corresponds one-to-one to the at least one initial sample annotation result.

[0050] According to an embodiment of the present disclosure, the sample types may include positive sample types and negative sample types. Among them, the initial sample annotation result is that the sample characters used to represent named entity categories, such as B-PER, I-PER, B-LOC, I-LOC, etc., are positive sample types, and the initial sample annotation result is that the sample characters used to represent non-named entities, such as O, are negative sample types. For each sample character in at least one sample character, the sample type of the sample character may be determined based on the initial sample annotation result.

[0051] According to an embodiment of the present disclosure, based on the sample types, at least one target sample character is determined from at least one sample character. For each target sample character in at least one target sample character, the initial sample annotation result corresponding to the target sample character is updated using a predetermined sample annotation result. The predetermined sample annotation result of each of the at least one target sample character and the initial sample annotation result except the predetermined sample annotation result of each of the at least one target sample character are used as the待确认样本标注结果序列 (to-be-confirmed sample annotation result sequence).

[0052] According to an embodiment of the present disclosure, the target sample character may be a sample character of a positive sample type or a sample character of a negative sample type. All sample characters of the negative sample type in the sample text may be used as target sample characters, or some sample characters of the negative sample type in the sample text may be used as target sample characters. For example, "参(O)加(O)" is used as a target sample character. The predetermined sample annotation result "参(N)加(N)" may be used to update the target sample character. The sample text "小A要去BB小学参加田径运动会" and a training sample such as "小(B-PER)A(I-PER)要(O)去(O)B(B-LOC)B(I-LOC)小(I-LOC)学(I-LOC)参(N)加(N)田(O)径(O)运(O)动(O)会(O)" of the待确认样本标注结果序列 (to-be-confirmed sample annotation result sequence) are obtained.

[0053] According to an embodiment of the present disclosure, in the process of training the initial named entity recognition model using the training sample, it may only involve using sample characters of positive sample types to train the initial named entity recognition model, but it is not limited thereto. The negative sampling mechanism may also be used to update some sample characters of the negative sample type in the sample text to target sample characters, and then use the sample characters of positive sample types and the target sample characters of negative sample types to jointly train the initial named entity recognition model.

[0054] The initial sample annotation result of part of the negative sample characters can be randomly updated as the to-be-confirmed sample annotation result by using the manner of the negative sampling mechanism of the embodiment of the present disclosure, for example, the “O” is updated by using the newly added predetermined sample annotation result “N” for representing the negative sample character. The initial named entity recognition model is trained by using the training sample including the negative sample character, so as to avoid the problem that the missing sample character cannot participate in the training of the initial named entity recognition model due to being annotated as “O”. In this way, the risk that the initial sample annotation result of the negative sample includes the missing entity is reduced, and thus the problem of data missing annotation is solved.

[0055] According to an embodiment of the present disclosure, the sample label confidence set sequence can include at least one sample label confidence set, and at least one sample character corresponds to at least one sample label confidence set in a one-to-one manner. The to-be-confirmed sample annotation result sequence can include at least one to-be-confirmed sample annotation result, and at least one sample character corresponds to at least one to-be-confirmed sample annotation result in a one-to-one manner.

[0056] According to an embodiment of the present disclosure, for operation S230, based on the sample label confidence set sequence and the to-be-confirmed sample annotation result sequence, determining the target sample annotation result sequence of the sample text can be performed by the following operations.

[0057] For example, for each sample character in the at least one sample character, based on the to-be-confirmed sample annotation result corresponding to the sample character in the to-be-confirmed sample annotation result sequence, determining the sample type of the sample character. In response to the sample type of the sample character being a negative sample type, determining the target sample annotation result of the sample character as a second target sample annotation result for representing a non-named entity.

[0058] According to an embodiment of the present disclosure, the sample character of the negative sample type can refer to the sample character whose to-be-confirmed sample annotation result is “N”. For the sample character of the negative sample type, the target sample annotation result of the sample character can be determined as the second target sample annotation result for representing a non-named entity, for example, the second target sample annotation result for representing a non-named entity is the label “N” and the value is 1.

[0059] According to an embodiment of the present disclosure, for each sample character in the at least one sample character, based on the to-be-confirmed sample annotation result corresponding to the sample character in the to-be-confirmed sample annotation result sequence, determining the sample type of the sample character. In response to the sample type of the sample character being a negative sample type, determining the target sample annotation result of the sample character as a second target sample annotation result for representing a non-named entity.

[0060] According to an embodiment of the present disclosure, the sample label confidence set can include a set of confidences of labels of a plurality of categories. For example, the sample label confidence set can include a confidence for characterizing a non-named entity label "N" and "O", a confidence for characterizing a named entity label "B-PER" of a start position-person category, a confidence for characterizing a named entity label "I-PER" of a middle position-person category, a confidence for characterizing a named entity label "B-LOC" of a start position-place category, and a confidence for characterizing a named entity label "I-LOC" of a middle position-place category. The first target sample annotation result can be determined based on the sample annotation result to be confirmed, and the confidence of "B-PER", the confidence of "I-PER", the confidence of "B-LOC", and the confidence of "I-LOC" in the sample label confidence set.

[0061] According to an embodiment of the present disclosure, the target position confidence can refer to a result related to the confidence of the position label in the sample label confidence set. For example, the sample annotation result to be confirmed is "B-PER", and the target position confidence can refer to the confidence of "B-PER" in the sample label confidence set, which is the sum of the confidence of "B-PER" and the confidence of "I-PER", or the confidence of "B-PER" in the single sample label confidence set. The sum of the confidence of "B-LOC" The sum of the confidence of "B-LOC" The sum of the confidence of "B-LOC" L .

[0062] ; (1)

[0064] According to an embodiment of the present disclosure, the target category confidence can refer to a result related to the confidence of the category label in the sample label confidence set. For example, the sample annotation result to be confirmed is "B-PER", and the target category confidence can refer to the sum of the confidence of "B-PER" and the confidence of "I-PER" in the sample label confidence set, or the confidence of "B-PER" in the single sample label confidence set. It can also be the category confidence S C .

[0065] ; (2)

[0067] According to an embodiment of the present disclosure, based on the target position confidence and the target category confidence, the target sample annotation result of the sample character is determined to be the first target sample annotation result for characterizing the named entity.

[0068] For example, when the target category confidence S CGreater than the target position confidence S L In this case, the labels "B-PER" and "I-PER" related to the category can both be determined as 1, and other labels are determined as 0, serving as the first target sample annotation result. When the target category confidence S C is less than the target position confidence S L In this case, the labels "B-LOC" and "B-PER" related to the position can both be determined as 1, and other labels are determined as 0, serving as the first target sample annotation result.

[0069] Training samples containing, for example, entity boundary noise, entity nesting noise, entity type noise, etc. can be utilized, and the target position confidence and target category confidence are used to correct the accuracy of the annotation result of the sample to be confirmed, so that the named entity recognition model is trained using the training samples including noise, improving the transferability and robustness of the trained named entity recognition model.

[0070] Figure 3 Schematically shows a model architecture diagram of an initial named entity recognition model according to an embodiment of the present disclosure.

[0071] As Figure 3 shown, the initial named entity recognition model may include an initial feature extraction module 310. The initial feature extraction module 310 can be used to process the sample text 320 to obtain a sample feature vector sequence 330.

[0072] According to an embodiment of the present disclosure, the initial feature extraction module can adopt ERNIE (Enhanced Representation through Knowledge Integration, enhancing language representation using information entities), but is not limited thereto, and BERT (Bidirectional Encoder Representations from Transformers, bidirectional encoder representations from transformers) can also be adopted. As long as it is an initial feature extraction module that can extract features in the sample text and obtain a sample feature vector sequence. Adopting ERNIE is more conducive to extracting Chinese text and can effectively extract the features of Chinese characters with multiple meanings.

[0073] According to an embodiment of the present disclosure, the sample text includes at least one sample character, the sample feature vector sequence includes at least one sample feature vector, and at least one sample character corresponds to at least one sample feature vector one by one. For example, the sample character "Xiao" corresponds to the sample feature vector "E0", and the sample character "A" corresponds to the sample feature vector "E1", which will not be elaborated here.

[0074] As Figure 3As shown, the initial named entity recognition model can further include an initial global confidence evaluation module 340 and an initial local confidence evaluation module 350. The initial global confidence evaluation module 340 can be used to process the sample feature vector sequence 330 to obtain a global sample label confidence set sequence. The initial local confidence evaluation module 350 can be used to process the sample feature vector sequence 330 to obtain a local sample label confidence set sequence. The sample label confidence set sequence can be obtained based on the global sample label confidence set sequence and the local sample label confidence set sequence.

[0075] According to an embodiment of the present disclosure, the global sample label confidence set sequence includes at least one global sample label confidence set, and the at least one global sample label confidence set corresponds to at least one sample character in a one-to-one manner. The local sample label confidence set sequence includes at least one local sample label confidence set, and the at least one local sample label confidence set corresponds to at least one sample character in a one-to-one manner.

[0076] According to an embodiment of the present disclosure, the sample label confidence set sequence can be determined based on the global sample label confidence set sequence and the local sample label confidence set sequence. The sample label confidence set sequence can include at least one sample label confidence set, and the at least one sample label confidence set corresponds to at least one sample character in a one-to-one manner.

[0077] According to an embodiment of the present disclosure, the global sample label confidence in the global sample label confidence set and the local sample label confidence in the local sample label confidence set can be compared, and the result value with the maximum or minimum value can be taken as the sample label confidence, but it is not limited thereto. In addition, a weighted average method can be used, different weights can be configured for the global sample label confidence and the local sample label confidence, and a weighted sum can be taken as the sample label confidence.

[0078] For example, the sample label confidence in each sample label confidence set The sample label confidence can be determined by formula (3).

[0079] ; (3)

[0080] wherein, represents the global sample label confidence, represents the local sample label confidence, represents the i-th label category sample character.

[0081] For example, taking the first sample character "B" in "BB Elementary School" as an example and combining Figure 3Specific description: The sample feature vector sequence is processed using the initial global confidence evaluation module to obtain a global sample label confidence set for the sample character "B". For example, the global sample label confidence of the label "N" is 0.341, the global sample label confidence of the label "B-PER" is 0.812, the global sample label confidence of the label "I-PER" is 0.246, the global sample label confidence of the label "B-LOC" is 0.742, and the global sample label confidence of the label "I-LOC" is 0.757. The sample feature vector sequence is processed using the initial local confidence evaluation module to obtain the local sample label confidence set of the sample character "B". For example, the local sample label confidence of the label "N" is 0.215, the local sample label confidence of the label "B-PER" is 0.612, the local sample label confidence of the label "I-PER" is 0.546, the local sample label confidence of the label "B-LOC" is 0.912, and the local sample label confidence of the label "I-LOC" is 0.757.

[0082] Based on the above results, we can determine that the sample label confidence of label “N” in the sample label confidence set is 0.278, the sample label confidence of label “B-PER” is 0.712, the sample label confidence of label “I-PER” is 0.396, the sample label confidence of label “B-LOC” is 0.872, and the sample label confidence of label “I-LOC” is 0.757. Then the target position confidence S L is 1.584, the target category confidence S C It is 1.629.

[0083] Compare the target location confidence S1 with the target category confidence S2 to determine the target location confidence S L Greater than the target category confidence S C The target sample labeling result may be that the first target sample labeling result for characterizing the named entity is: label “B-LOC”, value 1, and label “I-LOC”, value 1.

[0084] It should be noted that the initial named entity recognition model may include only the cascaded initial feature extraction module and the initial global confidence evaluation module, or only the cascaded initial feature extraction module and the initial local confidence evaluation module, or may include the initial feature extraction module, the parallel initial global confidence evaluation module and the initial local confidence evaluation module.

[0085] According to an embodiment of the present disclosure, the initial global confidence evaluation module may adopt CRF (Conditional Random Field), but is not limited thereto, and other confidence evaluation modules for named entity recognition may also be adopted.

[0086] According to an embodiment of the present disclosure, the initial local confidence evaluation module can employ a Softmax (activation function), but is not limited thereto, and can employ other activation functions as the local confidence evaluation module.

[0087] According to an embodiment of the present disclosure, the initial global confidence evaluation module can be utilized to learn the correlation between each sample character in the sample text, and better reflect the strong dependency between each target sample annotation result in the target sample annotation result sequence. For example, after the target sample annotation result of the current sample character is “O” label, the probability of appearing “B” label is greater than the probability of appearing “I” label, because “I” label cannot appear as the beginning of a named entity. In addition, the initial local confidence evaluation module can be utilized to learn the local semantic information of each sample character in the sample text. Thus, by utilizing the initial named entity recognition model provided by the embodiment of the present disclosure, which has the initial global confidence evaluation module and the initial local confidence evaluation module in parallel, the correlation between each sample character can be reflected, and the local semantic information of each sample character can be learned, so that the result of the final sample label confidence set sequence is more accurate and effective.

[0088] According to an embodiment of the present disclosure, for operation S230, adjusting the parameters of the initial named entity recognition model by utilizing the sample label confidence set sequence and the target sample annotation result sequence to obtain the named entity recognition model can include the following operations.

[0089] For example, based on the sample label confidence set sequence, the sample named entity label sequence is determined. Taking one sample label confidence set in the sample label confidence set sequence as an example, the sample named entity label corresponding to the sample label confidence set in the sample named entity label sequence is determined. For example, the sample label confidence of the label “N” in the sample label confidence set is 0.278, the sample label confidence of the label “B-PER” is 0.712, the sample label confidence of the label “I-PER” is 0.396, the sample label confidence of the label “B-LOC” is 0.872, and the sample label confidence of the label “I-LOC” is 0.757. Then, the sample named entity label is “B-LOC” and the value is 0.872.

[0090] The sample named entity label sequence can be used as the predicted result of the sample text, and the target sample annotation result sequence can be used as the true result of the sample text. The sample named entity label sequence and the target sample annotation result sequence can be used to adjust the parameters of the initial named entity recognition model to obtain a named entity recognition model. For example, the sample named entity label sequence and the target sample annotation result sequence are input into the cross-entropy loss function to obtain a loss value. Based on the loss value, the parameters of the initial named entity recognition model are adjusted until the loss value converges. The model with converged loss value is used as the named entity recognition model.

[0091] According to an embodiment of the present disclosure, an optimization training method, such as a self-training method, may be used to train a model using training samples including noise.

[0092] For example, the optimized sample text is input into the named entity recognition model to obtain an optimized sample named entity label sequence. The optimized sample text and the optimized sample named entity label sequence are used to optimize and train the named entity recognition model to obtain an optimized named entity recognition model.

[0093] According to an embodiment of the present disclosure, multiple training samples in an initial training sample set can be divided into two parts. One part of the initial training sample set is used as a training sample set to train an initial named entity recognition model to obtain a named entity recognition model. The other part of the initial training sample set is used as an optimized training sample set to train a named entity recognition model to obtain an optimized named entity recognition model. Training the model using multiple rounds of training can improve the training accuracy of the optimized named entity recognition model.

[0094] According to an embodiment of the present disclosure, the optimized training samples in the optimized training sample set may include optimized samples and a sequence of labeling results of samples to be confirmed corresponding to the optimized samples. The optimized samples and the sequence of labeling results of samples to be confirmed corresponding to the optimized samples can be directly used to train the named entity recognition model to obtain an optimized named entity recognition model. However, it is not limited to this. It is also possible to use a cross-validation training method to re-label the optimized samples using the named entity recognition model to obtain an optimized sample named entity label sequence. For example, the optimized samples are input into the named entity recognition model to obtain an optimized sample named entity label sequence. The named entity recognition model is trained using the optimized sample text and the optimized sample named entity label sequence to obtain an optimized named entity recognition model to improve the convergence speed of the model.

[0095] Figure 4 The flowchart of the named entity recognition method according to an embodiment of the present disclosure is schematically shown.

[0096] like Figure 4 As shown, the method includes operations S410 to S420.

[0097] In operation S410, the text to be recognized is obtained.

[0098] In operation S420, the text to be recognized is input into the named entity recognition model to obtain a named entity label sequence of the text to be recognized, where the text to be recognized includes at least one character, the named entity label sequence includes at least one named entity label, the at least one character and the at least one named entity label correspond to each other, and the named entity label is used to indicate whether the character is a named entity; and the named entity recognition model is trained by using the training method of the named entity recognition model.

[0099] According to an embodiment of the present disclosure, the named entity recognition method can include operations S410 and S420, but is not limited thereto, and can only include operation S420.

[0100] According to an embodiment of the present disclosure, named entity recognition (NER) is also called named recognition, which is a basic task in natural language processing, and can identify words, phrases and segments belonging to predefined categories, i.e., named entities, from the text to be recognized.

[0101] According to an embodiment of the present disclosure, the named entity recognition model provided by the embodiment of the present disclosure can have better industry transferability when processing the text to be recognized, and can realize the recognition of named entities of various types and professional texts to be recognized, and improve the recognition accuracy of named entities.

[0102] According to an embodiment of the present disclosure, the named entity recognition model can include a feature extraction module. The feature extraction module can be used to obtain a feature vector sequence from the text to be recognized.

[0103] According to an embodiment of the present disclosure, the feature extraction module can use ERNIE, but is not limited thereto, and can also use BERT. As long as the feature extraction module can extract features in the text to be recognized to obtain a feature vector sequence, it can be used. Using ERNIE is more conducive to extracting Chinese text and can effectively extract the features of Chinese characters with multiple meanings.

[0104] According to an embodiment of the present disclosure, the text to be recognized includes at least one character, the feature vector sequence includes at least one feature vector, and the at least one character and the at least one feature vector correspond to each other.

[0105] According to an embodiment of the present disclosure, the named entity recognition model can further include a global confidence evaluation module and a local confidence evaluation module. The global confidence evaluation module can be used to process the sequence of feature vectors to obtain a sequence of global label confidence sets. The local confidence evaluation module can be used to process the sequence of feature vectors to obtain a sequence of local label confidence sets. The sequence of label confidence sets can be obtained based on the sequence of global label confidence sets and the sequence of local label confidence sets.

[0106] According to an embodiment of the present disclosure, the global confidence evaluation module can employ CRF, but is not limited thereto, and can employ other confidence evaluation modules for named entity recognition.

[0107] According to an embodiment of the present disclosure, the local confidence evaluation module can employ Softmax, but is not limited thereto, and can employ other activation functions as the local confidence evaluation module.

[0108] According to an embodiment of the present disclosure, the sequence of global label confidence sets includes at least one global label confidence set, and the at least one global label confidence set corresponds to at least one character. The sequence of local label confidence sets includes at least one local label confidence set, and the at least one local label confidence set corresponds to at least one character.

[0109] According to an embodiment of the present disclosure, the sequence of label confidence sets can be determined based on the sequence of global label confidence sets and the sequence of local label confidence sets. The sequence of label confidence sets can include at least one label confidence set, and the at least one label confidence set corresponds to at least one character.

[0110] According to an embodiment of the present disclosure, the global label confidence in the global label confidence set and the local label confidence in the local label confidence set can be compared, and the result value with the maximum or minimum value can be taken as the label confidence, but is not limited thereto, and a weighted average method can be used, in which different weights are configured for the global label confidence and the local label confidence, and a weighted sum is taken as the label confidence.

[0111] According to an embodiment of the present disclosure, the sequence of named entity labels of the text to be recognized can be determined based on the sequence of label confidence sets. For example, for each label confidence set in the sequence of label confidence sets, a named entity label corresponding to the label confidence set in the sequence of named entity labels can be determined based on the label confidence set. The maximum label confidence in the label confidence set can be taken as the named entity label, but is not limited thereto, and can be set according to actual conditions.

[0112] For example, the label confidence in each label confidence set The label confidence set can be determined by formula (4).

[0113] (4)

[0114] wherein, represents the global label confidence, represents the local label confidence, represents the i-th label class, character.

[0115] For example, taking the first character "B" in "BB primary school" as an example. The global label confidence set of the character "B" is obtained by processing the feature vector sequence by using the global confidence evaluation module, for example, the global label confidence of label "N" is 0.341, the global label confidence of label "B-PER" is 0.812, the global label confidence of label "I-PER" is 0.246, the global label confidence of label "B-LOC" is 0.742, and the global label confidence of label "I-LOC" is 0.757. The local label confidence set of the character "B" is obtained by processing the feature vector sequence by using the local confidence evaluation module, for example, the local label confidence of label "N" is 0.215, the local label confidence of label "B-PER" is 0.612, the local label confidence of label "I-PER" is 0.546, the local label confidence of label "B-LOC" is 0.912, and the local label confidence of label "I-LOC" is 0.757.

[0116] Based on the above results, it can be determined that the label confidence of label "N" in the label confidence set is 0.278, the label confidence of label "B-PER" is 0.712, the label confidence of label "I-PER" is 0.396, the label confidence of label "B-LOC" is 0.872, and the label confidence of label "I-LOC" is 0.757.

[0117] Therefore, by the above calculation, it can be determined that the named entity label for the character "B" is "B-LOC".

[0118] According to an embodiment of the present disclosure, a global confidence evaluation module can be utilized to determine the correlation between each character in the to-be-recognized text, and better reflect the strong dependency between each named entity label in the named entity label sequence. For example, after the named entity label of the current character is an“O” label, the probability of the occurrence of a“B” label is greater than the probability of the occurrence of an“I” label, because an“I” label cannot appear as the beginning of a named entity. In addition, a local confidence evaluation module can be utilized to learn the local semantic information of each character in the text. Thus, by utilizing the named entity recognition model provided by the embodiment of the present disclosure, which has a global confidence evaluation module and a local confidence evaluation module in parallel, the correlation between each character can be reflected, and the local semantic information of each character can be learned, so that the prediction of the final named entity label sequence is more accurate and effective.

[0119] Figure 5 A flowchart of a named entity recognition method according to another embodiment of the present disclosure is schematically shown.

[0120] As shown in Figure 5 , the operations S420, S510-S530 are included.

[0121] In operation S420, the to-be-recognized text is input into the named entity recognition model to obtain a named entity label sequence of the to-be-recognized text.

[0122] In operation S510, based on the named entity label sequence, a first entity set is determined from the to-be-recognized text.

[0123] In operation S520, according to a predetermined named entity matching rule, a second entity set is determined from the to-be-recognized text.

[0124] In operation S530, based on the first entity set and the second entity set, a target entity set is determined.

[0125] According to an embodiment of the present disclosure, the predetermined named entity matching rule can be named entity matching by utilizing a predetermined named entity dictionary. The predetermined named entity dictionary can refer to a dictionary of mapping relationships between predetermined named entities and categories of named entities. The to-be-recognized text with professional field named entities can be processed according to the predetermined named entity dictionary, such as a chemical compound product in the chemical field as a named entity, a gene sequence named named entity in the biological field, or a drug name in the traditional Chinese medicine field as a named entity, etc. The predetermined named entity dictionary can be utilized to break through the limitation of the industry field by the content understanding task, and improve the named entity recognition of professional vocabulary and characters.

[0126] According to an embodiment of the present disclosure, in a case where the first entity set and the second entity set are determined, the first entity set and the second entity set can be processed by taking a union set to determine the target entity set. However, it is not limited thereto. The first entity set and the second entity set can also be processed by taking an intersection set to determine the target entity set. The specific processing manner can be determined according to actual conditions.

[0127] According to an embodiment of the present disclosure, in a case where the first entity set and the second entity set are determined, the first entity set and the second entity set can be processed by taking a union set to determine the target entity set. However, it is not limited thereto. The first entity set and the second entity set can also be processed by taking an intersection set to determine the target entity set. The specific processing manner can be determined according to actual conditions.

[0128] Figure 6 A block diagram of a training apparatus of a named entity recognition model according to an embodiment of the present disclosure is schematically shown.

[0129] As shown in Figure 6 The training apparatus 600 of the named entity recognition model can include a first determination module 610, a processing module 620, a second determination module 630, and an adjustment module 640.

[0130] The first determination module 610 is configured to determine a training sample, wherein the training sample includes a sample text and a to-be-confirmed sample annotation result sequence.

[0131] The processing module 620 is configured to process the sample text by using an initial named entity recognition model to obtain a sample label confidence set sequence.

[0132] The second determination module 630 is configured to determine a target sample annotation result sequence of the sample text based on the sample label confidence set sequence and the to-be-confirmed sample annotation result sequence.

[0133] The adjustment module 640 is configured to adjust parameters of the initial named entity recognition model by using the sample label confidence set sequence and the target sample annotation result sequence to obtain the named entity recognition model.

[0134] According to an embodiment of the present disclosure, the first determination module can include an obtaining unit, a first determination unit, a second determination unit, an updating unit, and a third determination unit.

[0135] The obtaining unit is configured to obtain an initial training sample, wherein the initial training sample includes a sample text and an initial sample annotation result sequence, the sample text includes at least one sample character, and the initial sample annotation result sequence includes at least one initial sample annotation result, and the at least one sample character and the at least one initial sample annotation result are in one-to-one correspondence.

[0136] The first determining unit is configured to determine, for each of the at least one sample character, a sample type of the sample character based on the initial sample annotation result.

[0137] The second determining unit is configured to determine, based on the sample type, at least one target sample character from the at least one sample character.

[0138] The updating unit is configured to update, for each of the at least one target sample character, the initial sample annotation result corresponding to the target sample character by using the predetermined sample annotation result.

[0139] The third determining unit is configured to take, as a sequence of to-be-confirmed sample annotation results, the predetermined sample annotation result of each of the at least one target sample character and the initial sample annotation result other than the predetermined sample annotation result of each of the at least one target sample character.

[0140] According to an embodiment of the present disclosure, the sequence of sample label confidence sets includes at least one sample label confidence set, and the at least one sample character is in one-to-one correspondence with the at least one sample label confidence set; the sequence of to-be-confirmed sample annotation results includes at least one to-be-confirmed sample annotation result, and the at least one sample character is in one-to-one correspondence with the at least one to-be-confirmed sample annotation result.

[0141] According to an embodiment of the present disclosure, the second determining module can include a fourth determining unit, a fifth determining unit, and a sixth determining unit.

[0142] The fourth determining unit is configured to determine, for each of the at least one sample character, a sample type of the sample character based on the to-be-confirmed sample annotation result.

[0143] The fifth determining unit is configured to, in response to the sample type being a positive sample type, determine, based on the sequence of sample label confidence sets, a target sample annotation result of the sample character as a first target sample annotation result for representing a named entity.

[0144] The sixth determining unit is configured to, in response to the sample type being a negative sample type, determine the target sample annotation result of the sample character as a second target sample annotation result for representing a non-named entity.

[0145] According to an embodiment of the present disclosure, the fifth determining unit can include a first determining subunit and a second determining subunit.

[0146] The first determining subunit is configured to determine, based on the sequence of sample label confidence sets, a target position confidence and a target category confidence of the sample character.

[0147] The second determining subunit is configured to determine, based on the target position confidence and the target category confidence, the target sample annotation result of the sample character as a first target sample annotation result used for representing a named entity.

[0148] According to an embodiment of the present disclosure, the initial named entity recognition model can comprise an initial feature extraction module.

[0149] According to an embodiment of the present disclosure, the training apparatus of the named entity recognition model can further comprise an extraction module.

[0150] The extraction module is configured to process the sample text by using the initial feature extraction module to obtain a sample feature vector sequence.

[0151] According to an embodiment of the present disclosure, the initial named entity recognition model further comprises an initial global confidence evaluation module and an initial local confidence evaluation module.

[0152] According to an embodiment of the present disclosure, the processing module can comprise a first processing unit, a second processing unit, and a seventh determining unit.

[0153] The first processing unit is configured to process the sample feature vector sequence by using the initial global confidence evaluation module to obtain a global sample label confidence set sequence.

[0154] The second processing unit is configured to process the sample feature vector sequence by using the initial local confidence evaluation module to obtain a local sample label confidence set sequence.

[0155] The seventh determining unit is configured to obtain a sample label confidence set sequence based on the global sample label confidence set sequence and the local sample label confidence set sequence.

[0156] According to an embodiment of the present disclosure, the adjustment module can comprise an eighth determining unit and an adjustment unit.

[0157] The eighth determining unit is configured to determine a sample named entity label sequence based on the sample label confidence set sequence.

[0158] The adjustment unit is configured to adjust parameters of the initial named entity recognition model by using the sample named entity label sequence and the target sample annotation result sequence to obtain the named entity recognition model.

[0159] According to an embodiment of the present disclosure, the training apparatus of the named entity recognition model can further comprise an optimization label module and an optimization training module.

[0160] The optimization label module is configured to input the optimization sample text into the named entity recognition model to obtain an optimization sample named entity label sequence.

[0161] The optimization training module is configured to optimize the training of the named entity recognition model by using the optimization sample text and the optimization sample named entity tag sequence, to obtain an optimized named entity recognition model.

[0162] Figure 7 A block diagram of a named entity recognition apparatus according to an embodiment of the present disclosure is shown schematically.

[0163] As shown in Figure 7 The named entity recognition apparatus 700 can include an obtaining module 710 and an input module 720.

[0164] The obtaining module 710 is configured to obtain a text to be recognized.

[0165] The input module 720 is configured to input the text to be recognized into a named entity recognition model, to obtain a named entity tag sequence of the text to be recognized, wherein the text to be recognized includes at least one character, the named entity tag sequence includes at least one named entity tag, the at least one character and the at least one named entity tag correspond to each other in a one-to-one manner, and the named entity tag is used to indicate whether the character is a named entity.

[0166] According to an embodiment of the present disclosure, the named entity recognition model is trained by using a training apparatus of the named entity recognition model.

[0167] According to an embodiment of the present disclosure, the named entity recognition apparatus can include the obtaining module 710 and the input module 720, but is not limited thereto, and the named entity recognition apparatus can also include only the input module 720.

[0168] According to an embodiment of the present disclosure, the named entity recognition apparatus can further include a third determining module, a matching module, and a fourth determining module.

[0169] The third determining module is configured to determine a first entity set from the text to be recognized based on the named entity tag sequence.

[0170] The matching module is configured to determine a second entity set from the text to be recognized according to a predetermined named entity matching rule.

[0171] The fourth determining module is configured to determine a target entity set based on the first entity set and the second entity set.

[0172] According to an embodiment of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium, and a computer program product.

[0173] According to an embodiment of the present disclosure, an electronic device comprises at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0174] According to an embodiment of the present disclosure, a non-transitory computer readable storage medium storing computer instructions, wherein the computer instructions are used to make a computer perform the method as described above.

[0175] According to an embodiment of the present disclosure, a computer program product comprises a computer program which, when executed by a processor, implements the method as described above.

[0176] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present disclosure described and / or claimed in this document.

[0177] As shown in Figure 8 , the device 800 includes a computing unit 801 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 802 or a computer program loaded into a random access memory (RAM) 803 from a storage unit 808. Various programs and data required for the operation of the device 800 can also be stored in the RAM 803. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0178] Various components in the device 800 are connected to the I / O interface 805, including an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0179] The computing unit 801 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, and the like. The computing unit 801 performs various methods and processes described above, such as a named entity recognition method or a training method of a named entity recognition model. For example, in some embodiments, the named entity recognition method or the training method of a named entity recognition model can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded onto the RAM 803 and executed by the computing unit 801, one or more steps of the named entity recognition method or the training method of a named entity recognition model described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the named entity recognition method or the training method of a named entity recognition model by any other appropriate means, such as by means of firmware.

[0180] Various implementations of the systems and techniques described above herein can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip system (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0181] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0182] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0183] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0184] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0185] The computer system can include clients and servers. This relationship can be. The servers are typically remote from the clients with the interactions between them occurring over a communication network. The relationship between a client and a server is one of client-server. The server can be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

[0186] It should be understood that the various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the scope of the present disclosure. For example, the steps recited in the present disclosure can be executed in parallel, executed in series, or executed in different orders, as long as the desired results of the technical solutions of the present disclosure are achieved, and the present disclosure is not limited herein.

[0187] The specific implementation described above does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements within the spirit and principles of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. A method for training a named entity recognition model, comprising: determining a training sample, wherein the training sample comprises a sample text and a to-be-confirmed sample annotation result sequence; processing the sample text by using an initial named entity recognition model to obtain a sample label confidence set sequence; determining a target sample annotation result sequence of the sample text based on the sample label confidence set sequence and the to-be-confirmed sample annotation result sequence; and adjusting parameters of the initial named entity recognition model by using the sample label confidence set sequence and the target sample annotation result sequence to obtain a named entity recognition model; the sample text comprises at least one sample character; the sample label confidence set sequence comprises at least one sample label confidence set, the at least one sample character corresponds to the at least one sample label confidence set one by one, the to-be-confirmed sample annotation result sequence comprises at least one to-be-confirmed sample annotation result, and the at least one sample character corresponds to the at least one to-be-confirmed sample annotation result one by one; the determining of the target sample annotation result sequence of the sample text based on the sample label confidence set sequence and the to-be-confirmed sample annotation result sequence comprises: for each sample character in the at least one sample character, determining a sample type of the sample character based on a corresponding to-be-confirmed sample annotation result; in response to the sample type being a positive sample type, determining a target sample annotation result of the sample character as a first target sample annotation result for representing a named entity based on a corresponding sample label confidence set; and in response to the sample type being a negative sample type, determining the target sample annotation result of the sample character as a second target sample annotation result for representing a non-named entity.

2. The method of claim 1, wherein, the determining of the training sample comprises: obtaining an initial training sample, wherein the initial training sample comprises a sample text and an initial sample annotation result sequence, the sample text comprises at least one sample character, the initial sample annotation result sequence comprises at least one initial sample annotation result, and the at least one sample character corresponds to the at least one initial sample annotation result one by one; for each sample character in the at least one sample character, determining a sample type of the sample character based on the initial sample annotation result; determining at least one target sample character from the at least one sample character based on the sample type; for each target sample character in the at least one target sample character, updating an initial sample annotation result corresponding to the target sample character by using a predetermined sample annotation result; and taking the predetermined sample annotation result of each of the at least one target sample character and the initial sample annotation result except the predetermined sample annotation result of each of the at least one target sample character as the to-be-confirmed sample annotation result sequence.

3. The method of claim 1, wherein, the determining of the target sample annotation result of the sample character as the first target sample annotation result for representing a named entity based on the corresponding sample label confidence set in response to the sample type being a positive sample type comprises: determine a target position confidence and a target category confidence of the sample character based on the corresponding sample label confidence set; and determine a target sample annotation result of the sample character as a first target sample annotation result for representing a named entity based on the target position confidence and the target category confidence.

4. The method of any one of claims 1 to 3, wherein, The initial named entity recognition model comprises an initial feature extraction module; Further comprising: processing the sample text by using the initial feature extraction module to obtain a sample feature vector sequence.

5. The method of claim 4, wherein, The initial named entity recognition model further comprises an initial global confidence evaluation module and an initial local confidence evaluation module; The processing of the sample text by using the initial named entity recognition model to obtain a sample label confidence set sequence comprises: processing the sample feature vector sequence by using the initial global confidence evaluation module to obtain a global sample label confidence set sequence; processing the sample feature vector sequence by using the initial local confidence evaluation module to obtain a local sample label confidence set sequence; and obtaining the sample label confidence set sequence based on the global sample label confidence set sequence and the local sample label confidence set sequence.

6. The method of claim 5, wherein, The adjustment of the parameters of the initial named entity recognition model by using the sample label confidence set sequence and the target sample annotation result sequence to obtain a named entity recognition model comprises: determining a sample named entity label sequence based on the sample label confidence set sequence; and adjusting the parameters of the initial named entity recognition model by using the sample named entity label sequence and the target sample annotation result sequence to obtain a named entity recognition model.

7. The method according to any one of claims 1, 2, 3, 5 and 6, further comprising: inputting an optimized sample text into the named entity recognition model to obtain an optimized sample named entity label sequence; and optimizing the training of the named entity recognition model by using the optimized sample text and the optimized sample named entity label sequence to obtain an optimized named entity recognition model.

8. A named entity recognition method, comprising: inputting a to-be-recognized text into a named entity recognition model to obtain a named entity label sequence of the to-be-recognized text, wherein the to-be-recognized text comprises at least one character, the named entity label sequence comprises at least one named entity label, the at least one character corresponds to the at least one named entity label one by one, and the named entity label is used to indicate whether the character is a named entity; wherein the named entity recognition model is trained by using the method according to any one of claims 1 to 7.

9. The named entity recognition method according to claim 8, further comprising: determining a first entity set from the to-be-recognized text based on the named entity label sequence; determining a second entity set from the to-be-recognized text according to a predetermined named entity matching rule; and determining a target entity set based on the first entity set and the second entity set.

10. A training device of a named entity recognition model, comprising: ​ The first determining module is configured to determine a training sample, wherein the training sample comprises a sample text and a to-be-confirmed sample annotation result sequence; The processing module is configured to process the sample text by using an initial named entity recognition model to obtain a sample label confidence set sequence; The second determining module is configured to determine a target sample annotation result sequence of the sample text based on the sample label confidence set sequence and the to-be-confirmed sample annotation result sequence; and The adjusting module is configured to adjust parameters of the initial named entity recognition model by using the sample label confidence set sequence and the target sample annotation result sequence to obtain a named entity recognition model. The sample text comprises at least one sample character; the sample label confidence set sequence comprises at least one sample label confidence set, the at least one sample character and the at least one sample label confidence set correspond to each other, the to-be-confirmed sample annotation result sequence comprises at least one to-be-confirmed sample annotation result, and the at least one sample character and the at least one to-be-confirmed sample annotation result correspond to each other; The second determining module comprises: The fourth determining unit is configured to, for each sample character in the at least one sample character, determine a sample type of the sample character based on a corresponding to-be-confirmed sample annotation result; The fifth determining unit is configured to, in response to the sample type being a positive sample type, determine, based on a corresponding sample label confidence set, that a target sample annotation result of the sample character is a first target sample annotation result for representing a named entity; and The sixth determining unit is configured to, in response to the sample type being a negative sample type, determine that the target sample annotation result of the sample character is a second target sample annotation result for representing a non-named entity.

11. The apparatus of claim 10, wherein, The first determining module comprises: The obtaining unit is configured to obtain an initial training sample, wherein the initial training sample comprises a sample text and an initial sample annotation result sequence, the sample text comprises at least one sample character, and the initial sample annotation result sequence comprises at least one initial sample annotation result, the at least one sample character and the at least one initial sample annotation result correspond to each other; The first determining unit is configured to, for each sample character in the at least one sample character, determine a sample type of the sample character based on the initial sample annotation result; The second determining unit is configured to determine at least one target sample character from the at least one sample character based on the sample type; The updating unit is configured to, for each target sample character in the at least one target sample character, update an initial sample annotation result corresponding to the target sample character by using a predetermined sample annotation result; and The third determining unit is configured to take the predetermined sample annotation result of each target sample character in the at least one target sample character and the initial sample annotation result except the predetermined sample annotation result of each target sample character in the at least one target sample character as the to-be-confirmed sample annotation result sequence.

12. The apparatus of claim 10, wherein, The fifth determining unit comprises: The first determining subunit is configured to determine a target position confidence and a target category confidence of the sample character based on the corresponding sample label confidence set. The second determining subunit is configured to determine, based on the target position confidence and the target category confidence, the target sample annotation result of the sample character as a first target sample annotation result for representing a named entity.

13. The apparatus of any one of claims 10-12, wherein, The initial named entity recognition model comprises an initial feature extraction module. Further comprising: The extraction module is configured to process the sample text by using the initial feature extraction module to obtain a sample feature vector sequence.

14. The apparatus of claim 13, wherein, The initial named entity recognition model further comprises an initial global confidence evaluation module and an initial local confidence evaluation module. The processing module comprises: The first processing unit is configured to process the sample feature vector sequence by using the initial global confidence evaluation module to obtain a global sample label confidence set sequence. The second processing unit is configured to process the sample feature vector sequence by using the initial local confidence evaluation module to obtain a local sample label confidence set sequence. The seventh determining unit is configured to obtain the sample label confidence set sequence based on the global sample label confidence set sequence and the local sample label confidence set sequence.

15. The apparatus of claim 14, wherein, The adjustment module comprises: The eighth determining unit is configured to determine a sample named entity label sequence based on the sample label confidence set sequence. The adjustment unit is configured to adjust parameters of the initial named entity recognition model by using the sample named entity label sequence and the target sample annotation result sequence to obtain a named entity recognition model.

16. The apparatus according to any one of claims 10, 11, 12, 14 and 15, further comprising: An optimization label module configured to input an optimization sample text into the named entity recognition model to obtain an optimization sample named entity label sequence. And An optimization training module configured to optimize training of the named entity recognition model by using the optimization sample text and the optimization sample named entity label sequence to obtain an optimized named entity recognition model.

17. A named entity recognition apparatus, comprising: An input module configured to input a to-be-recognized text into a named entity recognition model to obtain a named entity label sequence of the to-be-recognized text, wherein the to-be-recognized text comprises at least one character, the named entity label sequence comprises at least one named entity label, the at least one character corresponds to the at least one named entity label one by one, and the named entity label is used to indicate whether the character is a named entity. The named entity recognition model is trained by using the apparatus according to any one of claims 10 to 16.

18. The named entity recognition apparatus according to claim 17, further comprising: A third determining module configured to determine a first entity set from the to-be-recognized text based on the named entity label sequence. A matching module configured to determine a second entity set from the to-be-recognized text according to a predetermined named entity matching rule. And A fourth determining module, configured to determine a target entity set based on the first entity set and the second entity set. 19.An electronic device, comprising: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the training method of the named entity recognition model of any one of claims 1 to 7 or the named entity recognition method of claim 8 or 9.

20. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to perform the training method of the named entity recognition model of any one of claims 1 to 7 or the named entity recognition method of claim 8 or 9. 21.A computer program product, comprising a computer program which, when executed by a processor, implements the training method of the named entity recognition model of any one of claims 1 to 7 or the named entity recognition method of claim 8 or 9.

Images