Named entity recognition method and device, program product and electronic equipment

Abstract

The invention provides a named entity recognition method and device, a program product and electronic equipment, and relates to the technical field of natural language processing. The named entity recognition method comprises the following steps: encoding a target text to obtain text encoding information; decoding the text coding information by using a first decoder to obtain first decoding information; decoding the text coding information by using a multi-head double-affine decoder to obtain second decoding information; and determining a named entity recognition result of the target text according to the first decoding information and the second decoding information. The named entity recognition accuracy is improved.

Description

Technical Field

[0001] This disclosure relates to the field of natural language processing technology, and in particular to a named entity recognition method, named entity recognition device, computer program product, and electronic device. Background Technology

[0002] Named entity recognition (NER) is one of the core tasks in natural language processing, used to identify and classify entities with specific meanings from text, such as names of people, places, organizations, times, and numbers. However, the accuracy of NER still needs improvement in related technologies. Summary of the Invention

[0003] This disclosure provides a named entity recognition method, named entity recognition device, computer program product, and electronic device to improve the accuracy of named entity recognition to at least a certain extent.

[0004] According to a first aspect of this disclosure, a named entity recognition method is provided, the method comprising: encoding target text to obtain text encoding information; decoding the text encoding information using a first decoder to obtain first decoding information; decoding the text encoding information using a multi-head dual affine decoder to obtain second decoding information; and determining a named entity recognition result of the target text based on the first decoding information and the second decoding information.

[0005] According to a second aspect of this disclosure, a named entity recognition apparatus is provided, the apparatus comprising: an encoding module configured to encode target text to obtain text encoding information; a first decoding module configured to decode the text encoding information using a first decoder to obtain first decoding information; a second decoding module configured to decode the text encoding information using a multi-head dual-affine decoder to obtain second decoding information; and a result output module configured to determine a named entity recognition result of the target text based on the first decoding information and the second decoding information.

[0006] According to a third aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method of the first aspect described above and possible implementations thereof.

[0007] According to a fourth aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the method of the first aspect and possible implementations thereof by executing the executable instructions.

[0008] The technical solution disclosed herein has the following beneficial effects:

[0009] On the one hand, by setting up a multi-head dual-affine decoder, different heads can learn different feature relationships in the text. This allows for a comprehensive and thorough learning and mining of textual information, especially in cases where entity boundaries are not obvious. It flexibly captures complex relationships between labels, which is beneficial for accurately identifying entity boundaries and improving the accuracy of the final named entity recognition results. On the other hand, by combining the two decoders, both local and global features in the text can be captured, leading to a more comprehensive understanding of text structure and entity boundaries. This improves the generalization ability of the entire model, enabling it to handle different types of named entity recognition tasks and facilitating its application in a wider range of language scenarios. Attached Figure Description

[0010] Figure 1 A flowchart illustrating a named entity recognition method in this exemplary embodiment is shown.

[0011] Figure 2 A flowchart illustrating another named entity recognition method in this exemplary embodiment is shown.

[0012] Figure 3 A flowchart illustrating another named entity recognition method in this exemplary embodiment is shown.

[0013] Figure 4 A schematic architecture diagram of a multi-head dual affine decoder is shown in this exemplary embodiment.

[0014] Figure 5 A flowchart illustrating another named entity recognition method in this exemplary embodiment is shown.

[0015] Figure 6 A schematic diagram of a named entity recognition method in this exemplary embodiment is shown.

[0016] Figure 7 A schematic diagram of the structure of a named entity recognition device in this exemplary embodiment is shown.

[0017] Figure 8 A schematic diagram of the structure of an electronic device in this exemplary embodiment is shown. Detailed Implementation

[0018] Exemplary embodiments of this disclosure will be described more fully below with reference to the accompanying drawings.

[0019] The accompanying drawings are schematic illustrations of this disclosure and are not necessarily drawn to scale. Some block diagrams shown in the drawings may be functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in hardware modules or integrated circuits, or in networks, processors, or microcontrollers. Implementations can be carried out in various forms and should not be construed as limited to the examples set forth herein. The features, structures, or characteristics described in this disclosure can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough description of embodiments of this disclosure. However, those skilled in the art will recognize that one or more specific details may be omitted when implementing the technical solutions of this disclosure, or other methods, components, apparatuses, steps, etc., may be used to replace one or more specific details.

[0020] In related technologies, a neural network architecture combined with CRF (Conditional Random Fields) is used to achieve named entity recognition. The inventors have discovered that these related technologies have the following problems:

[0021] Named entity recognition requires determining entity boundaries. Algorithms like CRF typically only consider local dependencies between adjacent labels, ignoring broader or longer-range global dependencies. Relying solely on CRF decoders may struggle to accurately capture entity boundaries, especially when entity types are diverse and boundaries are indistinct. The diversity and complexity of language mean that entities may exhibit different forms in different contexts. Machine learning models in related technologies struggle to capture this contextual diversity, limiting their ability to generalize to new contexts.

[0022] The above factors have led to problems such as low accuracy of named entity recognition and difficulty in its widespread application in different language scenarios.

[0023] In view of one or more of the above-mentioned problems, exemplary embodiments of this disclosure provide a named entity recognition method, a named entity recognition device, a computer program product, and an electronic device.

[0024] Based on one embodiment of this disclosure, reference is made to... Figure 1 As shown, the named entity recognition method may include the following steps S110 to S140:

[0025] Step S110: Encode the target text to obtain text encoding information.

[0026] The target text is the text that needs to be named entity recognition, consisting of N words, where N can be any positive integer. Each word can be a Chinese character or an English word (separated by spaces).

[0027] In one implementation, an encoder can be used to encode the target text to obtain text encoding information. For example, the encoder can employ a BERT (Bidirectional Encoder Representations from Transformers)-BiLSTM (Bidirectional Long Short-Term Memory) architecture. After the target text is input into the encoder, it is bidirectionally encoded to output text encoding information.

[0028] In one implementation, each word in the target text can be encoded, and the text encoding information can be a set of encoding information for N words. For example, word embedding processing can be performed on each of the N words in the target text to obtain text encoding information H, as shown below:

[0029]

[0030] Where h1, h2, etc. represent the encoding information of each word (i.e., the word embedding representation), and d represents the encoding dimension of each word, which can be the hidden layer dimension of the encoder.

[0031] Step S120: Use the first decoder to decode the text encoding information to obtain the first decoded information.

[0032] The first decoder can be a model based on any algorithm. For example, the first decoder can be a CRF decoder. The text encoding information is input into the first decoder, processed, and the first decoded information is output.

[0033] The first decoded information can be feature information after further processing of the text encoding information, or it can be prediction information about named entity recognition, such as the first category prediction information, including the probability value of each word in the text corresponding to each category.

[0034] Step S130: Use a multi-head dual affine decoder to decode the text encoding information to obtain the second decoded information.

[0035] A multi-head biaffine decoder is a second decoder, placed in addition to the first decoder. It has at least two heads, each acting as an independent biaffine decoder that uses a biaffine algorithm for decoding. Different heads have different weights, biases, and other parameters, allowing them to learn different feature relationships within the text. For example, when text encoding information is input into a multi-head biaffine decoder, some heads can learn adjacent local dependencies, while others can learn broader or more distant global dependencies. This multi-head setup enables a comprehensive and thorough learning of feature relationships within the text, providing strong support for the final named entity recognition result and improving the accuracy of named entity recognition.

[0036] Step S140: Determine the named entity recognition result of the target text based on the first decoding information and the second decoding information.

[0037] By combining the decoding information output from the two decoders, the named entity recognition result is obtained. The named entity recognition result can include: which entity label category each word in the target text corresponds to, or the probability value corresponding to different entity label categories; the named entities in the target text and which entity category each named entity belongs to.

[0038] based on Figure 1 The proposed method employs two approaches. Firstly, it utilizes a multi-head dual-affine decoder. Different heads learn different feature relationships within the text, enabling a comprehensive and thorough learning and extraction of textual information. This is particularly effective in capturing complex relationships between tags when entity boundaries are indistinct, facilitating accurate entity boundary identification and improving the final accuracy of named entity recognition. Secondly, by combining the two decoders, it captures both local and global features in the text, achieving a more comprehensive understanding of text structure and entity boundaries. This enhances the generalization ability of the entire model, enabling it to handle different types of named entity recognition tasks and facilitating its application in a wider range of language scenarios.

[0039] Based on one embodiment of this disclosure, reference is made to... Figure 2 As shown, the named entity recognition method may include the following steps S210 to S230:

[0040] Step S210: Extract start boundary information and end boundary information based on text encoding information.

[0041] Among them, start boundary information and end boundary information are two types of feature information extracted from text encoding information. They can express the feature information of each word in the target text as the start of an entity and the feature information of each word as the end of an entity.

[0042] In one implementation, the text encoding information can be processed through two hidden layers (such as two linear layers) to obtain start boundary information and end boundary information.

[0043] For example, start boundary information H s End boundary information H e As shown below:

[0044] H s =LeakyReLU(HW s (1)

[0045] H e =LeakyReLU(HW e (2)

[0046] Among them, W s W e These represent two linear layers, which can be learnable weight matrices. LeakyReLU indicates the use of the Leaky ReLU (Leaky Rectified Linear Unit) activation function.

[0047] Of course, this disclosure does not limit the specific calculation method of the hidden layer or the specific form of the activation function.

[0048] Step S220: Input the start boundary information and end boundary information into each head of the multi-head dual affine decoder, and output the dual affine feature corresponding to each head.

[0049] In one embodiment of this disclosure, different heads can have the same structure but different hidden layer parameters. Start and end boundary information are input into each head, and after calculation, each head outputs its corresponding biaffine feature. The biaffine features corresponding to different heads are generally also different.

[0050] In one embodiment of this disclosure, each head is provided with a biaffine matrix, biaffine weights, and biaffine biases, all of which are learnable parameters. During training, the parameters of different heads can be trained to different values.

[0051] Based on one embodiment of this disclosure, the named entity recognition method may include: in each head, multiplying the transpose matrix of the start boundary information with the biaffine matrix and the end boundary information, multiplying the biaffine weights with the concatenated matrix of the start and end boundary information, and adding the biaffine biases, and obtaining the biaffine feature based on the sum. For example, the sum can be used as the biaffine feature, or the sum can be further processed, such as by activating it with an activation function, to obtain the biaffine feature.

[0052] For example, the i-th head can output the following biaffine feature R. i :

[0053]

[0054] Among them, U i Let Wf represent the biaffine matrix in the i-th head. i The biaffine weights in the i-th head can be represented by a weight matrix, bf. i This represents the double affine bias in the i-th head; these are all learnable parameters. d represents the concatenation operation; d represents the hidden layer dimension in the i-th head (the hidden layer dimension of different heads can be the same), which can be equal to the hidden layer dimension of the encoder mentioned above.

[0055] Step S230: Fuse the dual affine features corresponding to each head to obtain the second decoding information.

[0056] For example, the dual affine features corresponding to each head can be spliced, added, or fused in any way to obtain the second decoded information.

[0057] Based on one embodiment of this disclosure, reference is made to... Figure 3 As shown, the named entity recognition method may include the following steps S310 to S320:

[0058] Step S310: Input the start boundary information and end boundary information into the gating unit of the multi-head dual affine decoder, and output the contribution value of each head.

[0059] Step S320: Based on the contribution value of each head, fuse the dual affine features corresponding to each head to obtain the second decoding information.

[0060] In a multi-head dual-affine decoder, each head learns different feature relationships, and the importance of each head generally varies. To address this, a gating mechanism is introduced to dynamically adjust the importance of each head, determining its contribution value, and then fusing the dual-affine features corresponding to each head according to their contribution values. The head with the larger contribution value has a larger proportion of its corresponding dual-affine features during fusion. The fused features yield the second decoded information. This approach allows for more flexible handling of different contextual information, enhancing the model's performance in processing information of varying granularities.

[0061] According to one embodiment of this disclosure, a multi-head dual-affine decoder includes multiple gating units, each corresponding to one head. Each gating unit includes a first gating weight, a second gating weight, and a gating bias, all of which are learnable parameters. During training, the parameters of different gating units can be trained to different values.

[0062] According to one embodiment of this disclosure, the named entity recognition method may include: in each gating unit, adding the result of multiplying the start boundary information and the first gating weight, the result of multiplying the end boundary information and the second gating weight, and the gating bias, and obtaining a contribution value based on the sum. For example, the sum can be used as the contribution value, or the sum can be further processed, such as by activating it with an activation function, to obtain the contribution value.

[0063] Based on one embodiment of this disclosure Figure 4 A schematic architecture of a multi-head dual-affine decoder is shown, comprising three heads, denoted as dual-affine Head1, dual-affine Head2, and dual-affine Head3, each head corresponding to a gating unit, namely gating unit 1, gating unit 2, and gating unit 3. After obtaining start and end boundary information using linear layer 1 and linear layer 2 respectively, the start and end boundary information is input into each head and each gating unit.

[0064] Taking any gated unit j as an example, its contribution value g j You can refer to the following formula for calculation:

[0065] g " =sigmoid(H s Mg " +H e Ng " +bg " (4)

[0066] Where sigmoid represents the sigmoid activation function, Mg j Ng j bg j represents the first gating weight, the second gating weight, and the gating bias of gating unit j, respectively, all of which are learnable parameters.

[0067] Based on one embodiment of this disclosure, the contribution value can be normalized, and the dual affine features corresponding to each head can be weighted and fused according to the normalized contribution value to obtain the second decoding information.

[0068] For example, refer to the following formula:

[0069]

[0070] Where B represents the second decoded information, softmax represents the softmax activation function used for normalization, and K represents the number of headers.

[0071] According to one embodiment of this disclosure, the multi-head dual-affine decoder includes multiple gating units, each corresponding to one head. Each gating unit includes gating weights (compared to the first and second gating weights mentioned above, only one gating weight is set for each gating unit here) and gating biases, both of which are learnable parameters. During training, the parameters of different gating units can be trained to different values.

[0072] According to one embodiment of this disclosure, the named entity recognition method may include: in each gating unit, adding the result of multiplying the text encoding information and the gating weight to the gating bias, and obtaining a contribution value based on the sum. For example, the sum can be used as the contribution value, or the sum can be further processed, such as by activating it with an activation function, to obtain the contribution value.

[0073] According to one embodiment of this disclosure, a multi-head dual affine decoder includes a gating unit that can input text encoding information and / or start boundary information and end boundary information into the gating unit, calculate and output the contribution value of each head.

[0074] Based on one embodiment of this disclosure, since the contribution values ​​of each head can be dynamically adjusted, the multi-head dual affine decoder can be referred to as a dynamic multi-head dual affine decoder.

[0075] According to one embodiment of this disclosure, the named entity recognition method may include: determining first category prediction information based on first decoding information; determining second category prediction information based on second decoding information; and fusing the first category prediction information and the second category prediction information to obtain the named entity recognition result of the target text.

[0076] The first and second category prediction information are both category predictions for named entities in the target text, and each can include the probability value of each word in the target text corresponding to different categories. The first and second category prediction information are prediction results based on different decoding information. Fusing the first and second category prediction information is equivalent to complementing the information of the first decoder and the multi-head dual-affine decoder, thereby obtaining a more comprehensive and accurate named entity recognition result.

[0077] Based on one embodiment of this disclosure, the first decoded information can be used as the first category prediction information.

[0078] Based on one embodiment of this disclosure, the first decoded information can be further processed, such as by a first prediction unit, to obtain first category prediction information. The first prediction unit may include one or more hidden layers.

[0079] For example, text-encoded information is input into a conditional random field decoder, processed, and the first decoded information, i.e., the first category prediction information, is output. M represents the number of categories or the number of categories plus one.

[0080] Based on one embodiment of this disclosure, the category can be an entity label category, such as B (Begin, representing the first character of the entity name), I (Inside, representing the middle or last character of the entity name), O (Outside, representing characters that are not entity names), etc., or it can be an entity category, such as a person's name, place name, organization name, time, etc.

[0081] Based on one embodiment of this disclosure, reference is made to... Figure 5 As shown, the named entity recognition method may include the following steps S510 to S530:

[0082] Step S510: Perform average pooling on the second decoded information to obtain the first pooled feature;

[0083] Step S520: Perform max pooling on the second decoded information to obtain the second pooled feature;

[0084] Step S530: Obtain the second category prediction information based on the first pooling feature and the second pooling feature.

[0085] Average pooling calculates the average value within the pooling window, using it as the pooled feature value. Max pooling, on the other hand, takes the maximum value within the pooling window, using it as the pooled feature value. These two pooling methods fuse feature information at different levels, retaining important features and highlighting the specific characteristics of each word. The second category prediction information is obtained by combining the first and second pooling features.

[0086] According to one embodiment of this disclosure, the named entity recognition method may include: fusing a first pooling feature and a second pooling feature and then inputting the fused features into a multilayer perceptron for processing to obtain second category prediction information.

[0087] The first and second pooling features can be fused by methods such as concatenation, addition, and weighting. The fused result is then input into a multilayer perceptron. The multilayer perceptron can integrate information from different dimensions, predict the relationship between different words, and map the fused pooling features to the category space, thereby achieving category prediction for each word and obtaining the second category prediction information.

[0088] According to one embodiment of this disclosure, a named entity recognition method may include: processing text-encoded information using a dynamic fusion unit to obtain fusion weights; and fusing first-category prediction information and second-category prediction information according to the fusion weights to obtain a named entity recognition result for the target text. The dynamic fusion unit may include one or more hidden layers, such as a linear layer. The fusion weights may include the weights corresponding to one or more of the first-category and second-category prediction information, representing the proportion of the first-category or second-category prediction information in the final named entity recognition result. For example, if the dynamic fusion unit processes the text-encoded information to obtain a fusion weight α corresponding to the first-category prediction information, then the fusion weight corresponding to the first-category prediction information is 1-α.

[0089] refer to Figure 6 As shown, the text encoding information is input into a multi-head dual affine decoder, and after processing, the second decoded information B is output. To obtain the global features, average pooling and max pooling layers are used to pool the second decoded information. The window size for both pooling methods can be set to N×1×X, where X is a hyperparameter that can be set or adjusted according to specific needs. After pooling, all dimensions with a vector size of 1 are removed, retaining important features, resulting in the fused pooled feature B1, as shown below:

[0090]

[0091] Here, AvgPool(·) and MaxPool(·) represent average pooling and max pooling, respectively, and W1 and W2 represent the fusion ratios of the first and second pooling features, respectively, which are learnable parameters. Average pooling and max pooling may capture different information; for example, average pooling may capture global, average statistical information, while max pooling may capture local, most prominent features. Fusing the first and second pooling features can increase the feature diversity learned by the model, helping the model to generalize better and providing a more comprehensive perspective.

[0092] Then, a multi-layer perceptron (MLP) is used to learn the relationships between different words and predict the category of each word, thereby obtaining the second category prediction information R. B As shown below.

[0093]

[0094] Depending on the specific named entity recognition scenario, the first decoder and the multi-head dual-affine decoder offer different levels of assistance. The fusion weights of each decoder can be dynamically adjusted based on their information content and task requirements. For example, let α represent the fusion weight corresponding to the first decoder (i.e., the fusion weight corresponding to the first category of predicted information), and let 1-α represent the fusion weight corresponding to the multi-head dual-affine decoder (i.e., the fusion weight corresponding to the second category of predicted information). Softmax is used to ensure that the sum of the weights is 1, and the following relationship holds:

[0095]

[0096] Where softmax represents the Softmax activation function, W α The weight matrix in the dynamic fusion unit represents learnable parameters, and H represents the text encoding information. By fusing the first category prediction information and the second category prediction information, the named entity recognition result of the target text is obtained.

[0097] Based on one embodiment of this disclosure, first category prediction information and second category prediction information are fused to obtain fused prediction information; the fused prediction information is processed by an entity category fusion unit to obtain the final named entity recognition result.

[0098] For example, the fused prediction information Q can satisfy the following relationship:

[0099] Q = αR² + (1 - α)R; (9)

[0100] The Q input is the entity category fusion unit, which may include one or more hidden layers. After further processing, the output is the named entity recognition result, which may include the entity category corresponding to each named entity in the target text.

[0101] Based on one embodiment of this disclosure, any one or more of the above machine learning models, or units in the machine learning models, can be pre-trained models, or the initial model can be trained with certain training data and then put into practical application.

[0102] This exemplary implementation achieves information complementarity between two decoders by fusing two different decoders. For example, the Conditional Random Field (CRF) decoder can capture dependencies between tags, while the Multi-Headed Affine Decoder (MAD) can capture dependencies between words. Combining the two can more comprehensively model the complex relationships in the named entity recognition task. Furthermore, by fusing these two modules, the fine-grained feature modeling capability of the CRF and the global optimization capability of the MAD can be utilized, thereby improving the accuracy of the named entity recognition task.

[0103] Based on one embodiment of this disclosure, named entity recognition technology can be integrated into facilities such as intelligent systems, network management systems, and maintenance systems. For example, an intelligent system equipped with named entity recognition technology can understand users' natural language commands, identify key entities within the commands, and then intelligently execute corresponding control operations. As another example, a network management and maintenance system employing named entity recognition technology can automatically identify key technical entities and problem descriptions through analysis of technical documents and user feedback, helping technicians quickly locate problems and develop solutions. Based on named entity recognition technology, complex information in a scenario can be deeply mined, thereby improving the intelligence level of related products and services, better meeting users' personalized needs, enhancing user experience, and improving service efficiency and quality.

[0104] Based on one embodiment of this disclosure, reference is made to... Figure 7 As shown, the named entity recognition device 700 may include:

[0105] The encoding module 710 is configured to encode the target text to obtain text encoding information;

[0106] The first decoding module 720 is configured to decode the text encoding information using a first decoder to obtain first decoded information;

[0107] The second decoding module 730 is configured to decode the text encoding information using a multi-head dual affine decoder to obtain second decoded information;

[0108] The result output module 740 is configured to determine the named entity recognition result of the target text based on the first decoding information and the second decoding information.

[0109] According to one embodiment of this disclosure, the step of decoding the text encoded information using a multi-head dual affine decoder to obtain second decoded information includes:

[0110] The start boundary information and end boundary information are extracted based on the text encoding information;

[0111] The start boundary information and the end boundary information are respectively input into each head of the multi-head dual affine decoder, and the dual affine feature corresponding to each head is output.

[0112] The second decoding information is obtained by fusing the dual affine features corresponding to each head.

[0113] According to one embodiment of this disclosure, fusing the dual affine features corresponding to each head to obtain the second decoding information includes:

[0114] The start boundary information and the end boundary information are input into the gating unit of the multi-head dual affine decoder, and the contribution value of each head is output.

[0115] The second decoding information is obtained by fusing the dual affine features corresponding to each head based on the contribution value of each head.

[0116] According to one embodiment of the present disclosure, the multi-head dual affine decoder includes multiple gating units, each gating unit corresponding to one head; each gating unit includes a first gating weight, a second gating weight, and a gating bias.

[0117] The process of inputting the start boundary information and the end boundary information into the gating unit of the multi-head dual affine decoder and outputting the contribution value of each head includes:

[0118] In each gating unit, the result of multiplying the start boundary information and the first gating weight, the result of multiplying the end boundary information and the second gating weight, and the gating bias are added together, and the contribution value is obtained based on the sum.

[0119] According to one embodiment of the present disclosure, each head is provided with a dual affine matrix, dual affine weights, and dual affine biases;

[0120] The step of inputting the start boundary information and the end boundary information into each head of the multi-head dual-affine decoder and outputting the dual-affine feature corresponding to each head includes:

[0121] In each head, the result of multiplying the transpose matrix of the start boundary information with the double affine matrix and the end boundary information, the result of multiplying the double affine weight with the concatenated matrix of the start boundary information and the end boundary information, and the double affine bias are added together to obtain the double affine feature.

[0122] According to one embodiment of this disclosure, determining the named entity recognition result of the target text based on the first decoding information and the second decoding information includes:

[0123] Determine the first category prediction information based on the first decoding information;

[0124] The second category of prediction information is determined based on the second decoding information;

[0125] By fusing the prediction information of the first category and the prediction information of the second category, the named entity recognition result of the target text is obtained.

[0126] According to one embodiment of this disclosure, determining the second category prediction information based on the second decoding information includes:

[0127] The second decoded information is subjected to average pooling to obtain the first pooling feature;

[0128] The second decoded information is subjected to max pooling to obtain the second pooling feature;

[0129] The second category prediction information is obtained based on the first pooling feature and the second pooling feature.

[0130] According to one embodiment of this disclosure, obtaining the second category prediction information based on the first pooling feature and the second pooling feature includes:

[0131] The first pooling feature and the second pooling feature are fused and then input into a multilayer perceptron for processing to obtain the second category prediction information.

[0132] According to one embodiment of this disclosure, fusing the first category prediction information and the second category prediction information to obtain the named entity recognition result of the target text includes:

[0133] The text encoding information is processed using a dynamic fusion unit to obtain fusion weights;

[0134] The first category prediction information and the second category prediction information are fused according to the fusion weight to obtain the named entity recognition result of the target text.

[0135] According to one embodiment of this disclosure, the first decoder includes a conditional random field decoder.

[0136] The specific details of each part of the above-mentioned device have been described in detail in the method section of the implementation plan. For any undisclosed details, please refer to the implementation plan of the method section, and therefore will not be repeated here.

[0137] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to exemplary embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0138] Exemplary embodiments of this disclosure also provide a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the named entity recognition method described above.

[0139] In one embodiment, the computer program product can be a tangible product containing a computer program, such as a computer-readable storage medium storing the computer program. The readable storage medium can be a storage medium based on electrical, magnetic, optical, electromagnetic, infrared, or other signals, including but not limited to: random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory, hard disk drive (HDD), solid-state drive (SSD), etc. For example, the computer program product can be implemented as a non-volatile storage medium storing the computer program, such as read-only memory, NAND flash memory, etc.

[0140] In one implementation, the computer program product can be an intangible product containing a computer program. For example, the computer program product can be implemented as a virtual digital product, such as an executable file, installation package, or other digital file storing the computer program.

[0141] Computer program code can be written in one or more programming languages. Examples of programming languages ​​include C, Java, and C++. Program code can execute entirely on the user's computing device, partially on the user's computing device, or as a standalone software package. It can also execute partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, such as a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via an internet connection provided by a mobile network operator).

[0142] Computer programs can be carried or transmitted via signals such as electricity, magnetism, light, electromagnetic radiation, and infrared rays. Electronic devices can convert signals carrying computer programs into digital signals, thereby executing the computer programs. When a computer program runs on an electronic device, its code causes the electronic device to execute (more specifically, its processor) the method steps of various exemplary embodiments of this disclosure, such as those described above. Figure 1 The method and steps.

[0143] Exemplary embodiments of this disclosure also provide an electronic device. The electronic device may include a processor and a memory. The memory stores executable instructions for the processor, such as computer programs. The processor executes these executable instructions to perform the method steps of various exemplary embodiments of this disclosure.

[0144] The following is for reference. Figure 8 The electronic device is illustrated by way of a general-purpose computing device. It should be understood that... Figure 8The electronic device 800 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0145] like Figure 8 As shown, the electronic device 800 may include: a processor 810, a memory 820, a bus 830, an I / O (input / output) interface 840, and a network adapter 850.

[0146] The memory 820 may include volatile memory, such as RAM 821 and cache unit 822, and may also include non-volatile memory, such as ROM 823. The memory 820 may also include one or more program modules 824, including but not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. For example, program module 824 may include the modules described above.

[0147] The processor 810 may include one or more processing units, such as an AP (Application Processor), a modem processor, a GPU (Graphics Processing Unit), an ISP (Image Signal Processor), a controller, an encoder, a decoder, a DSP (Digital Signal Processor), a baseband processor, and / or an NPU (Neural-Network Processing Unit).

[0148] Processor 810 can be used to execute executable instructions stored in memory 820, which may include method steps of various exemplary embodiments of this disclosure, such as... Figure 1 The method and steps.

[0149] Bus 830 is used to connect different components of electronic device 800 and may include data bus, address bus and control bus.

[0150] Electronic device 800 can communicate with one or more external devices 900 (such as keyboard, mouse, external controller, etc.) through I / O interface 840.

[0151] Electronic device 800 can communicate with one or more networks via network adapter 850. For example, network adapter 850 can provide mobile communication solutions such as 3G / 4G / 5G, or wireless communication solutions such as wireless LAN, Bluetooth, and near-field communication. Network adapter 850 can communicate with other modules of electronic device 800 via bus 830.

[0152] although Figure 8 As not shown in the diagram, other hardware and / or software modules may also be configured in the electronic device 800, including but not limited to: a display, microcode, device driver, redundant processor, external disk drive array, RAID system, tape drive, and data backup storage system.

[0153] As can be seen from the above, the technical solutions disclosed herein can be implemented as methods, apparatus, systems, computer program products, storage media, electronic devices, etc. Those skilled in the art will understand that various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be referred to as "circuit," "module," or "system," respectively.

[0154] It should be understood that this disclosure is not limited to the specific methods, steps, or structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. Those skilled in the art will readily conceive of other embodiments based on the specific implementations provided in this disclosure. Therefore, the specific implementations provided in this disclosure are merely exemplary, and the scope and spirit of this disclosure are indicated by the claims, and should cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary technical means in the art not disclosed in this disclosure.

Claims

1. A named entity recognition method, characterized in that, The method includes: The target text is encoded to obtain text encoding information; The text encoding information is decoded using a first decoder to obtain first decoded information; The text encoding information is decoded using a multi-head dual affine decoder to obtain the second decoded information; Based on the first decoding information and the second decoding information, the named entity recognition result of the target text is determined.

2. The method according to claim 1, characterized in that, The process of decoding the text encoding information using a multi-head dual affine decoder to obtain second decoded information includes: The start boundary information and end boundary information are extracted based on the text encoding information; The start boundary information and the end boundary information are respectively input into each head of the multi-head dual affine decoder, and the dual affine feature corresponding to each head is output. The second decoding information is obtained by fusing the dual affine features corresponding to each head.

3. The method according to claim 2, characterized in that, The second decoding information is obtained by fusing the dual affine features corresponding to each head, including: The start boundary information and the end boundary information are input into the gating unit of the multi-head dual affine decoder, and the contribution value of each head is output. The second decoding information is obtained by fusing the dual affine features corresponding to each head based on the contribution value of each head.

4. The method according to claim 2, characterized in that, The multi-head dual affine decoder includes multiple gating units, each corresponding to one head; each gating unit includes a first gating weight, a second gating weight, and a gating bias. The process of inputting the start boundary information and the end boundary information into the gating unit of the multi-head dual affine decoder and outputting the contribution value of each head includes: In each gating unit, the result of multiplying the start boundary information and the first gating weight, the result of multiplying the end boundary information and the second gating weight, and the gating bias are added together, and the contribution value is obtained based on the sum.

5. The method according to claim 2, characterized in that, Each head is configured with a dual affine matrix, dual affine weights, and dual affine biases; The step of inputting the start boundary information and the end boundary information into each head of the multi-head dual-affine decoder and outputting the dual-affine feature corresponding to each head includes: In each head, the result of multiplying the transpose matrix of the start boundary information with the double affine matrix and the end boundary information, the result of multiplying the double affine weight with the concatenated matrix of the start boundary information and the end boundary information, and the double affine bias are added together to obtain the double affine feature.

6. The method according to claim 1, characterized in that, Determining the named entity recognition result of the target text based on the first decoding information and the second decoding information includes: Determine the first category prediction information based on the first decoding information; The second category of prediction information is determined based on the second decoding information; By fusing the prediction information of the first category and the prediction information of the second category, the named entity recognition result of the target text is obtained.

7. The method according to claim 6, characterized in that, The step of determining the second category prediction information based on the second decoding information includes: The second decoded information is subjected to average pooling to obtain the first pooling feature; The second decoded information is subjected to max pooling to obtain the second pooling feature; The second category prediction information is obtained based on the first pooling feature and the second pooling feature.

8. The method according to claim 7, characterized in that, The step of obtaining the second category prediction information based on the first pooling feature and the second pooling feature includes: The first pooling feature and the second pooling feature are fused and then input into a multilayer perceptron for processing to obtain the second category prediction information.

9. The method according to claim 6, characterized in that, The process of fusing the first category prediction information and the second category prediction information to obtain the named entity recognition result of the target text includes: The text encoding information is processed using a dynamic fusion unit to obtain fusion weights; The first category prediction information and the second category prediction information are fused according to the fusion weight to obtain the named entity recognition result of the target text.

10. The method according to any one of claims 1 to 9, characterized in that, The first decoder includes a conditional random field decoder.

11. A named entity recognition device, characterized in that, The device includes: The encoding module is configured to encode the target text to obtain text encoding information; The first decoding module is configured to decode the text encoding information using a first decoder to obtain first decoded information; The second decoding module is configured to decode the text encoding information using a multi-head dual affine decoder to obtain second decoded information; The result output module is configured to determine the named entity recognition result of the target text based on the first decoding information and the second decoding information.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method according to any one of claims 1 to 10.

13. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the method of any one of claims 1 to 10 by executing the executable instructions.

Images