Named entity recognition method, device, computer readable medium and electronic device

Abstract

The present disclosure provides a method, device, computer-readable medium and electronic device for named entity recognition, and relates to the field of artificial intelligence technology. The method includes: performing word-granularity serialization processing on a text to be processed to obtain a word sequence corresponding to the text to be processed; taking any character in the word sequence as a target character, and determining a unigram word vector and a multi-element word vector corresponding to the target character based on the target character and the context corresponding to the target character; fusing the unigram word vector and the multi-element word vector corresponding to the target character to obtain a fusion vector corresponding to the target character; after obtaining the fusion vectors corresponding to all characters in the word sequence, performing named entity recognition according to the fusion vectors corresponding to all characters to obtain named entity annotation information corresponding to the text to be processed. The present disclosure can improve the accuracy and efficiency of named entity recognition, and reduce the complexity of prediction.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and more specifically, to a named entity recognition method, a named entity recognition device, a computer-readable medium, and an electronic device. Background Technology

[0002] Named Entity Recognition (NER) is a fundamental underlying technology in Natural Language Processing (NLP) that aims to identify named entities in text.

[0003] Named entity recognition (NER) is an indispensable technology in many products and applications, such as information extraction, question answering systems, syntactic analysis, and machine translation. Currently, sequence labeling models and span prediction models are commonly used for NER tasks. However, sequence labeling models are based on character or word serialization. Word-based serialization requires a tokenizer, which can affect the recognition of entity boundaries and lead to OOV (out-of-vocabulary) errors. Character-based serialization avoids the need for a tokenizer, but it lacks the basic semantics of words, significantly impacting recognition performance. Meanwhile, span prediction models suffer from high prediction complexity, greatly affecting prediction speed.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The embodiments of this disclosure provide a named entity recognition method, a named entity recognition device, a computer-readable medium, and an electronic device, which can at least to some extent improve the accuracy and efficiency of named entity recognition and reduce prediction complexity.

[0006] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0007] According to one aspect of the present disclosure, a named entity recognition method is provided, comprising: performing character-level serialization processing on a text to be processed to obtain a character sequence corresponding to the text to be processed; taking any character in the character sequence as a target character, determining a unigram character vector and a multi-dimensional character vector corresponding to the target character based on the target character and the context corresponding to the target character; fusing the unigram character vector and the multi-dimensional character vector corresponding to the target character to obtain a fused vector corresponding to the target character; and after obtaining the fused vectors corresponding to all characters in the character sequence, performing named entity recognition based on the fused vectors corresponding to all characters to obtain named entity annotation information corresponding to the text to be processed.

[0008] According to one aspect of the present disclosure, a named entity recognition device is provided, comprising: a serialization processing module, configured to perform character-level serialization processing on a text to be processed to obtain a character sequence corresponding to the text to be processed; a character vector determination module, configured to take any character in the character sequence as a target character, and determine a unigram character vector and a multi-dimensional character vector corresponding to the target character based on the target character and the context corresponding to the target character; a character vector fusion module, configured to fuse the unigram character vector and the multi-dimensional character vector corresponding to the target character to obtain a fused vector corresponding to the target character; and a named entity recognition module, configured to, after obtaining the fused vectors corresponding to all characters in the character sequence, perform named entity recognition based on the fused vectors corresponding to all characters to obtain named entity annotation information corresponding to the text to be processed.

[0009] In some embodiments of this disclosure, based on the above scheme, the character vector determination module includes: a character information determination unit, configured to determine unary character information and multi-element character information based on the target character and the context corresponding to the target character; a unary character vector determination unit, configured to determine the unary character vector according to the unary character information and a first vector lookup table corresponding to the unary character information; and a multi-element character vector determination unit, configured to determine the multi-element character vector according to the multi-element character information and a second vector lookup table corresponding to the multi-element character information.

[0010] In some embodiments of this disclosure, the first vector lookup table includes multiple unary characters and vectors corresponding to each unary character; based on the above scheme, the unary character vector determination unit is configured to: match the unary character information with the unary characters in the first vector lookup table; when there is a target unary character that matches the unary character information, obtain the vector corresponding to the target unary character as the unary character vector.

[0011] In some embodiments of this disclosure, based on the above scheme, the multi-character vector determination unit includes: a hash processing unit, used to perform hash processing on the multi-character information to obtain an integer hash value corresponding to the multi-character information; and a vector query unit, used to determine the multi-character vector according to the integer hash value and the second vector query table.

[0012] In some embodiments of this disclosure, the second vector lookup table includes multiple hash values ​​in a preset hash value range and vectors corresponding to each hash value. Based on the above scheme, the vector lookup unit is configured to: determine whether the integer hash value is greater than the maximum value of the preset hash value range; when the integer hash value is less than or equal to the maximum value, determine the target hash value corresponding to the integer hash value in the second vector lookup table according to the integer hash value, and obtain the vector corresponding to the target hash value as the multi-element word vector; when the integer hash value is greater than the maximum value, perform a modulo operation on the integer hash value to obtain an updated hash value corresponding to the integer hash value and within the preset hash value range, so as to determine the multi-element word vector in the second vector lookup table according to the updated hash value.

[0013] In some embodiments of this disclosure, based on the above scheme, the word vector fusion module is configured to calculate the fusion vector according to the following formulas (1)-(3):

[0014] s i =v T tanh(w i x i (1)

[0015] A i =softmax(s i (2)

[0016]

[0017] Where, x i Let i = 1, 2, ..., n be the univariate word vector and the multivariate word vector; w i Is with x i The corresponding weight coefficients; v is a trainable vector; s i Is with x i The corresponding calculated value; A i Is with x i The corresponding self-attention weights; Y is the fusion vector; n is the maximum number of characters corresponding to the multi-dimensional word vector.

[0018] In some embodiments of this disclosure, based on the above scheme, the named entity recognition method is executed by a named entity recognition model, which includes an input layer, a serialization layer, a vectorization layer, a fusion layer, a named entity recognition layer, and an output layer.

[0019] In some embodiments of this disclosure, the named entity recognition layer includes multiple sequentially connected convolutional layers and normalization layers; based on the above scheme, the named entity recognition module is configured to: perform convolution and activation operations on the fusion vectors corresponding to all characters through the convolutional layers to obtain the representation vectors corresponding to each character; and perform normalization processing on the representation vectors through the normalization layers to obtain named entity information corresponding to all characters.

[0020] In some embodiments of this disclosure, based on the above scheme, the named entity recognition device further includes: a pre-training module, used to pre-train N-gram vectors using unlabeled text, so as to determine the unary character vector and the multi-element character vector based on the target character, the context corresponding to the target character, and the pre-trained N-gram vector.

[0021] In some embodiments of this disclosure, based on the above scheme, the pre-training module includes: a serialization unit, used to perform character-level serialization processing on each text sentence in the unannotated text to obtain character sequence samples corresponding to each text sentence; a character information acquisition unit, used to take any character in the character sequence samples as a target sample character, and traverse the text sentence according to the target sample character and a preset window to determine unary character information or multi-dimensional character information to be predicted; and a pre-training unit, used to pre-train N-gram vectors based on the target sample character, the unary character information to be predicted and the first vector lookup table, or based on the target sample character, the multi-dimensional character information to be predicted, the first vector lookup table and the second vector lookup table.

[0022] In some embodiments of this disclosure, based on the above scheme, the pre-training unit is configured to: determine a first input vector corresponding to the target sample character based on the target sample character and the first vector lookup table, and simultaneously determine a first output vector corresponding to the unary character information to be predicted based on the unary character information to be predicted based on the first vector lookup table; determine a first loss function based on the first input vector and the first output vector, and perform parameter optimization based on the first loss function; or, determine a second input vector corresponding to the target sample character based on the target sample character and the first vector lookup table, and simultaneously determine a second output vector corresponding to the unary character information to be predicted based on the unary character information to be predicted based on the unary character information to be predicted based on the unary character information to be predicted based on the unary character lookup table; determine a second loss function based on the second input vector and the second output vector, and perform parameter optimization based on the second loss function.

[0023] In some embodiments of this disclosure, both the first loss function and the second loss function include a first loss part and a second loss part; based on the above scheme, the loss function is determined according to the input vector and the output vector, and the parameters are optimized and configured based on the loss function as follows: the loss function is determined according to the following formula (4):

[0024]

[0025] Among them, Loss + For the first loss part, Loss - For the second loss part, v wi v is the input vector corresponding to the target sample character. wo v' is the output vector corresponding to the unary character information or the multivariate character information to be predicted. wk The input vector is the randomly selected characters that do not belong to the text sentence, k is the number of randomly selected characters that do not belong to the text sentence, and N is the maximum number of randomly selected characters that do not belong to the text sentence.

[0026] In some embodiments of this disclosure, the multi-element character information includes binary character information and ternary character information. Based on the above scheme, the character information determining unit is configured to: use the target character as the unary character information; form the binary character information based on the target character and the next character or identifier adjacent to the target character; form the ternary character information based on the target character, the previous character or identifier adjacent to the target character, and the next character adjacent to the target character, or form the ternary character information based on the target character, the previous character adjacent to the target character, and the next character or identifier adjacent to the target character.

[0027] According to one aspect of the present disclosure, a computer storage medium is provided that stores a computer program thereon, characterized in that the program, when executed by a processor, implements the named entity recognition method provided in the optional implementation described above.

[0028] According to one aspect of the embodiments of this disclosure, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform the named entity recognition method provided in the optional implementation described above.

[0029] According to one aspect of the present disclosure, an electronic device is provided, comprising: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the method provided in the optional implementation described above.

[0030] In some embodiments of this disclosure, the technical solutions first perform character-level serialization on the text to be processed. After obtaining the character sequence corresponding to the text to be processed, any character in the character sequence is taken as the target character. Based on the target character and the context corresponding to the target character, a unigram character vector and a multi-dimensional character vector corresponding to the target character are determined. Then, all character vectors corresponding to the target character are fused to obtain a fused vector corresponding to the target character. Finally, after obtaining the fused vectors corresponding to all characters in the character sequence, named entity recognition is performed on the fused vectors corresponding to all characters using a sequence labeling model to obtain named entity labeling information corresponding to the text to be processed. This technical solution avoids word segmentation errors caused by using a word segmenter. Furthermore, it incorporates character-based N-gram vector features, providing both character-level and word-level vector features during named entity recognition, thus improving the accuracy of named entity recognition. It also avoids predicting all text segments of the input sentence that are less than or equal to a preset length, reducing prediction complexity and improving the efficiency of named entity recognition.

[0031] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0032] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0033] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of this disclosure can be applied is shown;

[0034] Figure 2 The schematic diagram illustrates a flowchart of a named entity recognition method according to an embodiment of the present disclosure;

[0035] Figure 3 This illustration schematically shows a structural diagram of a named entity recognition model according to an embodiment of the present disclosure;

[0036] Figure 4 The illustration schematically shows a flowchart of determining a unary word vector according to an embodiment of the present disclosure;

[0037] Figure 5 The illustration schematically shows a flowchart of determining multi-word vectors according to an embodiment of the present disclosure;

[0038] Figure 6 The illustration schematically shows a flowchart of determining multi-word vectors according to an embodiment of the present disclosure;

[0039] Figure 7 The schematic diagram illustrates the structure of a named entity recognition layer 305 according to an embodiment of the present disclosure;

[0040] Figure 8 The illustration schematically shows a flowchart of a pre-trained N-gram vector according to an embodiment of the present disclosure;

[0041] Figure 9 A block diagram of a named entity recognition device according to an embodiment of the present disclosure is shown schematically;

[0042] Figure 10 A schematic diagram of the structure of a computer system suitable for implementing the embodiments of the present disclosure is shown. Detailed Implementation

[0043] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art.

[0044] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0045] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0046] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0047] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of this disclosure can be applied is shown.

[0048] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102, and a server 103. The terminal device 101 may be a mobile phone, portable computer, tablet computer, or other terminal device with a display screen. The network 102 serves as a medium for providing a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired communication links, wireless communication links, etc. In this embodiment, the network 102 between the terminal device 101 and the server 103 may be a wireless communication link, specifically a mobile network.

[0049] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included. It is worth noting that the servers in this disclosure can be independent servers or server clusters formed by multiple servers.

[0050] In one embodiment of this disclosure, a user inputs text to be processed through a terminal device 101, and the terminal device 101 sends the text to be processed to a server 103 via a network 102. The text to be processed can be a sentence, a paragraph, or a sentence from an article. After receiving the text to be processed, the server 103 can perform character-level serialization processing on each sentence in the text to obtain a character sequence corresponding to the text. Then, it can determine the unigram and multi-dimensional character vectors corresponding to each character in the character sequence and the context corresponding to that character in the text. Next, it can fuse all the character vectors corresponding to each character to obtain a fused vector. After obtaining the fused vectors corresponding to all characters, it can perform named entity recognition based on the fused vectors corresponding to all characters through the named entity recognition layer in the named entity recognition model to obtain the named entity annotation information corresponding to the text to be processed. To improve the accuracy of named entity recognition, unigram and multigram vectors can be determined based on the target character, the context corresponding to the target character, and pre-trained N-gram vectors. The accuracy of the unigram and multigram vectors corresponding to the target character can be improved by using pre-trained N-gram vectors, thereby improving the accuracy of named entity recognition.

[0051] It should be noted that the named entity recognition method provided in this disclosure is generally executed by a server, and correspondingly, the named entity recognition device is generally located in the server. However, in other embodiments of this disclosure, the named entity recognition method provided in this disclosure can also be executed by a terminal device.

[0052] In related technologies in this field, named entity recognition is mainly performed using two types of models: sequence labeling models and span prediction models. For sequence labeling models, the sentence is first serialized (tokenized), segmenting it into individual characters or words. Then, an encoder is used to obtain the vector of each token. After obtaining the vector of each character or word, it can be classified to determine whether the character or word belongs to an entity. For span prediction models, arbitrary segments of the input sentence are classified. Specifically, given a maximum length L, all segments of the sentence with a length less than or equal to L are traversed. For example, in "Zhang San attended the ceremony," L=3, so "Zhang San" and "Zhang San attend" are both segments. Then, each segment is classified to determine whether the entire segment is an entity.

[0053] While both of the aforementioned models can achieve named entity recognition, they also have corresponding limitations. For sequence labeling models, both character-based and word-based serialization methods have their advantages and disadvantages. The advantage of word-based serialization is that it preserves the complete basic semantics of words, but the disadvantage is that it relies on a word segmenter, and incorrect word segmentation can negatively affect the results, sometimes affecting the recognition of entity boundaries and easily leading to OOV (out-of-vocabulary) problems. The advantage of character-based serialization is that it does not rely on word segmentation and is not affected by segmentation errors, but using only characters lacks the basic semantics of words, which has a significant impact on recognition performance. For the span prediction model, since it needs to predict all text segments of length L or less in the input sentence, the complexity is O(L*N) (N is the sentence length), which is much higher than the O(N) of the sequence labeling model, greatly affecting the prediction speed.

[0054] In view of the problems existing in related technologies, this disclosure provides a named entity recognition method based on machine learning. Machine learning is a type of artificial intelligence (AI). AI is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain optimal results. In other words, AI is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling machines to have the functions of perception, reasoning, and decision-making.

[0055] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0056] Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in tasks such as target recognition, tracking, and measurement, and further performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0057] Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0058] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, and smart customer service. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0059] The solutions provided in this disclosure relate to natural language processing technology in artificial intelligence, and are specifically illustrated through the following embodiments:

[0060] This disclosure first provides a named entity recognition method, and the implementation details of the technical solution of this disclosure embodiment are described in detail below:

[0061] Figure 2 A flowchart illustrating a named entity recognition method according to an embodiment of the present disclosure is shown. This named entity recognition method can be executed by a server, which may be... Figure 1 Service 103 is shown in the image. (See reference...) Figure 2As shown, the named entity recognition method includes at least steps S210 to S240, which are described in detail below:

[0062] In step S210, the text to be processed is serialized at the character level to obtain the character sequence corresponding to the text to be processed.

[0063] In one embodiment of this disclosure, the text to be processed refers to text in units of sentences, such as the text corresponding to a single sentence, a paragraph, or any sentence from an article, etc. The text to be processed and the text or article related to it can be text manually entered by the user requiring named entity recognition, or text downloaded from the internet requiring named entity recognition; this embodiment of the disclosure does not specifically limit this. After obtaining the text to be processed, it can be serialized at the character level to obtain the character sequence corresponding to the text. That is, after character-level serialization, each character in the text becomes a token. For example, if the text to be processed is "I went to the zoo today," then the corresponding character sequence is "I went to the zoo today." It is worth noting that for text containing multiple sentences, the text can be divided into multiple text sentences, and then each text sentence can be serialized sequentially to obtain its corresponding named entity annotation information.

[0064] In one embodiment of this disclosure, a lightweight named entity recognition model can be used to process the text to be processed in order to obtain the named entity annotation information corresponding to the text to be processed. Figure 3 This diagram illustrates the structure of a named entity recognition model, such as... Figure 3 As shown, the named entity recognition model 300 includes an input layer 301, a serialization layer 302, a vectorization layer 303, a fusion layer 304, a named entity recognition layer 305, and an output layer 306. The input layer 301 receives the user-inputted text and inputs it into the serialization layer 302. The serialization layer 302 performs character-level serialization processing on the text to be processed to obtain the character sequence corresponding to the text. For example, if the input layer 301 inputs the text "Singer C said" into the serialization layer 302, the character sequence "Singer C said" can be obtained after processing by the serialization layer 302.

[0065] In step S220, any character in the character sequence is taken as the target character, and a univariate character vector and a multivariate character vector corresponding to the target character are determined based on the target character and the context corresponding to the target character.

[0066] In one embodiment of this disclosure, after obtaining the character sequence corresponding to the text to be processed, each character in the character sequence can be used as a target character and subjected to corresponding subsequent processing. Specifically, the vectorization layer 303 can determine the unary character vector and the multi-element character vector corresponding to the target character based on the target character and the context corresponding to the target character. The unary character vector is the vector corresponding to unary character information, and the multi-element character vector is the vector corresponding to multi-element character information. Both unary and multi-element refer to the number of characters, and unary is the token itself, while multi-element is the string consisting of the token itself and one or more characters adjacent to the token.

[0067] When determining unary and multi-ary character vectors based on the target character and its corresponding context, the unary character information and multi-ary character information can first be determined based on the target character and its corresponding context; then, the unary character vector can be determined based on the unary character information and the first vector lookup table corresponding to the unary character information; finally, the multi-ary character vector can be determined based on the multi-ary character information and the second vector lookup table corresponding to the multi-ary character information.

[0068] Considering that the purpose of introducing multi-element character vectors is to overcome the problem of poor recognition accuracy caused by the lack of complete basic semantics of words when using character sequences for named entity recognition, and considering that the number of characters constituting a word is usually two or three, and of course, four, but more than four is relatively rare, in order to ensure both the introduction of complete basic semantics of words and the density of the word space, in the embodiments of this disclosure, the multi-element vector corresponding to the target character can specifically be a binary character vector and a ternary character vector. Accordingly, when obtaining multi-element character information, only the binary character information and the ternary character information corresponding to the target character need to be obtained. When determining the binary character information, the combination of the target character and the next character or the combination of the target character and the termination identifier can be used as the binary character information; when determining the ternary character information, the combination of the target character, the preceding character adjacent to the target character, and the following character, or the combination of the start identifier, the target character, and the following character adjacent to the target character, or the combination of the target character, the preceding character adjacent to the target character, and the termination identifier can be used as the ternary character information, wherein the start identifier is marked as... <beg>The termination identifier is marked as <end>Words can be directly modeled using binary and ternary character information. Continuing with the word sequence "Singer C said," any single character can be used as the target character. When "song" is the target character, the unary character information is determined to be "song" itself, the binary character information is "singer," and the ternary character information is... <beg>"singer"; when "C" is the target character, the unary character information can be determined as "C", the binary character information as "C said", and the ternary character information as "hand C said"; when "said" is the target character, the unary character information can be determined as "said", and the binary character information as "said <end>”, the ternary character information is "Bing said <end>".

[0069] In one embodiment of this disclosure, after determining unary character information and multi-element character information, the corresponding vector can be determined based on the unary character information, multi-element character information, and the corresponding vector lookup table. Since the space of unary character information is much smaller than that of multi-element character information, a first vector lookup table can be directly constructed based on the unary character information and its corresponding vector to determine the unary character vector corresponding to the unary character information in the text to be processed. However, for the second vector lookup table corresponding to the multi-element character information, it is necessary to consider how to reduce the space occupied while ensuring the acquisition of the multi-element character vector corresponding to the multi-element character information.

[0070] The first vector lookup table includes multiple unary characters and vectors corresponding to each unary character. Figure 4 shows a flowchart illustrating the process of determining unary character vectors, such as... Figure 4 As shown, in step S401, the unary character information is matched with the unary characters in the first vector lookup table; in step S402, when there is a target unary character that matches the unary character information, the vector corresponding to the target unary character is obtained as the unary word vector.

[0071] Because the space for binary and ternary character vectors, and even other multi-element character vectors, is particularly large, storing them directly based on multi-element character information and their corresponding vectors would require a significant amount of space. Therefore, in the embodiments of this disclosure, the second vector lookup table corresponding to the multi-element character information is not constructed based on the multi-element character information and its corresponding vector, but rather based on the hash value corresponding to the multi-element character information and its corresponding vector. In the embodiments of this disclosure, taking binary and ternary character vectors as examples, the second vector lookup table can specifically be constructed based on the hash value corresponding to the binary and ternary character information and its corresponding vector. That is, when determining the vector corresponding to the binary or ternary character information, the binary or ternary character information first needs to be hashed to a preset hash value range of [0, N]. This preset hash value range is a fixed integer range, where N is a positive integer. Then, based on the hash value corresponding to the binary or ternary character information, the corresponding vector is determined in the second vector lookup table, thus obtaining the binary or ternary character vector. The second vector lookup table contains multiple hash values ​​within a preset hash value range and a vector corresponding to each hash value.

[0072] Figure 5 A flowchart illustrating the process of determining multi-element character vectors is shown, such as... Figure 5 As shown, in step S501, the multi-character information is hashed to obtain the integer hash value corresponding to the multi-character information; in step S502, the multi-character vector is determined according to the integer hash value and the second vector lookup table.

[0073] Furthermore, during the hashing process, the integer hash value obtained after hashing the multi-character information may be greater than the maximum value of the preset hash value range. Therefore, in step S502, it is necessary to determine the multi-character vector according to different cases. Figure 6 A flowchart illustrating the process of determining multi-element character vectors is shown, such as... Figure 6 As shown, in step S601, it is determined whether the integer hash value is greater than the maximum value of the preset hash value range; in step S602, when the integer hash value is less than or equal to the maximum value, the target hash value corresponding to the integer hash value is determined in the second vector lookup table according to the integer hash value, and the vector corresponding to the target hash value is obtained as the multi-character vector; in step S603, when the integer hash value is greater than the maximum value, the integer hash value is modified to obtain an updated hash value corresponding to the integer hash value and within the preset hash value range, so as to determine the multi-character vector in the second vector lookup table according to the updated hash value. For the case of modifying the integer hash value, for example, if the preset hash value range is [0, 100], then the integer hash value can be divided by 100 and the remainder can be obtained. If the remainder is within the preset hash value range, then the vector corresponding to the multi-character information can be determined according to the vector corresponding to the remainder.

[0074] In step S230, the unary character vector and the multi-element character vector corresponding to the target character are fused to obtain the fused vector corresponding to the target character.

[0075] In one embodiment of this disclosure, after obtaining the unary character vector and the multi-element character vector corresponding to the target character, it can be achieved through... Figure 3 The fusion layer 304 in the middle fuses all word vectors corresponding to the target character to obtain the fusion vector corresponding to the target character. In the embodiments of this disclosure, the fusion vector corresponding to the target character can be determined based on the self-attention method according to the calculation formulas (1)-(3), specifically:

[0076] s i =v T tanh(w i x i (1)

[0077] A i =softmax(s i (2)

[0078]

[0079] Where, x i For univariate and multivariate character vectors, i = 1, 2, ..., n; w i Is with x i The corresponding weight coefficients; v is a trainable vector; s i Is with x i The corresponding calculated value; A i Is with x i The corresponding self-attention weights; Y is the fusion vector; n is the maximum number of characters corresponding to the multi-dimensional word vector.

[0080] Continuing with the example of multi-element character vectors, which include binary and ternary character vectors, then x i Let x1 be a unigram character vector, x2 be a unigram character vector, and x3 be a ternary character vector, i = 1, 2, 3. From the calculation formulas (1)-(3), it can be seen that after obtaining the unigram character vector x1, the unigram character vector x2, and the ternary character vector x3, the calculated values ​​s1, s2, and s3 corresponding to the unigram character vector, the unigram character vector, and the ternary character vector can be obtained according to the calculation formula (1). Then, the calculated values ​​s1, s2, and s3 are normalized according to the obtained calculated values ​​s1, s2, and s3 to obtain the self-attention weights A1, A2, and A3 corresponding to each calculated value. Finally, the fusion vector Y corresponding to the target character can be obtained by weighted summing the self-attention weights A1, A2, and A3 and the unigram character vector x1, the unigram character vector x2, and the ternary character vector x3.

[0081] Each character in the text to be processed is taken as the target character. According to the above embodiment, the unigram character vector and multi-element character vector corresponding to each target character are processed to obtain the fusion vector corresponding to each character. Then, named entity recognition can be performed based on the fusion vector corresponding to each character.

[0082] In step S240, after obtaining the fusion vectors corresponding to all characters in the word sequence, named entity recognition is performed based on the fusion vectors corresponding to all characters to obtain named entity annotation information corresponding to the text to be processed.

[0083] In one embodiment of this disclosure, after obtaining the fusion vectors corresponding to all characters in the text to be processed, the fusion layer 304 can send the fusion vectors corresponding to all characters to the named entity recognition layer 305, so that the named entity recognition layer 305 can perform named entity recognition on each character in the text to be processed based on the fusion vectors corresponding to all characters.

[0084] In one embodiment of this disclosure, the named entity recognition layer 305 can be a named entity annotation model layer based on a convolutional neural network, which has the characteristics of fast computing speed and is suitable for use in real-time scenarios, including CPU environments. Figure 7 A schematic diagram of the named entity recognition layer 305 is shown, as follows: Figure 7 As shown, the named entity recognition layer 305 includes convolutional layers 701-1, 701-2, ..., 701-N (N is a positive integer) and a normalization layer 702. The convolutional layers 701-1, 701-2, ..., 701-N are used to perform convolution and activation operations on the fusion vectors corresponding to all characters in the text to be processed, so as to obtain the representation vectors corresponding to each character. The normalization layer 702 is used to normalize the representation vectors corresponding to each character to obtain the named entity information corresponding to each character.

[0085] Next, the specific working principle of the named entity recognition layer 305 will be explained.

[0086] Each convolutional layer 701-1, 701-2, ..., 701-N mainly consists of two operations: a convolution operation and an activation operation. When performing a convolution operation on the fused vector, the fused vectors of the target character and the characters within the range of the convolution width D can be concatenated according to the set convolution width D. Then, a parameter matrix is ​​applied to the concatenated fused vector to obtain the vector after the convolution operation. The specific calculation formula is shown in formula (4):

[0087] X i =WX concat (4)

[0088] Among them, X i Let W be the vector resulting from the convolution operation with the target character, and let X be a parameter matrix. concat This is the merged vector after splicing.

[0089] It is worth noting that the characters within the convolution width D usually refer to the characters located within a range of D / 2 to the left and right of the target character.

[0090] After the convolution operation is completed, the activation operation can be performed. Its main function is to process the input vector through an activation function. In the embodiments of this disclosure, the ReLU function can be used, and its specific form is shown in formula (5):

[0091]

[0092] Of course, other activation functions can also be used to process the input vector, which will not be described in detail here.

[0093] After each convolutional layer processes the fusion vector corresponding to all characters and outputs the representation vector corresponding to each character, the normalization layer 702 performs normalization processing based on the representation vector corresponding to each character to obtain the named entity annotation information corresponding to each character, and outputs the named entity annotation information through the output layer 306. In the embodiments of this disclosure, the named entity annotation method for the text to be processed can adopt the BIO annotation method, or the BIOES annotation method, or other annotation methods, which are not specifically limited in this embodiment. In the BIO annotation method, B represents the beginning (Begin), BX represents the beginning of entity X, I represents the intermediate (Intermediate), IX represents the non-first position of the entity, and O (Other) represents not belonging to any type; in the BIOES annotation method, B (Begin) represents the beginning, I (Intermediate) represents the middle, E (End) represents the end, S (Single) represents a single character, and O (Other) represents others, used to mark irrelevant characters. Taking the text to be processed, “Singer C said”, if the BIOES annotation method is used, the final output named entity annotation information is: [B-PER,I-PER,E-PER,O]. If the BIO annotation method is used, the final output named entity annotation information is: [B-PER,I-PER,I-PER,O].

[0094] In the embodiments of this disclosure, before applying the named entity recognition model for named entity recognition, it is necessary to train the named entity recognition model to optimize the parameters in the model and obtain a stable named entity recognition model. For the parameters of the named entity recognition layer in the model, optimization can be performed based on the named entity annotation loss function determined by the named entity annotation information output by the named entity recognition layer. In the embodiments of this disclosure, the expression of the named entity annotation loss function is shown in formula (6):

[0095]

[0096] Where k is the k-th character in the text, P k is the probability value of the entity type corresponding to the k-th character, and n is the total number of characters contained in the text.

[0097] It is worth noting that the named entity recognition layer 305 can be a named entity annotation model layer based on a convolutional neural network, or it can be a named entity annotation model layer based on a recurrent neural network, BERT, or LSTM / GRU. The embodiments disclosed herein will not be described in detail here.

[0098] In the embodiments of this disclosure, after serializing the text to be processed at the character level, a character sequence with each independent character as a token can be obtained. When performing named entity recognition based on the character sequence, both the unary character vector corresponding to each token and the multi-dimensional character vectors corresponding to each token, such as binary and ternary character vectors, are considered for named entity recognition. This avoids word segmentation errors and incorporates the complete basic semantics of words during named entity recognition, thus improving the accuracy of named entity recognition. However, since the space of multi-dimensional character vectors corresponding to multi-dimensional character information is very large, it is difficult to fully train the multi-dimensional character vectors using only labeled training data. In other words, during the training of the named entity recognition model, some unary and multi-dimensional character vectors will not be trained. As a parameter of the named entity recognition model, the unary and multi-dimensional character vectors will be randomly initialized during each model run. Consequently, when the named entity recognition model vectorizes the character information in the text to be processed, there will be inaccurate vectorization results due to random vector initialization, which in turn reduces the accuracy of named entity recognition. To address this issue, this disclosure proposes a method for pre-training N-gram vectors from unlabeled text. By introducing pre-trained N-gram vectors during named entity recognition, the accuracy of named entity recognition is further improved. The method for pre-training character vectors in this disclosure will be described in detail below.

[0099] Figure 8 The flowchart illustrating the pre-training process of N-gram vectors is shown, as follows: Figure 8 As shown, in step S801, each text sentence in the unlabeled text is serialized at the character level to obtain character sequence samples corresponding to each text sentence; in step S802, any character in the character sequence sample is taken as the target sample character, and the text sentence is traversed according to the target sample character and the preset window to determine the unary character information or the multi-character information to be predicted; in step S803, N-gram character vectors are pre-trained based on the target sample character, the unary character information to be predicted and the first vector lookup table, or based on the target sample character, the multi-character information to be predicted, the first vector lookup table and the second vector lookup table.

[0100] Further, when pre-training the N-gram vectors in step S803, the pre-training can be specifically performed according to the following method: determine the first input vector corresponding to the target sample character according to the target sample character and the first vector lookup table, and at the same time determine the first output vector corresponding to the one-character information to be predicted according to the one-character information to be predicted and the first vector lookup table; then determine the first loss function according to the first input vector and the first output vector, and perform parameter optimization based on the first loss function; or, determine the second input vector corresponding to the target sample character according to the target sample character and the first vector lookup table, and at the same time determine the second output vector corresponding to the multi-character information to be predicted according to the multi-character information to be predicted and the second vector lookup table; then determine the second loss function according to the second input vector and the second output vector, and perform parameter optimization based on the second loss function.

[0101] To make the method of pre-training N-gram vectors clearer, a specific example will be given below for illustration.

[0102] For the text "After visiting Tencent", first serialize the text at the character granularity to obtain the character sequence "After visiting Tencent", where each character is a token; then for each token, use the current token to predict the tokens within the surrounding window L. For example, if the target sample character is "ten", and L = 2, then based on the target sample character, traverse the text with this window to obtain the one-character information to be predicted such as "visiting", "company" and the multi-character information to be predicted such as "After visiting", "company", "company after". Furthermore, the word vector corresponding to the token "ten" can be used to predict the word vectors corresponding to the one-character information to be predicted such as "visiting", "company", and can also predict the word vectors corresponding to the multi-character information to be predicted such as "After visiting", "company", "company after".

[0103] After determining the target sample character, the one-character information to be predicted, and the multi-character information to be predicted, the first loss function and the second loss function can be determined according to the input vector corresponding to the target sample character and the output vectors corresponding to the one-character information to be predicted and the multi-character information to be predicted. Furthermore, parameter optimization can be performed based on the first loss function and the second loss function.

[0104] Among them, the expressions of the first loss function and the second loss function are the same, as shown in formula (7):

[0105]

[0106] Among them, Loss + is the first loss part, Loss - is the second loss part, v wi is the input vector corresponding to the target sample character, v wo is the output vector corresponding to the unary character information to be predicted or the multi-character information to be predicted, v' wk is the input vector corresponding to the characters randomly selected and not belonging to the text sentence, k is the number of characters randomly selected and not belonging to the text sentence, and N is the maximum number of characters randomly selected and not belonging to the text sentence.

[0107] In the embodiments of the present disclosure, the first loss part corresponds to the loss function for positive example training, and the second loss part corresponds to the loss function for negative example training. Here, positive examples refer to the characters appearing within the preset window, such as "参", "公", etc., and negative examples refer to other characters randomly selected from the dictionary that do not appear in the processed text, such as "王", "家", etc. After completing the pre-training of the N-gram vectors, the vectors in the first vector query table and the second vector query table can be initialized with the pre-trained N-gram vectors. Then, when converting the character vectors of the processed text through the named entity recognition model, vectorization can be performed based on the already pre-trained vector query table, ensuring the accuracy of the unary character vectors and multi-character vectors, and thus also ensuring the accuracy of the finally obtained named entity annotation information.

[0108] The named entity recognition method in the embodiments of the present disclosure first performs a character-level serialization process on the processed text to obtain the corresponding character sequence; then determines the unary character vector and multi-character vector corresponding to each character in the character sequence based on the character and its corresponding context; then fuses the unary character vector and multi-character vector corresponding to the character to obtain the corresponding fusion vector; finally, after obtaining the fusion vectors corresponding to all characters in the character sequence, named entity recognition is performed based on the fusion vectors corresponding to all characters to obtain the named entity annotation information corresponding to the processed text. On the one hand, the technical solution of the present disclosure integrates the pre-trained N-gram vectors in the named entity recognition model, and thus in the process of named entity recognition, both the information of the characters and the complete basic semantics of the words are incorporated, improving the accuracy of the named entity annotation information; on the other hand, the named entity recognition model in the embodiments of the present disclosure is a lightweight model, ensuring the real-time running speed and improving the named entity recognition efficiency.

[0109] The named entity recognition method in this disclosure can be applied to any scenario requiring named entity recognition, such as word segmentation tools, question-and-answer systems, recommendation systems, search modules of chat tools, and various information flow products. Applying the named entity recognition method in various scenarios improves the efficiency and accuracy of named entity recognition, enabling product platforms to quickly provide users with more accurate feedback, further enhancing user experience and increasing user stickiness. For example, in a question-and-answer system, when a user inputs the question "When was the Model 3 released?", the system can use the named entity recognition method in this disclosure for quick identification, recognizing "Model 3" as a vehicle model entity. Since the user wants time information, the system can search for the release time attribute of the vehicle model in the background and provide the obtained release time attribute to the user.

[0110] The following describes an apparatus embodiment of this disclosure, which can be used to execute the named entity recognition method in the above embodiments of this disclosure. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the named entity recognition method described above.

[0111] Figure 9 A block diagram of a named entity recognition device according to an embodiment of the present disclosure is shown schematically.

[0112] Reference Figure 9 As shown, a named entity recognition device 900 according to an embodiment of the present disclosure includes: a serialization processing module 901, a character vector determination module 902, a character vector fusion module 903, and a named entity recognition module 904.

[0113] The system includes a serialization module for performing character-level serialization on the text to be processed to obtain a character sequence corresponding to the text; a character vector determination module for taking any character in the character sequence as a target character and determining a unigram character vector and a multi-dimensional character vector corresponding to the target character based on the target character and the context corresponding to the target character; a character vector fusion module for fusing the unigram character vector and the multi-dimensional character vector corresponding to the target character to obtain a fused vector corresponding to the target character; and a named entity recognition module for performing named entity recognition based on the fused vectors corresponding to all characters in the character sequence after obtaining the fused vectors corresponding to all characters to obtain named entity annotation information corresponding to the text to be processed.

[0114] In one embodiment of this disclosure, the character vector determination module 902 includes: a character information determination unit, configured to determine unary character information and multi-element character information based on the target character and the context corresponding to the target character; a unary character vector determination unit, configured to determine the unary character vector according to the unary character information and a first vector lookup table corresponding to the unary character information; and a multi-element character vector determination unit, configured to determine the multi-element character vector according to the multi-element character information and a second vector lookup table corresponding to the multi-element character information.

[0115] In one embodiment of this disclosure, the first vector lookup table includes a plurality of unary characters and vectors corresponding to each unary character; the unary character vector determination unit is configured to: match the unary character information with the unary characters in the first vector lookup table; when there is a target unary character that matches the unary character information, obtain the vector corresponding to the target unary character as the unary character vector.

[0116] In one embodiment of this disclosure, the multi-character vector determination unit includes: a hash processing unit, configured to perform hash processing on the multi-character information to obtain an integer hash value corresponding to the multi-character information; and a vector query unit, configured to determine the multi-character vector based on the integer hash value and the second vector query table.

[0117] In one embodiment of this disclosure, the second vector lookup table includes multiple hash values ​​in a preset hash value range and vectors corresponding to each hash value; the vector lookup unit is configured to: determine whether the integer hash value is greater than the maximum value of the preset hash value range; when the integer hash value is less than or equal to the maximum value, determine a target hash value corresponding to the integer hash value in the second vector lookup table based on the integer hash value, and obtain the vector corresponding to the target hash value as the multi-element word vector; when the integer hash value is greater than the maximum value, perform a modulo operation on the integer hash value to obtain an updated hash value corresponding to the integer hash value and within the preset hash value range, so as to determine the multi-element word vector in the second vector lookup table based on the updated hash value.

[0118] In one embodiment of this disclosure, the word vector fusion module 903 is configured to calculate the fusion vector according to the following formulas (1)-(3):

[0119] s i =v T tanh(w i x i (1)

[0120] A i =softmax(s i (2)

[0121]

[0122] Where, x i Let i = 1, 2, ..., n be the univariate word vector and the multivariate word vector; w i Is with x i The corresponding weight coefficients; v is a trainable vector; s i Is with x i The corresponding calculated value; A i Is with x i The corresponding self-attention weights; Y is the fusion vector; n is the maximum number of characters corresponding to the multi-dimensional word vector.

[0123] In one embodiment of this disclosure, the named entity recognition method is executed by a named entity recognition model, which includes an input layer, a serialization layer, a vectorization layer, a fusion layer, a named entity recognition layer, and an output layer.

[0124] In one embodiment of this disclosure, the named entity recognition layer includes a plurality of sequentially connected convolutional layers and normalization layers; the named entity recognition module 904 is configured to: perform convolution and activation operations on the fusion vectors corresponding to all characters through the convolutional layers to obtain representation vectors corresponding to each character; and perform normalization processing on the representation vectors through the normalization layers to obtain named entity information corresponding to all characters.

[0125] In one embodiment of this disclosure, the named entity recognition device 900 further includes: a pre-training module, used to pre-train N-gram vectors using unlabeled text, so as to determine the unary character vector and the multi-element character vector based on the target character, the context corresponding to the target character, and the pre-trained N-gram vector.

[0126] In one embodiment of this disclosure, the pre-training module includes: a serialization unit, configured to perform character-level serialization processing on each text sentence in the unannotated text to obtain character sequence samples corresponding to each text sentence; a character information acquisition unit, configured to take any character in the character sequence samples as a target sample character, and traverse the text sentences according to the target sample character and a preset window to determine unary character information or multi-dimensional character information to be predicted; and a pre-training unit, configured to pre-train N-gram vectors based on the target sample character, the unary character information to be predicted, and the first vector lookup table, or based on the target sample character, the multi-dimensional character information to be predicted, the first vector lookup table, and the second vector lookup table.

[0127] In one embodiment of this disclosure, the pre-training unit is configured to: determine a first input vector corresponding to the target sample character based on the target sample character and the first vector lookup table, and simultaneously determine a first output vector corresponding to the unary character information to be predicted based on the unary character information to be predicted and the first vector lookup table; determine a first loss function based on the first input vector and the first output vector, and perform parameter optimization based on the first loss function; or, determine a second input vector corresponding to the target sample character based on the target sample character and the first vector lookup table, and simultaneously determine a second output vector corresponding to the unary character information to be predicted based on the unary character information to be predicted and the second vector lookup table; determine a second loss function based on the second input vector and the second output vector, and perform parameter optimization based on the second loss function.

[0128] In one embodiment of this disclosure, both the first loss function and the second loss function include a first loss part and a second loss part; the loss function is determined based on the input vector and the output vector, and the parameters are optimized based on the loss function as follows: the loss function is determined according to the following formula (4):

[0129]

[0130] Among them, Loss + For the first loss part, Loss - For the second loss part, v wi v is the input vector corresponding to the target sample character. wo v' is the output vector corresponding to the unary character information or the multivariate character information to be predicted. wk The input vector is the randomly selected characters that do not belong to the text sentence, k is the number of randomly selected characters that do not belong to the text sentence, and N is the maximum number of randomly selected characters that do not belong to the text sentence.

[0131] In one embodiment of this disclosure, the multi-element character information includes binary character information and ternary character information; the character information determining unit is configured to: use the target character as the unary character information; form the binary character information based on the target character and the next character or identifier adjacent to the target character; form the ternary character information based on the target character, the previous character or identifier adjacent to the target character, and the next character adjacent to the target character, or form the ternary character information based on the target character, the previous character adjacent to the target character, and the next character or identifier adjacent to the target character.

[0132] Figure 10 A schematic diagram of the structure of a computer system suitable for implementing the embodiments of the present disclosure is shown.

[0133] It should be noted that, Figure 10 The computer system 1000 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0134] like Figure 10 As shown, the computer system 1000 includes a Central Processing Unit (CPU) 1001, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 1002 or programs loaded from storage portion 1008 into Random Access Memory (RAM) 1003, thereby implementing the search string processing method described in the above embodiments. The RAM 1003 also stores various programs and data required for system operation. The CPU 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An Input / Output (I / O) interface 1005 is also connected to the bus 1004.

[0135] The following components are connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to I / O interface 1005 as needed. Removable media 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1010 as needed so that computer programs read from them can be installed into storage section 1008 as needed.

[0136] In particular, according to embodiments of this disclosure, the processes described below with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1009, and / or installed from removable medium 1011. When the computer program is executed by central processing unit (CPU) 1001, it performs various functions defined in the system of this disclosure.

[0137] It should be noted that the computer-readable medium shown in the embodiments of this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0138] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0139] The units described in the embodiments of this disclosure can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the unit itself.

[0140] In another aspect, this disclosure also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to perform the methods described in the above embodiments.

[0141] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0142] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of this disclosure.

[0143] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein.

[0144] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.< / end> < / end> < / beg> < / end> < / beg>

Claims

1. A method for named entity recognition, characterized in that: include: Performing character-granularity serialization processing on the text to be processed to obtain a character sequence corresponding to the text to be processed; Taking any character in the word sequence as a target character, determining unigram character information and multi-character information based on the target character and a context corresponding to the target character; the unigram character information is the target character, and the multi-character information is a character string consisting of the target character and at least one character adjacent to the target character, and the length of which is less than a set length; Determine the unigram vector corresponding to the target character according to the unigram character information and a first vector lookup table corresponding to the unigram character information; the first vector lookup table includes a plurality of unigram characters and vectors corresponding to each unigram character; Determine the multi-character vector corresponding to the target character according to the hash value corresponding to the multi-character information and a second vector lookup table corresponding to the multi-character information; the second vector lookup table includes a plurality of hash values ​​in a preset hash value interval and vectors corresponding to each hash value; Fusing the unigram word vector and the multi-word vector corresponding to the target character to obtain a fused vector corresponding to the target character; After obtaining the fusion vectors corresponding to all the characters in the word sequence, named entity recognition is performed according to the fusion vectors corresponding to all the characters to obtain named entity annotation information corresponding to the text to be processed.

2. The method according to claim 1, characterized in that The determining the unigram vector corresponding to the target character according to the unigram character information and a first vector lookup table corresponding to the unigram character information comprises: Matching the unary character information with the unary characters in the first vector query table; When there is a target unigram character matching the unigram character information, a vector corresponding to the target unigram character is obtained as the unigram word vector.

3. The method according to claim 1, characterized in that The determining the multi-element word vector corresponding to the target character according to the hash value corresponding to the multi-element character information and the second vector lookup table corresponding to the multi-element character information includes: Performing hash processing on the multi-character information to obtain an integer hash value corresponding to the multi-character information; The multivariate word vector is determined based on the integer hash value and the second vector lookup table.

4. The method according to claim 3, characterized in that The determining the multi-element word vector according to the integer hash value and the second vector lookup table comprises: Determine whether the integer hash value is greater than the maximum value of the preset hash value interval; When the integer hash value is less than or equal to the maximum value, determining a target hash value corresponding to the integer hash value in the second vector lookup table according to the integer hash value, and obtaining a vector corresponding to the target hash value as the multi-element word vector; When the integer hash value is greater than the maximum value, a modulo process is performed on the integer hash value to obtain an updated hash value corresponding to the integer hash value and between the preset hash value interval, so as to determine the multi-element word vector in the second vector query table according to the updated hash value.

5. The method according to claim 1, characterized in that The step of fusing the unary word vector and the multi-element word vector corresponding to the target character to obtain a fused vector corresponding to the target character includes: The fusion vector is calculated according to the following formulas (1)-(3): S i =v T tanh(w i x i ) (1) A i =softmax(s i ) (2) Among them, x i are the unigram word vector and the multi-element word vector, i=1,2…,n; w i is with x i The corresponding weight coefficient; v is a trainable vector; s i is with x i The corresponding calculated value; A i is with x i The corresponding self-attention weight; Y is the fusion vector; n is the maximum number of characters corresponding to the multi-character vector.

6. The method according to claim 1, characterized in that The named entity recognition method is executed by a named entity recognition model, which includes an input layer, a serialization layer, a vectorization layer, a fusion layer, a named entity recognition layer and an output layer.

7. The method according to claim 6, characterized in that The named entity recognition layer includes a plurality of convolutional layers and normalization layers connected in sequence; The step of performing named entity recognition according to the fusion vectors corresponding to all characters to obtain named entity annotation information corresponding to the text to be processed includes: Performing convolution operations and activation operations on the fusion vectors corresponding to all characters through the convolution layer to obtain representation vectors corresponding to each character; The representation vector is normalized by the normalization layer to obtain named entity information corresponding to all the characters.

8. The method according to claim 1, characterized in that The method further comprises: The unlabeled text is used to pre-train N-gram vectors to determine the unigram vector and the multi-word vector according to the target character, the context corresponding to the target character, and the pre-trained N-gram vectors.

9. The method according to claim 8, characterized in that The method of using unlabeled text to pre-train N-gram vectors includes: Performing character granularity serialization processing on each text sentence in the unlabeled text to obtain a character sequence sample corresponding to each text sentence; Taking any character in the word sequence sample as a target sample character, traversing the text sentence according to the target sample character and a preset window to determine the unary character information to be predicted or the multi-ary character information to be predicted; An N-gram vector is pre-trained based on the target sample character, the unary character information to be predicted and the first vector lookup table, or based on the target sample character, the unary character information to be predicted, the first vector lookup table and the second vector lookup table.

10. The method according to claim 9, characterized in that The pre-training of the N-gram vector based on the target sample character, the to-be-predicted unigram character information and the first vector lookup table, or based on the target sample character, the to-be-predicted multi-gram character information, the first vector lookup table and the second vector lookup table, comprises: Determine a first input vector corresponding to the target sample character according to the target sample character and the first vector query table, and determine a first output vector corresponding to the unary character information to be predicted according to the unary character information to be predicted and the first vector query table; determining a first loss function according to the first input vector and the first output vector, and performing parameter optimization based on the first loss function; or Determine a second input vector corresponding to the target sample character according to the target sample character and the first vector lookup table, and determine a second output vector corresponding to the to-be-predicted multi-character information according to the to-be-predicted multi-character information and the second vector lookup table; A second loss function is determined according to the second input vector and the second output vector, and parameter optimization is performed based on the second loss function.

11. The method according to claim 10, characterized in that The first loss function and the second loss function both include a first loss part and a second loss part; Determining a loss function according to the input vector and the output vector, and performing parameter optimization based on the loss function, including: The loss function is determined according to the following formula (4): Among them, Loss + is the first loss part, Loss - is the second loss part, v wi is the input vector corresponding to the target sample character, v wo is the output vector corresponding to the unary character information to be predicted or the multi-character information to be predicted, v' wk is the input vector corresponding to the randomly selected characters that do not belong to the text sentence, k is the number of randomly selected characters that do not belong to the text sentence, and N is the maximum number of randomly selected characters that do not belong to the text sentence.

12. The method according to claim 1, characterized in that The multi-character information includes binary character information and ternary character information; The determining of the unary character information and the multi-ary character information based on the target character and the context corresponding to the target character comprises: Using the target character as the unary character information; Forming the binary character information according to the target character and a next character or identifier adjacent to the target character; The tri-character information is formed according to the target character, the previous character or identifier adjacent to the target character, and the next character adjacent to the target character, or the tri-character information is formed according to the target character, the previous character adjacent to the target character, and the next character or identifier adjacent to the target character.

13. A named entity recognition device, characterized in that: include: A serialization processing module, used for performing character granularity serialization processing on the text to be processed to obtain a character sequence corresponding to the text to be processed; A word vector determination module, used to take any character in the word sequence as a target character, determine unigram character information and multi-character information based on the target character and a context corresponding to the target character, determine a unigram word vector corresponding to the target character according to the unigram character information and a first vector query table corresponding to the unigram character information, and determine a multi-character word vector corresponding to the target character according to a hash value corresponding to the multi-character information and a second vector query table corresponding to the multi-character information; the unigram character information is the target character, the multi-character information is a character string consisting of the target character and at least one character adjacent to the target character, and the length of which is less than a set length, the first vector query table includes a plurality of unigram characters and vectors corresponding to each unigram character, and the second vector query table includes a plurality of hash values ​​in a preset hash value interval and vectors corresponding to each hash value; A word vector fusion module, used for fusing the unary word vector and the multi-element word vector corresponding to the target character to obtain a fused vector corresponding to the target character; The named entity recognition module is used to perform named entity recognition according to the fusion vectors corresponding to all characters in the word sequence after obtaining the fusion vectors corresponding to all characters, so as to obtain the named entity annotation information corresponding to the text to be processed.

14. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enables the one or more processors to implement the named entity recognition method as described in any one of claims 1 to 12.

15. A computer storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a processor, the method for named entity recognition as claimed in any one of claims 1 to 12 is implemented.

16. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor of a computer device, enable the computer device to implement the method for named entity recognition as claimed in any one of claims 1 to 12.

Images