Named entity identification method and device, computer equipment and storage medium

Abstract

The invention relates to a named entity recognition method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the steps of obtaining an original text to be recognized by a named entity; inputting the original text into the trained entity recognition model to obtain a named entity recognition result of the original text; wherein the trained entity recognition model comprises a BERT layer, an attention layer, a first adder and a CRF layer, the input end of the attention layer is connected with the output end of the BERT layer, the first input end of the first adder is connected with the output end of the attention layer, the second input end of the first adder is connected with the output end of the BERT layer, and the second input end of the first adder is connected with the output end of the CRF layer. The input end of the CRF layer is connected with the output end of the first adder, and the attention layer is used for extracting local features and entity boundary features. By adopting the method, the entity recognition capability can be improved, and the recognition accuracy can be improved.

Description

Technical Field

[0001] This application relates to the field of named entity recognition technology, and in particular to a named entity recognition method, apparatus, computer device and storage medium. Background Technology

[0002] Named Entity Recognition (NER) is a fundamental task in Natural Language Processing (NLP). Its goal is to identify meaningful "named entities" from text and classify them into predefined categories. BERT (Bidirectional Encoder Representations from Transformers) is a bidirectional pre-trained language model based on the Transformer encoder structure. It can understand the meaning of words in context and is used to improve the performance of text representation and downstream tasks.

[0003] In related technologies, NER entity recognition is achieved through entity recognition models including BERT, but there is a problem of low recognition capability. Summary of the Invention

[0004] Therefore, it is necessary to provide a named entity recognition method, apparatus, computer device, computer-readable storage medium, and computer program product that can improve recognition capabilities in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a named entity recognition method, including:

[0006] Obtain the original text to be named entity recognition;

[0007] The original text is input into the trained entity recognition model to obtain the named entity recognition result of the original text;

[0008] The trained entity recognition model includes a BERT layer, an attention layer, a first adder, and a CRF (Conditional Random Field) layer. The input of the attention layer is connected to the output of the BERT layer, the first input of the first adder is connected to the output of the attention layer, the second input of the first adder is connected to the output of the BERT layer, and the input of the CRF layer is connected to the output of the first adder. The attention layer is used to extract local features and entity boundary features.

[0009] The trained entity recognition model is obtained based on labeled text using entity boundary and type labeling rules.

[0010] In one embodiment, the attention layer includes a first EAM (Embedding Attention Module) layer, a SAM (Sequence Attention Module) layer, and a second adder. The inputs of the first EAM layer and the SAM layer are respectively connected to the output of the BERT layer. The output of the first EAM layer is connected to the first input of the second adder. The output of the SAM layer is connected to the second input of the second adder. The output of the second adder is connected to the first input of the first adder.

[0011] In one embodiment, the trained entity recognition model further includes a second EAM layer and a third adder. The second EAM layer is located between two adjacent coding layers inside the BERT layer. The input of the second EAM layer is connected to the output of the previous coding layer among the two adjacent coding layers. The output of the second EAM layer is connected to the first input of the third adder. The output of the previous coding layer is also connected to the second input of the third adder. The output of the third adder is connected to the input of the next coding layer among the two adjacent coding layers.

[0012] In one embodiment, the step of inputting the original text into a trained entity recognition model to obtain the entity recognition result of the original text includes:

[0013] The original text is input into the BERT layer of the trained entity recognition model to obtain the first text vector information;

[0014] The first text vector information is input into the attention layer of the trained entity recognition model to obtain the second text vector information;

[0015] The second text vector information and the first text vector information are input into the first adder for addition and fusion to obtain a fused vector;

[0016] The fusion vector is input into the CRF layer of the trained entity recognition model to obtain the named entity recognition result of the original text.

[0017] In one embodiment, the first adder is specifically used to perform element-level addition fusion or weighted addition fusion.

[0018] In one embodiment, the trained entity recognition model is obtained through the following steps:

[0019] Obtain the sample text and the corresponding labeled text, wherein the labeled text is labeled using entity boundary and type labeling rules;

[0020] The entity recognition model is trained using the sample text and the labeled text to obtain a trained entity recognition model.

[0021] Secondly, this application also provides a named entity recognition device, comprising:

[0022] The acquisition module is used to acquire the raw text to be identified by named entity recognition;

[0023] The recognition module is used to input the original text into the trained entity recognition model and obtain the named entity recognition result of the original text.

[0024] The trained entity recognition model includes a BERT layer, an attention layer, a first adder, and a CRF layer. The input of the attention layer is connected to the output of the BERT layer, the first input of the first adder is connected to the output of the attention layer, the second input of the first adder is connected to the output of the BERT layer, and the input of the CRF layer is connected to the output of the first adder. The attention layer is used to extract local features and entity boundary features.

[0025] The trained entity recognition model is obtained based on labeled text using entity boundary and type labeling rules.

[0026] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in the first aspect above.

[0027] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect above.

[0028] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect above.

[0029] The aforementioned named entity recognition method, apparatus, computer equipment, computer-readable storage medium, and computer program product first acquire the original text to be named entity recognized, then input the original text into a trained entity recognition model to obtain the named entity recognition result of the original text. The trained entity recognition model includes a BERT layer, an attention layer, a first adder, and a CRF layer. The input of the attention layer is connected to the output of the BERT layer, the first input of the first adder is connected to the output of the attention layer, the second input of the first adder is connected to the output of the BERT layer, and the input of the CRF layer is connected to the output of the first adder. The attention layer is used to extract local features and entity boundary features. The trained entity recognition model is obtained based on labeled text marked using entity boundary and type labeling rules. Therefore, by combining BERT, the attention mechanism, and the adder, local features and entity boundary features can be recognized, and the CRF can be used for decoding to obtain the entity recognition result, thereby improving entity recognition capability and accuracy. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a diagram illustrating the application environment of a named entity recognition method in one embodiment.

[0032] Figure 2 This is a flowchart illustrating a named entity recognition method in one embodiment;

[0033] Figure 3 This is a schematic diagram of the entity recognition model in one embodiment;

[0034] Figure 4 This is a schematic diagram of the entity recognition model in another embodiment;

[0035] Figure 5 Here is a schematic diagram of the structure of a BERT layer in an example;

[0036] Figure 6 for Figure 2 A flowchart illustrating step 202 in the middle section;

[0037] Figure 7 This is a structural block diagram of a named entity recognition device in one embodiment;

[0038] Figure 8This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0040] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0041] The definitions of abbreviations and key terms used in the embodiments of this application are as follows:

[0042] Named Entity Recognition (NER) is a fundamental task in natural language processing. Its goal is to identify "named entities" with specific meanings from text and classify them into predefined categories.

[0043] Convolutional Neural Network (CNN): A type of feedforward neural network with a convolutional layer structure, which is good at automatically extracting features from raw data and is widely used for data modeling with local correlations.

[0044] Bidirectional Long Short-Term Memory (BiLSTM) is an improved recurrent neural network structure that can simultaneously capture forward and backward contextual information in a sequence. It is widely used in natural language processing tasks such as text classification, sequence labeling, and NER.

[0045] BERT (Bidirectional Encoder Representations from Transformers) is a bidirectional pre-trained language model based on the Transformer encoder structure. It can understand the meaning of words in context and is used to improve the performance of text representation and downstream tasks.

[0046] CRF (Conditional Random Field): A discriminative model used to model the contextual dependencies between labeled sequences. It can output the globally optimal label sequence and is suitable for complex sequence prediction tasks.

[0047] Avionics System (AS): A collection of integrated electronic devices and systems that enable functions such as flight control and management, navigation and communication, mission execution and safety monitoring. It is a key foundation for the intelligence and automation of aircraft.

[0048] Embedding Attention Module (EAM): This is a module that combines attention mechanisms with embedding layers. It has wide applications in deep learning and can enhance feature representation and solve context dependency problems.

[0049] Sequence Attention Module (SAM): This is a module used in deep learning to process sequential data. It can automatically learn the importance of different positions in a sequence, thereby better capturing key information in the sequence.

[0050] In related technologies, NER technology is applied in various fields. For example, when applied in the aerospace field, there are the following three technical solutions:

[0051] First, the BiLSTM-CRF model is used to extract text information of avionics display equipment faults and to build a fault knowledge graph.

[0052] Second, fault information is extracted using the BERT-CRF model;

[0053] Third, based on the entity recognition model of BERT-BiLSTM-CRF, the event element entities in the content of Notice to Airmen E are extracted.

[0054] In the aforementioned approaches, the BiLSTM-CRF model is less capable of understanding complex text than the BERT-CRF model. However, BERT still has limitations in local modeling and entity boundary recognition. BERT-BiLSTM-CRF aims to further enhance the model's ability to model long-distance dependencies in text. However, in entity recognition tasks, entities are often embedded within the text. Therefore, the model needs to consider both global semantics and local features, but this model only focuses on global features. To address this, a CNN module is introduced. Although CNNs can effectively extract local features, their fixed window mechanism limits their ability to accurately perceive entity boundaries. Related technologies also introduce external dictionaries into the model, improving entity recognition accuracy by matching latent words in the dictionary. However, this method is highly dependent on dictionary quality. If applied to specialized fields, a specialized dictionary needs to be constructed, which is difficult to obtain and often requires significant manpower.

[0055] In summary, current named entity recognition technologies either focus solely on global features or extract local features only under limited perception of entity boundaries. However, global features, local features, and entity boundary features are extremely important. Therefore, existing technologies suffer from weak local modeling capabilities and inaccurate entity boundary recognition, leading to low recognition accuracy. While existing BERT-based models possess powerful contextual modeling capabilities, they perform poorly in scenes with nested entities or blurred entity boundaries. An improved method combining global and local information is urgently needed to enhance recognition accuracy.

[0056] Therefore, this application proposes a named entity recognition method to overcome the limitations of related technologies, improve recognition capabilities, and increase recognition accuracy.

[0057] The named entity recognition method provided in this application embodiment can be applied to, for example, Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104, or it can be located in the cloud or on another network server.

[0058] The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and drones. The server 104 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing cloud computing services.

[0059] In the embodiments of this application, such as Figure 2 As shown, a named entity recognition method is provided, which can be applied to... Figure 1Taking server 104 as an example, the explanation includes the following steps 201 and 202. Wherein:

[0060] Step 201: Obtain the original text of the entity to be named.

[0061] Raw text can be user-inputted text that has not yet been named entity recognition. It can be text from datasets in various fields such as healthcare, automotive, networking, aviation safety reports, and aviation products. Raw text may possess specialized characteristics, meaning it may include proper nouns of varying lengths, strong contextual dependencies, ambiguous boundaries, and other complex situations. Named entities are the entities that need to be recognized, such as fault information, equipment names, and operating status, depending on the specific requirements.

[0062] For example, the original text can be text to be identified in the field of automotive fault identification. Through the embodiments of this application, automotive fault information is extracted from the original text.

[0063] For example, the original text can be fault text of an avionics system. Through embodiments of this application, fault information of the avionics is extracted from the original text. Avionics systems play a crucial role in data processing, signal processing, and system control. In-depth analysis of the fault modes and mechanisms of avionics systems is essential for improving their reliability and safety. Over time, avionics fault diagnosis accumulates a large number of maintenance records, which provide valuable references for subsequent maintenance work. With the help of NER technology, fault information can be quickly extracted from maintenance texts, accelerating experience accumulation, forming a knowledge base, and helping maintenance personnel efficiently track, locate, and analyze avionics system faults, thereby reducing labor costs and mitigating the impact of subjective factors.

[0064] Step 202: Input the original text into the trained entity recognition model to obtain the named entity recognition result of the original text.

[0065] Among them, such as Figure 3 As shown, the trained entity recognition model includes a BERT layer 301, an attention layer 302, a first adder 303, and a CRF layer 304. The input of the attention layer 302 is connected to the output of the BERT layer 301, the first input of the first adder 303 is connected to the output of the attention layer 302, the second input of the first adder 303 is connected to the output of the BERT layer 301, and the input of the CRF layer 304 is connected to the output of the first adder 303. The attention layer 302 is used to extract local features and entity boundary features.

[0066] The trained entity recognition model is obtained based on the labeled text marked using entity boundary and type labeling rules.

[0067] Entity boundary and type marking rules can include BIO annotation rules, BIOES annotation rules, etc.

[0068] B (Begin): Indicates the beginning of an entity (entity header);

[0069] I (Inside): Indicates the middle or end part of an entity (the inside of the entity);

[0070] O (Outside): Indicates that the word or character does not belong to any entity (outside the entity);

[0071] E (End): The end of the entity;

[0072] S (Single): An entity consisting of a single element.

[0073] For example, the original text is used as the input sequence W = {w1, w2, ..., wn}, where n represents the length of the text. This sequence is then input into a trained entity recognition model. For instance, in the original text "I love fruit!", w1 represents "I", w2 represents "love", and w6 represents "!". BERT layer 301 can be BERT-base (12 layers). The original text is encoded through the embedding and encoding layers of BERT layer 301 to obtain the contextual semantic vector information for each word. This vector information has 768-dimensional embedding features, which can fully express the semantic information of the text and its dependence on the context. The vector information enters the attention layer 302 for feature enhancement through an attention mechanism, extracting local features and entity boundary features. The attention layer 302 can include multiple parallel self-attention layers. The vector information output from BERT layer 301 and the vector information output from attention layer 302 are both input into the first adder 303 for addition and fusion to obtain a fused vector, which is then output to CRF layer 304. Conditional random fields are used to annotate and globally decode the sequence, obtaining the final entity recognition result {O,O,B,...,I}, where OOBI represents the output label of CRF, where O represents the entity exterior, B represents the entity head, and I represents the entity interior. The BI label can represent an entity, such as "I love fruit!", where the entity is fruit, and the corresponding label is {O,O,B,I,O,O}. CRF can consider the dependencies between labels, making the output sequence superior in terms of semantic consistency and structural rationality.

[0074] For example, suppose the input is "I love fruit!", and the entity is fruit. Each word entering BERT will be mapped to a 768-dimensional embedding vector. Then the input sentence will become a (6, 768) vector. The attention layer further extracts features from the BERT output, extracting entity boundaries and local features, and then adds and fuses them with the BERT output. While preserving the original structure, the fused features are fed into CRF for decoding the answer label and extracting the entity "fruit".

[0075] In the aforementioned named entity recognition method, the original text enters through BERT, then passes through the attention layer to further enhance data features, extracting local features and entity boundary features. The output of the attention layer is then added to and fused with the output of the BERT layer, and finally, the data is transformed into a label sequence using CRF to obtain entities. Therefore, by combining BERT, the attention mechanism, and the adder, local features and entity boundary features can be further identified, and the result is decoded using CRF to obtain the entity recognition result, thus improving entity recognition capability and accuracy.

[0076] In one exemplary embodiment, such as Figure 4 As shown, the attention layer includes a first EAM layer 401, a SAM layer 402, and a second adder 403. The input terminals of the first EAM layer 401 and the SAM layer 402 are respectively connected to the output terminal of the BERT layer 301. The output terminal of the first EAM layer 401 is connected to the first input terminal of the second adder 403. The output terminal of the SAM layer 402 is connected to the second input terminal of the second adder 403. The output terminal of the second adder 403 is connected to the first input terminal of the first adder 303.

[0077] The first EAM layer, 401, treats the high-dimensional embedding space as a multi-channel semantic carrier. By assigning dynamic weights to different embedding dimensions, it allows the model to focus more on dimensions carrying key information (such as the specialized semantic dimensions related to entities like "module failure" and "detection method" in aviation malfunction texts), while reducing attention to noisy or redundant dimensions. This helps the model accurately capture important local information in the text, especially suited to the characteristics of aviation malfunction texts, which contain many technical terms and have strong local contextual dependencies. This effect also occurs in other professional fields.

[0078] SAM layer 402: By modeling the interaction between global and local contexts at the sequence level, it aggregates element features of similar entities and suppresses interference from elements near entity boundaries, thereby improving the model's entity boundary detection capability and entity extraction accuracy.

[0079] For example, the output of the original text after BERT layer 301 is further processed by the first EAM layer 401 and SAM layer 402 to extract features. The first EAM layer 401 and SAM layer 402 process in parallel. The first EAM layer 401 assigns attention weights to different dimensions of each word embedding vector, highlighting dimensions containing key information while suppressing noise dimensions, thereby enhancing the model's ability to capture semantic details. The implementation references the channel attention mechanism, combining matrix transformation and residual connections to ensure feature stability. The SAM layer 402 introduces convolutional compression and attention modeling in the sequence dimension to model the dependencies between positions in the sequence, highlighting structural information near entity boundaries. Convolutional operations not only reduce computational complexity but also enhance the model's ability to perceive local contextual patterns, improving boundary recognition accuracy. The outputs of the first EAM layer 401 and the SAM layer 402 are added and fused by the second adder 403, and then input into the first adder 303. The first adder 303 adds and fuses the fused features with the BERT output and outputs them to the CRF for decoding to obtain the entity recognition result of the original text. This allows the model to obtain key information from different aspects (embedding and sequence).

[0080] EAM enhances the model's ability to perceive important semantic features by dynamically weighting the embedding dimension, thereby optimizing content learning performance. SAM improves the model's ability to identify entity locations and boundaries by modeling the global and local relationships of elements in the sequence domain. The features output by EAM and SAM are added and fused, and then superimposed with the BERT output using a residual connection method to preserve the original semantic information while improving the model's information representation ability in deep structures and mitigating feature degradation.

[0081] For example, suppose the input is "I love fruit!" and the entity is fruit. Each word in the BERT input will be mapped to a 768-dimensional embedding vector. The input sentence will then become a (6, 768) data set. The attention layer further extracts features from the BERT output. SAM extracts key features in the sequence dimension (the 6th dimension in the data set), which is beneficial for entity boundary detection. EAM extracts key features in the 768th dimension. Extracting key features in the embedding dimension can further optimize model performance. The two modules are then added and fused, and finally added to the BERT output again. This structure is a residual structure. To preserve the original features, the result is fed into a CRF for decoding and answering the label, extracting the entity "fruit".

[0082] Therefore, by further extracting key local features contained in the embedding dimension through EAM from the output of BERT, and further extracting entity boundary features through SAM, and then adding and fusing them before outputting them to CRF for label decoding to obtain entities, the entity recognition capability can be further improved. It is also suitable for handling complex situations in professional domain texts with proper nouns, ambiguous boundaries, and strong contextual dependencies. It has good generalization ability and engineering application value, and can also be extended to NER task scenarios in various industry fields.

[0083] To enhance BERT's ability to learn the content implied by the embedded dimension, an EAM module can be introduced into BERT.

[0084] That is, in an exemplary embodiment, such as Figure 5 As shown, the trained entity recognition model also includes a second EAM layer 501 and a third adder 502. The second EAM layer is located between two adjacent coding layers inside the BERT layer (such as between the third and fourth encoder layers). The input of the second EAM layer is connected to the output of the previous coding layer in the two adjacent coding layers. The output of the second EAM layer is connected to the first input of the third adder. The output of the previous coding layer is also connected to the second input of the third adder. The output of the third adder is connected to the input of the next coding layer in the two adjacent coding layers.

[0085] Figure 5 The diagram shows that the second EAM layer 501 and the third adder 502 are located between the sixth and seventh coding layers of the BERT layer, resulting in a BERT with inserted EAM. The output data of the sixth layer of the BERT coding layer is fed into the EAM module for learning and then fed into subsequent coding layers, enabling BERT to acquire embedded dimension information during text learning, thus compensating for BERT's deficiency in acquiring embedded dimension information.

[0086] For example, after the original data is encoded by the sixth layer of the BERT layer, features are further extracted by the second EAM layer 501. The output of the second EAM layer 501 and the output of the sixth layer of the BERT layer are added and fused together before being input into the seventh layer of the BERT layer for encoding.

[0087] Therefore, inserting EAM between two adjacent layers of the BERT encoder to obtain an EAM-inserted BERT can compensate for the shortcoming of BERT itself not paying attention to the embedded dimension information, optimize BERT performance, and further optimize the overall model with the attention layer to further improve the model's recognition ability.

[0088] In one exemplary embodiment, such as Figure 6 As shown, step 202 includes steps 601 to 604. Wherein:

[0089] Step 601: Input the original text into the BERT layer of the trained entity recognition model to obtain the first text vector information.

[0090] The first text vector information refers to the vector information obtained after the original text has undergone feature processing by the BERT layer.

[0091] The BERT layer consists of a connected second EAM layer and a third adder, located between two adjacent coding layers within the BERT layer. The third adder adds the output of the second EAM layer and the output of the previous coding layer and then outputs the result to the next coding layer.

[0092] Step 602: Input the first text vector information into the attention layer of the trained entity recognition model to obtain the second text vector information.

[0093] The second text vector information refers to the vector information obtained after the output of the BERT layer has been processed by the attention layer. The attention layer includes a first EAM layer, a SAM layer, and a second adder, and their connection relationships are described above.

[0094] Step 603: Input the second text vector information and the first text vector information into the first adder for addition and fusion to obtain the fused vector.

[0095] The first adder is specifically used for element-level addition fusion or weighted addition fusion.

[0096] Step 604: Input the fused vector into the CRF layer of the trained entity recognition model to obtain the named entity recognition result of the original text.

[0097] For example, the original text is "equipment cooling light frequently flashes in mid-air" indicating an avionics system malfunction, with the entity being the equipment name. When this text enters BERT, each character is mapped to a 768-dimensional embedding vector. The input sentence then becomes a (6, 768) data set. The attention layer further extracts features from the BERT output. SAM extracts key features in the sequence dimension (6 in the data set), which is beneficial for entity boundary detection; while EAM extracts key features in the 6, 7, and 8 dimensions. Extracting key features in the embedding dimension further optimizes model performance. The two modules are then fused together and added to the BERT output again. This structure is a residual structure. To preserve the original features, the result is fed into a CRF for decoding and answering the label, resulting in the entity recognition result {B, I, I, I, I, O, O, O}, which extracts the entity "equipment cooling light".

[0098] Therefore, by embedding an attention module into the BERT model, introducing a hybrid domain attention mechanism (embedded attention and sequence attention), and combining it with CRF for label decoding, the ability to perceive entity boundaries and model contextual features can be improved.

[0099] The entity recognition model described above was trained in advance based on training text.

[0100] In one exemplary embodiment, a trained entity recognition model is obtained through the following steps: obtaining sample text and corresponding labeled text, wherein the labeled text is labeled using entity boundary and type labeling rules; and training the entity recognition model using the sample text and labeled text to obtain a trained entity recognition model.

[0101] In one possible implementation, an entity recognition model is first constructed based on a neural network, which includes a BERT layer, an attention layer, a first adder, and a CRF layer, as well as a second EAM layer and a third adder. The attention layer comprises a first EAM layer, a SAM layer, and a second adder, with connections as described above. Figures 3 to 5 The process involves acquiring sample text and labeled text. The labeled text is generated using entity boundary and type labeling rules (BIO labeling rules, BIOES labeling rules). The labeled object is the target entity, such as faulty equipment or fault time, depending on the specific requirements. Each labeled text corresponds one-to-one with the sample text. The labeled text can be expert-labeled, derived from machine learning or deep learning, or a combination of both. The entity recognition model is iteratively trained using the sample text and labeled text until the iteration ends, resulting in a trained entity recognition model. This model is then used for entity recognition in various professional fields. The recognition capability can be optimized by continuously improving the parameters of the entity recognition model.

[0102] The above model can be used for professional text datasets such as avionics system fault text datasets, and its recognition capability is significantly improved. Experiments were also conducted on public datasets such as MSRA, Resume, and OntoNotes 4.0. The experimental results also show significant improvement compared with related technologies. Therefore, the above model has wide applicability.

[0103] In summary, the embodiments of this application can effectively improve the accuracy and robustness of NER in professional texts. This method comprehensively considers embedded semantic features and sequence structure information, introduces a hybrid domain attention mechanism to enhance the model's ability to identify key entities in fault texts, and combines CRF for decoding to achieve the globally optimal output of the label sequence. By dynamically weighting the embedding dimension through EAM, the model's ability to perceive important semantic features is enhanced, thereby optimizing the content learning effect. By modeling the global and local relationships of elements in the sequence domain through SAM, the model's ability to identify entity positions and boundaries is improved. EAM is introduced into BERT, enabling BERT to acquire information implied in the embedding dimension during text learning, making up for BERT's lack of information acquisition in the embedding dimension. A model architecture based on BERT encoder, hybrid domain attention and CRF decoder is constructed. This model can simultaneously focus on semantic content and entity position features, and is suitable for handling complex situations such as proper nouns, ambiguous boundaries and strong contextual dependencies in professional domain texts (such as avionics system fault texts). It has good generalization ability and engineering application value, and can also be extended to NER task scenarios in various industry fields.

[0104] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0105] Based on the same inventive concept, this application also provides a named entity recognition device for implementing the named entity recognition method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more named entity recognition device embodiments provided below can be found in the limitations of the named entity recognition method described above, and will not be repeated here.

[0106] In one exemplary embodiment, such as Figure 7 As shown, a named entity recognition device is provided, including: an acquisition module 701 and a recognition module 702, wherein:

[0107] The acquisition module 701 is used to acquire the original text to be named entity recognition;

[0108] The recognition module 702 is used to input the original text into the trained entity recognition model and obtain the named entity recognition result of the original text.

[0109] The trained entity recognition model includes a BERT layer, an attention layer, a first adder, and a CRF layer. The input of the attention layer is connected to the output of the BERT layer, the first input of the first adder is connected to the output of the attention layer, the second input of the first adder is connected to the output of the BERT layer, and the input of the CRF layer is connected to the output of the first adder. The attention layer is used to extract local features and entity boundary features.

[0110] The trained entity recognition model is obtained based on the labeled text marked using entity boundary and type labeling rules.

[0111] In one embodiment, the attention layer includes a first EAM layer, a SAM layer, and a second adder. The inputs of the first EAM layer and the SAM layer are respectively connected to the output of the BERT layer. The output of the first EAM layer is connected to the first input of the second adder. The output of the SAM layer is connected to the second input of the second adder. The output of the second adder is connected to the first input of the first adder.

[0112] In one embodiment, the trained entity recognition model further includes a second EAM layer and a third adder. The second EAM layer is located between two adjacent coding layers inside the BERT layer. The input of the second EAM layer is connected to the output of the previous coding layer in the two adjacent coding layers. The output of the second EAM layer is connected to the first input of the third adder. The output of the previous coding layer is also connected to the second input of the third adder. The output of the third adder is connected to the input of the next coding layer in the two adjacent coding layers.

[0113] In one embodiment, the identification module 702 includes a first identification unit, a second identification unit, a fusion unit, and a third identification unit, wherein:

[0114] The first recognition unit is used to input the original text into the BERT layer of the trained entity recognition model to obtain the first text vector information;

[0115] The second recognition unit is used to input the first text vector information into the attention layer of the trained entity recognition model to obtain the second text vector information.

[0116] The fusion unit is used to input the second text vector information and the first text vector information into the first adder for addition and fusion to obtain a fused vector;

[0117] The third recognition unit is used to input the fused vector into the CRF layer of the trained entity recognition model to obtain the named entity recognition result of the original text.

[0118] In one embodiment, the first adder is specifically used to perform element-level addition fusion or weighted addition fusion.

[0119] In one embodiment, the named entity recognition device further includes a training module, which is used to obtain a trained entity recognition model through the following steps: obtaining sample text and the corresponding labeled text, wherein the labeled text is labeled using entity boundary and type labeling rules; and training the entity recognition model using the sample text and the labeled text with entity labels to obtain a trained entity recognition model.

[0120] Each module in the aforementioned named entity recognition device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0121] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 8 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data related to named entity recognition. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements a named entity recognition method.

[0122] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0123] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in any of the above method embodiments.

[0124] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in any of the above method embodiments.

[0125] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of any of the above method embodiments.

[0126] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0127] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0128] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method of named entity recognition, characterized by, The method comprises: obtaining original text to be named entity recognition; inputting the original text into a trained entity recognition model to obtain a named entity recognition result of the original text; wherein the trained entity recognition model comprises a BERT layer, an attention layer, a first adder and a CRF layer, an input end of the attention layer is connected with an output end of the BERT layer, a first input end of the first adder is connected with an output end of the attention layer, a second input end of the first adder is connected with an output end of the BERT layer, an input end of the CRF layer is connected with an output end of the first adder, and the attention layer is used to extract local features and entity boundary features; the trained entity recognition model is obtained based on a marked text marked by using an entity boundary and type marking rule.

2. The method of claim 1, wherein, The attention layer comprises a first EAM layer, a SAM layer and a second adder, an input end of the first EAM layer and an input end of the SAM layer are respectively connected with an output end of the BERT layer, an output end of the first EAM layer is connected with a first input end of the second adder, an output end of the SAM layer is connected with a second input end of the second adder, and an output end of the second adder is connected with a first input end of the first adder.

3. The method of claim 1, wherein, The trained entity recognition model further comprises a second EAM layer and a third adder, the second EAM layer is located between two adjacent encoding layers inside the BERT layer, an input end of the second EAM layer is connected with an output end of an upper encoding layer of the two adjacent encoding layers, an output end of the second EAM layer is connected with a first input end of the third adder, the output end of the upper encoding layer is also connected with a second input end of the third adder, and an output end of the third adder is connected with an input end of a lower encoding layer of the two adjacent encoding layers.

4. The method according to any one of claims 1 to 3, characterized in that, The inputting of the original text into the trained entity recognition model to obtain the entity recognition result of the original text comprises: inputting the original text into the BERT layer of the trained entity recognition model to obtain first text vector information; inputting the first text vector information into the attention layer of the trained entity recognition model to obtain second text vector information; inputting the second text vector information and the first text vector information into the first adder for addition fusion to obtain a fusion vector; inputting the fusion vector into the CRF layer of the trained entity recognition model to obtain the named entity recognition result of the original text.

5. The method of claim 1, wherein, The first adder is specifically used for element-level addition fusion or weighted addition fusion.

6. The method of claim 1, wherein, The trained entity recognition model is obtained by the following steps: obtaining sample text and a marked text corresponding to the sample text, wherein the marked text is marked by using an entity boundary and type marking rule; training an entity recognition model by using the sample text and the marked text to obtain a trained entity recognition model.

7. A named entity recognition apparatus characterized by comprising: The device comprises: an obtaining module configured to obtain original text to be named entity recognition; The identification module is configured to input the original text into a trained entity identification model to obtain a named entity recognition result of the original text. The trained entity identification model comprises a BERT layer, an attention layer, a first adder, and a CRF layer. An input end of the attention layer is connected with an output end of the BERT layer. A first input end of the first adder is connected with an output end of the attention layer. A second input end of the first adder is connected with an output end of the BERT layer. An input end of the CRF layer is connected with an output end of the first adder. The attention layer is configured to extract local features and entity boundary features. The trained entity identification model is obtained based on a marked text which is marked by using an entity boundary and type marking rule. 8.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-7. The computer program, when executed by the processor, implements the steps of the method of any one of claims 1-6.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the method of any one of claims 1-6.

10. A computer program product comprising a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the method of any one of claims 1-6.

Images