Self-adaptive learning named entity recognition method, system and equipment based on span representation and storage medium

Abstract

The invention provides an adaptive learning named entity recognition method, system and device based on span representation and a storage medium, and the method comprises the following steps: obtaining historical data of a target, preprocessing the historical data to generate a training sample set, and expanding the diversity of the training sample set through a data enhancement mode; according to the training sample set, the entity is marked through a suspension marking method, by means of the method, the text nested entity can be effectively recognized, uneven distribution of labels is relieved, the diversity of the training sample set is expanded through data enhancement, and robust data enhancement is achieved through an adaptive learning algorithm. According to the method, firstly, data enhancement is carried out, then a text is emphasized by using suspension marks, the suspension marks are grouped, an unbalanced loss function is adopted for training, adaptive learning is used for online calculation and enhancement of data weight during training, and experimental results show that the method effectively improves the model effect.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method, system, device and storage medium for adaptive learning named entity recognition based on span representation. Background Art

[0002] Named entity recognition is the primary task of information extraction, which aims to extract pre-defined entity information from text. In the field of text processing, identifying and classifying key entities in text, such as names of people, places, and organizations, is called named entity recognition (NER), which can also be understood as proper noun recognition. This process involves determining the boundaries of entities in the text and the categories they belong to. As a key branch of natural language processing (NLP), the development of NER has gone through several stages, including the initial rule-based and dictionary-based systems, followed by machine learning methods that rely on statistics, such as hidden Markov models (HMM), and up to the technology based on deep learning frameworks that are widely used today.

[0003] Traditional machine learning methods use HMM and CRF to solve the shortcomings of dictionary and rule matching.

[0004] The two algorithm models are relatively simple and have few parameters, but

[0005] It is difficult to achieve advanced results. Random deep learning technology continues to develop, and its effects are constantly improving, including the named entity recognition model based on BiLSTM+CRF. At the end of 2018, the pre-trained model BERT was proposed, which combines multiple layers of transformers in series and dynamically generates word vectors based on the text context semantic information, refreshing multiple task list records of natural language processing tasks. From then on, the BERT model can be used to obtain word representations when the NLP task model is used as a baseline, and its effect in NER tasks is also significantly improved. The use of the pre-trained model BERT for entity recognition also includes a large number of architectures.

[0006] Current research finds it difficult to effectively handle nested recognition problems when performing entity recognition tasks. At the same time, in specific vertical fields, data annotation requires relevant domain knowledge and expert assistance, which is costly and inefficient. It is difficult to achieve good results in supervised named entity recognition tasks.

[0007] When dealing with small sample entity recognition tasks, although the use of data augmentation technology includes a variety of implementation ideas, the methods are relatively simple, and methods that do not add constraints to data augmentation still need to be further studied. Unrestricted data augmentation will generate a large number of negative samples, thereby reducing the accuracy and robustness of the model. At the same time, in complex scenarios, entity recognition has limited ability to handle entity nesting problems. Therefore, it is necessary to design a stable data augmentation method to more comprehensively identify entities and enhance model capabilities. To this end, an adaptive learning named entity recognition method, system, device, and storage medium based on span representation are proposed. Summary of the invention

[0008] In view of this, the present invention provides a method, system, device and storage medium for adaptive learning named entity recognition based on span representation. The method can effectively identify nested entities in text, and use an unbalanced loss function to alleviate uneven label distribution. In low-resource scenarios, data enhancement is performed, and an adaptive algorithm is proposed to online calculate enhanced data weights to solve or alleviate technical problems existing in the prior art, thereby providing at least a beneficial option.

[0009] The technical solution of the embodiment of the present invention is achieved as follows:

[0010] In a first aspect, the present invention provides a method for adaptive learning named entity recognition based on span representation, comprising the following steps:

[0011] Acquire historical data of the target, wherein the historical data is preprocessed to generate a training sample set, and the training sample set is expanded in diversity by data enhancement;

[0012] Marking entities using a suspended marking method according to the training sample set, grouping the suspended markings, and grouping adjacent data based on an adjacent grouping strategy;

[0013] In response to the training sample set, the model is trained through an adaptive learning algorithm, the learning path is automatically optimized according to feedback during the training process, the weight of the training sample set is adjusted, and the positive benefit of the data to the model is enhanced.

[0014] Further preferably, the data enhancement method includes: entity word replacement and context replacement; the data enhancement method randomly replaces entity words and context words in the sentence according to a ratio of 4:6 for the data of the training sample set;

[0015] The suspension marking method establishes contextual connections between different entities. The suspension marking method uses span representation for the data of the training sample set, wherein the marking entity includes a start character and an end character; and at the same time, the suspension mark represented by the marking entity is added to the end of the text.

[0016] Further preferably: the adaptive learning algorithm specifically includes:

[0017] Recalculate each batch of data; first sample the original data and enhanced data of this batch, and initialize the weight parameters;

[0018] Calculate the loss of the model on the augmented data;

[0019] Use gradient descent to obtain new model parameters;

[0020] The new model parameters are inferred on the original data to obtain the loss;

[0021] Use the loss of the new model on the original data to update the weight parameters of the model.

[0022] Further preferably, the model is a Word2vec algorithm, and the Word2vec algorithm finds several words closest to the word as its synonyms by calculating cosine similarity.

[0023] Further preferably, the method further comprises generating an imbalanced loss function in response to the imbalanced label distribution, specifically:

[0024]

[0025] Among them, M represents the number of data, y i,c is the label of whether sample i belongs to category c, z i,c is the raw score of the model output for sample i corresponding to category c, and μ is the adjustment factor, which takes a fixed value of 2.

[0026] Further preferably, the adjacent grouping strategy includes sorting all the floating marks according to the size of the starting mark id, selecting a fixed size, dividing the floating marks into multiple groups, and constructing multiple pieces of data.

[0027] Further preferably, the grouped floating mark specifically includes:

[0028] A pair of suspension marks is set for each span, and the suspension marks correspond to the start and end characters of the span;

[0029] Encoding the start and end characters of the corresponding span in accordance with the shared position of the span boundary;

[0030] A pair of the suspended markers are connected via a directed attention mechanism.

[0031] Aspect 2: An adaptive learning named entity recognition system based on span representation, comprising:

[0032] A data acquisition module is used to acquire historical data of a target, wherein the historical data is preprocessed to generate a training sample set, and the diversity of the training sample set is expanded by data enhancement;

[0033] A suspension marking module; used to mark entities according to the training sample set by a suspension marking method, group the suspension marks, and group adjacent data based on an adjacent grouping strategy;

[0034] Model training module; used to perform model training through an adaptive learning algorithm in response to the training sample set, automatically optimize the learning path according to feedback during the training process, adjust the weight of the training sample set, and enhance the positive benefits of data to the model;

[0035] A label distribution module; used to generate an imbalanced loss function in response to an imbalanced label distribution.

[0036] A third aspect: A computer device comprising:

[0037] processor;

[0038] A memory for storing executable instructions;

[0039] The processor is used to read the executable instructions from the memory and execute the executable instructions to implement the adaptive learning named entity recognition method based on span representation.

[0040] A fourth aspect: A computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor implements the adaptive learning named entity recognition method based on span representation.

[0041] The embodiment of the present invention has the following advantages due to the adoption of the above technical solution:

[0042] First, the present invention is aimed at the problem of named entity recognition in low-resource scenarios, so that better performance can be achieved without the need for a large amount of data annotation, and robust data enhancement can be achieved.

[0043] Second, the present invention introduces a suspended mark method for span representation, using suspended marks to represent entities, strengthening the modeling of entity boundaries and type information, and thus obtaining better entity context representation.

[0044] Third, compared with the prior art, the present invention has better effects. The present invention does not require a high amount of data annotation in specific vertical fields, can effectively alleviate the long-tail problem, and perform data enhancement more robustly.

[0045] The above summary is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present invention will be readily apparent by reference to the accompanying drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 The present invention is a flowchart of the adaptive learning named entity recognition method based on span representation.

[0048] Figure 2 This is a schematic diagram of the computer device structure of Example 3 of the present invention.

[0049] Figure 3 This is a diagram of the architecture of the adaptive learning named entity recognition based on span representation of the present invention.

[0050] Figure 4 It is the overall architecture diagram of the model of the present invention.

[0051] Figure 5 This is an illustration of the directional attention mechanism of the present invention.

[0052] Figure 6 This is a floating mark grouping strategy diagram of the present invention.

[0053] Figure 7 It is a diagram of the data enhancement method of the present invention.

[0054] Figure 8 It is a flow chart of the adaptive learning algorithm of the present invention.

[0055] Fig. 9 It is a pseudo code diagram of the adaptive learning algorithm of the present invention.

[0056] In the figure, 10 is a computer device; 1002 is a processor; 1004 is a memory; and 1006 is a transmission system. DETAILED DESCRIPTION

[0057] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the spirit or scope of the present invention. Therefore, the drawings and descriptions are considered to be exemplary and non-restrictive in nature.

[0058] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0059] Example 1

[0060] like Figure 1 As shown, a span representation-based adaptive learning named entity recognition method includes the following steps:

[0061] Acquire historical data of the target, wherein the historical data is preprocessed to generate a training sample set, and the training sample set is expanded in diversity by data enhancement;

[0062] Marking entities using a suspended marking method according to the training sample set, grouping the suspended markings, and grouping adjacent data based on an adjacent grouping strategy;

[0063] In response to the training sample set, the model is trained through an adaptive learning algorithm, the learning path is automatically optimized according to feedback during the training process, the weight of the training sample set is adjusted, and the positive benefit of the data to the model is enhanced.

[0064] At present, the entity recognition model based on BIo annotation cannot handle nested or overlapping entities in text, and it is difficult to recognize long entities and entities with unclear boundaries. The span-based representation method can concisely and effectively solve the entity nesting problem existing in most named entity recognition models. The research methods of span representation enumerate all span pairs and use span feature fusion to determine the entity type. These methods only obtain word representations through pre-trained models, and then splice the span representation prediction results. Although the boundary information of the entity is integrated, the context information of the entity is not fully utilized.

[0065] like Figure 4 As shown in the figure, in the entity extraction stage, a pair of tags is introduced for each span representation and concatenated at the end of the article to strengthen the entity's learning of the context. This tag is called a suspended tag. At the same time, all tags are grouped to model the relationship between different entities and entity boundary information. Finally, the suspended tag representation and width representation are introduced for the entity information fusion method, so as to more effectively improve the effect of the university teacher entity recognition model.

[0066] For suspended markers, use Figure 5The directional attention mechanism is proposed. Specifically, the suspended marker and its corresponding word share position encoding. Different suspended marker pairs are invisible to each other, and the same suspended marker pair is visible to each other. For the original sentence, all suspended markers are invisible. In addition, the suspended markers of all spans are spliced ​​after the sentence, so it is necessary to divide all suspended markers into multiple groups. After sorting all suspended markers, the adjacent suspended markers are grouped together to establish the connection between different entities. The specific method is to construct a directional attention mask matrix. When calculating the attention of the original text, the mask value of the text part of this row of the matrix is ​​set to 1, and the rest is set to 0. When calculating the attention of the suspended marker corresponding to the span, the mask value of the span start and end markers and the text position is set to 1, and the rest is set to 0. In this way, the original text will not notice the suspended marker when calculating the attention, and the original text can be seen when calculating the suspended marker, and other markers will not be seen.

[0067] The strategy grouping method is as follows Figure 6 As shown, specifically: the input text is represented as X = {x1, x2, x3, ..., xn}, where n is the length of the input sentence, then all entities can be enumerated (the entity consists of the start position and the end position characters), and n×(n+1)2 entities can be obtained. In actual training, it is necessary to limit the maximum length of the entity, thereby reducing the number of training samples and reducing the training time. The maximum length of an entity can be determined based on the actual situation of the task to limit the number of entities to be predicted (the maximum length is used as a hyperparameter). Considering the relationship between entity pairs, the model uses a suspended marker to model the relationship between entity pairs. For the input data of the model, some suspended markers are added to represent the head and tail characters of the entity.

[0068] When the model is initially trained, Figure 7 Data augmentation is performed in the manner shown in the figure. Using entity word replacement and context replacement can generate a large amount of training data. However, the model is highly sensitive to context semantics. Therefore, during model training, it is necessary to make every effort to learn data that is beneficial to improving the model effect while reducing data that has a negative impact on model performance. After data augmentation, the following methods are used: Figure 8 The adaptive algorithm shown measures data quality, which is specifically expressed as follows: The implementation process uses recalculation of each batch of data. First, sample the original data (X, Y) and enhanced data (Xp, Yp) of this batch. And initialize the weight parameter ω. The value of each item of ω represents the contribution to the loss function. Specifically: The first step is to calculate the loss of the model on the enhanced data.

[0069]

[0070] Among them, Θ represents the training parameters of the entity recognition model, and f(Xp,Θ) represents the call of the model using parameter Θ. Then the new model parameters are obtained by gradient descent (the original model parameters are not updated at this time). Grad_descent represents the gradient descent function, and the Adam optimizer is used for optimization.

[0071]

[0072] Using this new model Inference is performed on the original data to obtain the loss Lt, and no weight parameters are added to the original dataset because the original data is high-quality labeled data.

[0073]

[0074] Using the new model The loss Lt on the original data is used to update the weight parameter ω of the model. The new model is only valid for the current data, that is, it is not saved to the next batch of data.

[0075]

[0076]

[0077] Where σ is the activation function, using the Re lu function. n is the number of data in a batch of the augmented dataset. In this way, the updated weight parameters are obtained to measure the importance of each sample in the augmented dataset. The weights are standardized to ensure that the weights are in the range of 0-1. Finally, the updated weight parameters are used to calculate the loss value of the augmented data and actually update the target model.

[0078]

[0079] The above realizes the adaptive learning of the model, that is, without changing the model and using additional hyperparameters, the model can learn the importance of samples to reduce the impact of negative samples on the model, thereby improving the model effect. The pseudo code of the adaptive algorithm model training is as follows Fig. 9 shown.

[0080] In this embodiment: the data enhancement method includes: entity word replacement and context replacement; the data enhancement method randomly replaces the entity words and context words in the sentence according to the ratio of 4:6 for the data of the training sample set;

[0081] The suspension marking method establishes contextual connections between different entities. The suspension marking method uses span representation for the data of the training sample set, wherein the marking entity includes a start character and an end character; and at the same time, the suspension mark represented by the marking entity is added to the end of the text.

[0082] In this embodiment: the adaptive learning algorithm specifically includes:

[0083] Recalculate each batch of data; first sample the original data and enhanced data of this batch, and initialize the weight parameters;

[0084] Calculate the loss of the model on the augmented data;

[0085] Use gradient descent to obtain new model parameters;

[0086] The new model parameters are inferred on the original data to obtain the loss;

[0087] Use the loss of the new model on the original data to update the weight parameters of the model.

[0088] In this embodiment, the model is the Word2vec algorithm, which finds several words closest to the word as its synonyms by calculating cosine similarity.

[0089] In this embodiment: the method further includes generating an imbalanced loss function in response to the imbalanced label distribution, specifically:

[0090]

[0091] Among them, M represents the number of data, y i,c is the label of whether sample i belongs to category c, z i,c is the raw score of the model output for sample i corresponding to category c, and μ is the adjustment factor, which takes a fixed value of 2.

[0092] For the accurate classification samples (1-p i,c ) μ Approaching 0, for misclassified samples (1-p i,c ) μ It approaches 1, that is, compared with the cross entropy loss, the loss of this loss function is almost not attenuated for samples with inaccurate classification, and the loss becomes smaller for samples with accurate classification. This is equivalent to increasing the weight of inaccurately classified samples in the loss function.

[0093] In this embodiment: the adjacent grouping strategy includes sorting all the floating marks according to the size of the starting mark id, selecting a fixed size, dividing the floating marks into multiple groups, and constructing multiple data.

[0094] In this embodiment: the group suspension mark specifically includes:

[0095] A pair of suspension marks is set for each span, and the suspension marks correspond to the start and end characters of the span;

[0096] Encoding the start and end characters of the corresponding span in accordance with the shared position of the span boundary;

[0097] A pair of the suspended markers are connected via a directed attention mechanism.

[0098] The suspension mark grouping method sets a pair of suspension marks [O][ / O] for each span. The suspension marks [O][ / O] correspond to the start and end characters of the span, and share the position code with the boundary of the span. Then, the pair of marks is connected through directional attention. The suspension marks share the position code with the corresponding words. Different suspension mark pairs are invisible to each other, and the same suspension mark pair is visible to each other. For the original sentence, all suspension marks are invisible.

[0099] In this embodiment: data enhancement, the specific method is: type adopts two methods of entity word replacement and context replacement to enhance data. In entity word replacement, the entities in the sentence are randomly replaced with entities of the same type in the entity word dictionary. The entity word dictionary is constructed by traversing all entities in the training sample set data and adding them to the corresponding type, that is, the key of the dictionary is the entity type, and the value is the entity set of the entity type. For context replacement, the context is defined as non-entity words after sentence segmentation. For these non-entity words, the model can be based on the Word2vec algorithm, and find the closest words to the word as its synonyms by calculating the cosine similarity. In a sentence, entity word replacement and context replacement are performed in a certain proportion to balance entity diversity and context diversity.

[0100] In this embodiment: the adaptive learning algorithm enables the model to have the ability of adaptive learning, and can be efficiently fine-tuned on different tasks, that is, to maximize the sensitivity of the new task loss function to the parameters, so that a small change in the parameters can cause a large fluctuation in the loss function. This method can guide the enhanced data according to the gradient direction of the original data, and correct the enhanced data gradient by minimizing the loss on the original data set, providing robust data enhancement.

[0101] Example 2

[0102] like Figure 3 Shown: An adaptive learning named entity recognition system based on span representation, comprising:

[0103] A data acquisition module is used to acquire historical data of a target, wherein the historical data is preprocessed to generate a training sample set, and the diversity of the training sample set is expanded by data enhancement;

[0104] A suspension marking module; used to mark entities according to the training sample set by a suspension marking method, group the suspension marks, and group adjacent data based on an adjacent grouping strategy;

[0105] Model training module; used to perform model training through an adaptive learning algorithm in response to the training sample set, automatically optimize the learning path according to feedback during the training process, adjust the weight of the training sample set, and enhance the positive benefits of data to the model;

[0106] A label distribution module; used to generate an imbalanced loss function in response to an imbalanced label distribution.

[0107] The foregoing Figure 1 The various variations and specific examples of an adaptive learning named entity recognition method based on span representation in Example 1 are also applicable to an adaptive learning named entity recognition system based on span representation in this embodiment. Through the above detailed description of an adaptive learning named entity recognition method based on span representation, those skilled in the art can clearly understand the implementation method of an adaptive learning named entity recognition system based on span representation in this embodiment, so for the sake of brevity of the specification, it will not be described in detail here.

[0108] Example 3

[0109] An embodiment of the present application provides a computer device, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement an adaptive learning named entity recognition method based on span representation as provided in the above method embodiment.

[0110] Figure 2 The hardware structure diagram of a device for implementing an adaptive learning named entity recognition method based on span representation provided by an embodiment of the present application is shown. The device can participate in or include the system or system provided by an embodiment of the present application. Figure 2 As shown, the computer device 10 may include one or more processors 1002 (the processor may include but is not limited to a processing system such as a microprocessor MCU or a programmable logic device FPGA), a memory 1004 for storing data, and a transmission system 1006 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 2 The structure shown is only for illustration and does not limit the structure of the above electronic system. Figure 2More or fewer components as shown, or with Figure 2 Different configurations shown.

[0111] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer device 10 (or mobile device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0112] The memory 1004 can be used to store software programs and modules of application software, such as a program instruction / data storage system corresponding to an adaptive learning named entity recognition method based on span representation in an embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 1004, that is, implementing the above-mentioned method. The memory 1004 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage systems, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1004 may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer device 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0113] The transmission system 1006 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer device 10. In one example, the transmission system 1006 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission system 1006 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0114] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer device 10 (or mobile device).

[0115] Example 4

[0116] An embodiment of the present application also provides a computer-readable storage medium, which can be set in a server to store at least one instruction or at least one program related to an adaptive learning named entity recognition method based on span representation in an embodiment of the method. The at least one instruction or the at least one program is loaded and executed by the processor to implement an adaptive learning named entity recognition method based on span representation provided in the above-mentioned embodiment of the method.

[0117] Optionally, in this embodiment, the storage medium may be located in at least one of the multiple network servers of the computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0118] Example 5

[0119] The embodiment of the present invention also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes an adaptive learning named entity recognition method based on span representation provided in the above various optional implementations.

[0120] It should be noted that the above-mentioned sequence of the embodiments of the present application is for description only and does not represent the advantages and disadvantages of the embodiments. The above-mentioned specific embodiments of the present application are described. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0121] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system, device, and storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0122] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0123] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.

[0124] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of various changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. An adaptive learning named entity recognition method based on span representation, characterized in that: Applied on a span basis, the following steps are involved: Acquire historical data of the target, wherein the historical data is preprocessed to generate a training sample set, and the training sample set is expanded in diversity by data enhancement; Marking entities using a suspended marking method according to the training sample set, grouping the suspended markings, and grouping adjacent data based on an adjacent grouping strategy; In response to the training sample set, the model is trained through an adaptive learning algorithm, the learning path is automatically optimized according to feedback during the training process, the weight of the training sample set is adjusted, and the positive benefit of the data to the model is enhanced.

2. The method according to claim 1, characterized in that: The data enhancement method includes: entity word replacement and context replacement; the data enhancement method randomly replaces the entity words and context words in the sentence according to the ratio of 4:6 for the data of the training sample set; The suspension marking method establishes contextual connections between different entities. The suspension marking method uses span representation for the data of the training sample set, wherein the marking entity includes a start character and an end character; and at the same time, the suspension mark represented by the marking entity is added to the end of the text.

3. The method according to claim 1, characterized in that: The adaptive learning algorithm specifically includes: Recalculate each batch of data; first sample the original data and enhanced data of this batch, and initialize the weight parameters; Calculate the loss of the model on the augmented data; Use gradient descent to obtain new model parameters; The new model parameters are inferred on the original data to obtain the loss; Use the loss of the new model on the original data to update the weight parameters of the model.

4. The method according to claim 3, characterized in that: The model is the Word2vec algorithm, which finds several words closest to the word as its synonyms by calculating cosine similarity.

5. The method according to claim 1, characterized in that: The method also includes generating an imbalanced loss function in response to the imbalanced label distribution, specifically: Among them, M represents the number of data, y i,c is the label of whether sample i belongs to category c, z i,c is the raw score of the model output for sample i corresponding to category c, and μ is the adjustment factor, which takes a fixed value of 2.

6. The method according to claim 1, characterized in that: The adjacent grouping strategy includes sorting all the suspended markers according to the size of the starting marker id, selecting a fixed size, dividing the suspended markers into multiple groups, and constructing multiple pieces of data.

7. The method according to claim 1, characterized in that: The group suspension mark specifically includes: A pair of suspension marks is set for each span, and the suspension marks correspond to the start and end characters of the span; Encoding the start and end characters of the corresponding span in accordance with the shared position of the span boundary; A pair of the suspended markers are connected via a directed attention mechanism.

8. An adaptive learning named entity recognition system based on span representation, applied to the adaptive learning named entity recognition method based on span representation according to any one of claims 1 to 7, characterized in that: include: A data acquisition module is used to acquire historical data of a target, wherein the historical data is preprocessed to generate a training sample set, and the diversity of the training sample set is expanded by data enhancement; Floating mark module; Used to mark entities by a suspended marking method according to the training sample set, group the suspended markings, and group adjacent data based on an adjacent grouping strategy; Model training module; Used to perform model training through an adaptive learning algorithm in response to the training sample set, automatically optimize the learning path according to feedback during the training process, adjust the weight of the training sample set, and enhance the positive benefits of data to the model; Label distribution module; Used to generate an imbalanced loss function in response to the imbalanced label distribution.

9. A computer device, characterized in that: include: processor; A memory for storing executable instructions; The processor is used to read the executable instructions from the memory and execute the executable instructions to implement the adaptive learning named entity recognition method based on span representation as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the adaptive learning named entity recognition method based on span representation as claimed in any one of claims 1 to 7.

Images