Nested entity identification method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of natural language processing, and provides a nested entity recognition method and device, electronic equipment and a storage medium, and the method comprises the steps: determining a recognition instruction and a to-be-recognized text; inputting the recognition instruction and the to-be-recognized text into the nested entity recognition model to obtain a recognition result output by the nested entity recognition model; wherein the nested entity recognition model is obtained based on sample data, a positive sample pair and a negative sample pair by taking a nested entity recognition task and a contrast learning task as training tasks for joint training, and the sample data comprises a sample instruction and a sample text and a sample recognition result corresponding to the sample instruction; the positive sample pair and the negative sample pair are determined based on the sample data and are used for guiding the nested entity recognition model to carry out comparative learning. By introducing the contrast learning task, the characterization capability of the nested entity recognition model can be enhanced, so that the accuracy of nested entity recognition is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a nested entity recognition method, device, electronic device and storage medium. Background Art

[0002] Named Entity Recognition (NER) is a basic task in the field of natural language processing, which aims to identify entities with specific meanings in text, including names of people, places, institutions, proper nouns, etc. This task plays a key role in the effects of downstream tasks such as text generation and sentiment analysis. Nested Named Entity Recognition (NER) further processes the nested relationship between entities based on named entity recognition and extracts named entities with hierarchical or nested relationships from text.

[0003] Traditional named entity recognition methods, such as rule-based, statistical-based, and deep learning-based methods, have achieved certain results in the task of named entity recognition, but these methods are unable to cope with nested entities and have limited recognition effects on nested entities. In addition, although current research has proposed methods such as sequence-to-sequence and graph neural networks to handle nested entities, the effects of these methods in practical applications are still limited. For nested entities, it is easy to over-extract or under-extract, which makes it difficult to meet the high-precision requirements of nested entity recognition. Summary of the invention

[0004] The present invention provides a nested entity recognition method, device, electronic device and storage medium, which are used to solve the defect in the related art that nested entities are prone to over-extraction or under-extraction, resulting in low recognition accuracy.

[0005] The present invention provides a nested entity recognition method, comprising: Determine a recognition instruction and a text to be recognized, wherein the recognition instruction is used to describe a nested entity recognition task; Inputting the recognition instruction and the text to be recognized into a nested entity recognition model to obtain a recognition result output by the nested entity recognition model; Among them, the nested entity recognition model is obtained by joint training based on sample data, positive sample pairs and negative sample pairs, with nested entity recognition tasks and contrastive learning tasks as training tasks. The sample data includes sample instructions and sample text and sample recognition results corresponding to the sample instructions. The positive sample pairs and the negative sample pairs are used to guide the nested entity recognition model to perform contrastive learning.

[0006] According to a nested entity recognition method provided by the present invention, the training step of the nested entity recognition model includes: Acquire sample data, and determine positive sample pairs and negative sample pairs based on the sample data; Determining a contrast loss based on the positive sample pair and the negative sample pair; Inputting sample instructions and sample text in the sample data into an initial model to obtain a predicted recognition result output by the initial model, and determining a recognition loss based on the predicted recognition result and a sample recognition result in the sample data; Based on the recognition loss and the contrast loss, parameters of the initial model are iterated to obtain the nested entity recognition model.

[0007] According to a nested entity recognition method provided by the present invention, the determining of positive sample pairs and negative sample pairs based on the sample data includes: Extracting features of each entity type in the sample instruction to obtain an embedding vector of each entity type; Performing feature extraction on each entity and each edge information in the sample text to obtain an embedding vector of each entity and an embedding vector of each edge information; Determine a positive sample pair based on an embedding vector of any entity type and an embedding vector of an entity corresponding to the any entity type, wherein the entity corresponding to the any entity type is determined based on the sample recognition result; A negative sample pair is determined based on the embedding vector of any entity type and the embedding vector of any edge information.

[0008] According to a nested entity recognition method provided by the present invention, determining the contrast loss based on the positive sample pair and the negative sample pair includes: Determining a first distance based on the embedding vector of the entity type and the embedding vector of the entity in the positive sample pair; Determining a second distance based on the embedding vector of the entity type and the embedding vector of the edge information in the negative sample pair; Based on the first distance and the second distance, a contrast loss is determined.

[0009] According to a nested entity recognition method provided by the present invention, the obtaining of sample data includes: Collect data about nested entities; Performing fuzzy deduplication on the nested entity related data, and cleaning the deduplicated data to obtain initial nested entity data; Based on the initial nested entity data, sample data is constructed.

[0010] According to a nested entity recognition method provided by the present invention, fuzzy deduplication of the nested entity related data is performed, and the deduplication data is cleaned, including: Deduplication of the nested entity related data is performed based on a text similarity measurement algorithm and / or a clustering algorithm; The deduplicated data is cleaned based on preset filtering rules, wherein the preset filtering rules include at least one of language-based filtering rules, metric-based filtering rules, and keyword-based filtering rules.

[0011] According to a nested entity recognition method provided by the present invention, the initial model is a pre-trained large model, and the pre-trained large model refers to a model pre-trained based on a training data set, and the pre-trained large model adopts a Decoder-Only architecture.

[0012] The present invention also provides a nested entity recognition device, comprising: A determination unit, used to determine a recognition instruction and a text to be recognized, wherein the recognition instruction is used to describe a nested entity recognition task; A recognition unit, used for inputting the recognition instruction and the text to be recognized into a nested entity recognition model to obtain a recognition result output by the nested entity recognition model; Among them, the nested entity recognition model is obtained by joint training based on sample data, positive sample pairs and negative sample pairs, with nested entity recognition tasks and contrastive learning tasks as training tasks. The sample data includes sample instructions and sample text and sample recognition results corresponding to the sample instructions. The positive sample pairs and the negative sample pairs are used to guide the nested entity recognition model to perform contrastive learning.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the computer program, any of the nested entity recognition methods described above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for recognizing nested entities as described in any one of the above is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the nested entity recognition methods described above.

[0016] The nested entity recognition method, device, electronic device and storage medium provided by the present invention can obtain accurate recognition results by inputting recognition instructions and text to be recognized into a nested entity recognition model, and the recognition instructions are used to describe the nested entity recognition task, so that the present invention is highly flexible and can adapt to different nested entity recognition needs. Since the nested entity recognition model is based on sample data, positive sample pairs and negative sample pairs, and is obtained by joint training with nested entity recognition tasks and contrastive learning tasks as training tasks, the model can better learn the characteristics and laws of nested entities. By introducing contrastive learning tasks, the representation ability of the nested entity recognition model can be enhanced, thereby improving the accuracy of recognition, so that the model can more accurately recognize nested entities in the text, and effectively reduce the situation of over-extraction or under-extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a schematic diagram of the flow of the nested entity recognition method provided by the present invention; Figure 2 It is a schematic diagram of the process of training the nested entity recognition model provided by the present invention; Figure 3 is a schematic diagram of a positive and negative sample pair provided by the present invention; Figure 4 It is a flow chart of a nested entity recognition method based on contrastive learning enhanced large model provided by the present invention; Figure 5 It is a structural schematic diagram of a nested entity recognition device provided by the present invention; Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] Nested entity recognition is a key task in the field of natural language processing, which aims to extract named entities with hierarchical or nested relationships from text. A nested entity refers to an entity that contains other types of entities. For example, in the address information "a certain street in a certain district of A city", "A city" is a location entity, "a certain district" is a sub-entity under "A city", and "a certain street" is a sub-entity under "a certain district", which constitutes a nested entity.

[0021] Traditional named entity recognition (NER) methods, such as rule-based methods, statistical methods, and deep learning-based methods, have achieved certain results in the recognition of non-nested entities, but they all have some limitations when dealing with nested entities. Among them, rule-based methods have high development costs and are difficult to expand to large-scale data; statistical-based methods, such as Hidden Markov Model (HMM) and Conditional Random Field (CRF), are prone to overfitting or insufficient generalization; and deep learning-based methods, such as the BERT (Bidirectional Encoder Representation from Transformers) model, although there are improvements in performance, still require a large amount of labeled data as support. These traditional NER methods have limited effects on nested entity recognition and cannot handle nested relationships between entities.

[0022] To address this problem, current research has proposed methods based on sequence to sequence, graph neural network, and multi-feature fusion to identify nested entities. Among them, although the sequence to sequence method can handle nested entities, its complex decoding process increases the difficulty of implementation and computational cost. The method based on graph neural network represents entities and their relationships by constructing a graph structure, but its effect in practical applications is still limited. The method based on multi-feature fusion uses feature extraction fusion layer and decoding layer to process by fusing the word embedding representation and part-of-speech embedding representation of the text. However, some of these methods rely on a large amount of labeled data and have high requirements on the data set, while others are prone to over-extraction or under-extraction for nested entities, resulting in inaccurate recognition results.

[0023] In this regard, the present invention provides a nested entity recognition method based on contrastive learning. By using the rich knowledge of a pre-trained large model and supervised fine-tuning with only a small number of samples, nested entity recognition can be completed. At the same time, contrastive learning is introduced to enhance the representation ability of the large model, thereby solving the problem of over-extraction or under-extraction of nested entities by traditional methods, thereby overcoming the above-mentioned defects.

[0024] Figure 1It is a schematic diagram of the process of the nested entity recognition method provided by the present invention, such as Figure 1 As shown, the method includes: Step 110, determining a recognition instruction and a text to be recognized, wherein the recognition instruction is used to describe a nested entity recognition task.

[0025] Specifically, the text to be recognized refers to the specific text content that needs to be recognized by nested entities. These text contents may contain multiple levels of entity information. For example, in a text sentence, a company name contains a department name as its child entity, which constitutes a nested entity. Recognition instructions refer to instructions used to describe the specific requirements of the nested entity recognition task, which may include background information about the text to be recognized, the entity type to be recognized, and the nested relationship between entities. Recognition instructions provide guidance for the nested entity recognition model to perform tasks. Here, the nested entity recognition task refers to identifying and extracting entities with nested relationships from text.

[0026] It is understandable that the recognition instruction can be determined according to the specific application scenario and needs, so as to clearly tell the nested entity recognition model which tasks need to be performed. For example, the recognition instruction can be "Please extract the location entity type one and location entity type two in the following sentence. The entity types are nested with each other, and output the entities in the format <<>>". The text to be recognized is the text content provided by the user that needs to be recognized as an entity. It can be an electronic text, or it can be obtained by applying text recognition technology such as OCR (Optical Character Recognition) to an image of paper text for text recognition. The embodiment of the present invention does not specifically limit this. After the text to be recognized is determined, it can be input into the nested entity recognition model together with the recognition instruction for processing.

[0027] In the embodiment of the present invention, by introducing recognition instructions to describe the nested entity recognition task, the method can have high flexibility. By modifying the recognition instructions, it can adapt to different nested entity recognition requirements without a large amount of retraining of the model.

[0028] Step 120, inputting the recognition instruction and the text to be recognized into a nested entity recognition model to obtain a recognition result output by the nested entity recognition model; Among them, the nested entity recognition model is obtained by joint training based on sample data, positive sample pairs and negative sample pairs, with nested entity recognition tasks and contrastive learning tasks as training tasks. The sample data includes sample instructions and sample text and sample recognition results corresponding to the sample instructions. The positive sample pairs and the negative sample pairs are used to guide the nested entity recognition model to perform contrastive learning.

[0029] Specifically, after inputting the recognition instruction and the text to be recognized into the nested entity recognition model, the nested entity recognition model will extract the nested entities indicated by the recognition instruction from the text to be recognized according to the recognition instruction. Specifically, the nested entity recognition model will first parse the recognition instruction to understand the entity types and nested relationships to be recognized. Then, the model will read the text to be recognized and preprocess the text according to the requirements of the instruction, such as word segmentation, part-of-speech tagging, etc. Next, the model will extract features from the preprocessed text to obtain the vector representation of the text, and search for matching entities in the vector representation according to the entity types and nested relationships defined in the instruction. Since the model has enhanced its representation ability through contrastive learning, it can more accurately identify nested entities and reduce the situation of over-extraction or under-extraction. Finally, the model will output the recognized nested entities in a specified format.

[0030] Before performing step 120, the nested entity recognition model can be pre-trained. Specifically, the nested entity recognition model can be trained in the following way: First, collect data related to nested entities, including entity type tasks such as person names, place names, and organization names, and preprocess the collected data, such as deduplication and cleaning, to obtain high-quality data. Then, use the high-quality data to construct supervised fine-tuning data for nested entities to obtain sample data, which mainly consists of three parts, including sample instructions, sample texts, and sample recognition results. Next, determine positive sample pairs and negative sample pairs according to the sample data to guide the model to perform contrastive learning. Finally, according to specific task requirements, select a suitable initial model. For example, a pre-trained large model trained on a large-scale dataset can be selected. Take the nested entity recognition task and the contrastive learning task as training tasks, and jointly train the large model. During the training process, the loss of the recognition task can be used as the main loss function, and the loss of the contrastive learning task can be used as the auxiliary loss function. By optimizing these two loss functions, the performance and generalization ability of the model can be improved, thereby obtaining the nested entity recognition model.

[0031] Exemplarily, a specific example of the sample data is as follows: instruction: Please extract the location entity type one and location entity type two in the following sentence. The entity types are nested with each other, and output the entities in the format <<>>; input: I am a child who grew up near a scenic spot in City A; output: The location entity type one is <<City A>>, and the location entity type two is <>.

[0032] In the above example, instruction is the sample instruction, which is used to define the nested entity recognition task; input is the sample text, which refers to the input sentence to be recognized; output is the sample recognition result, that is, the actual nested entity recognition result (or answer).

[0033] It can be understood that, based on the above sample data, positive and negative sample pairs can be designed for comparative learning. Here, the positive sample pair refers to the sample pair consisting of the embedding (i.e., embedding vector) corresponding to the entity type to be extracted defined in the sample instruction and the embedding corresponding to the real entity in the sample text; the negative sample pair refers to the sample pair consisting of the embedding corresponding to the entity type to be extracted defined in the sample instruction and the embedding corresponding to the edge information in the sample text. For example, in the above sample data example, the embedding corresponding to "location entity type 1" in the instruction (i.e., the category to which "City A" belongs, specifically a vector representing the type at the embedding level) and the embedding corresponding to "City A" in the input, as well as the embedding corresponding to "location entity type 2" (i.e., the category to which "a certain scenic spot" belongs, also a vector at the embedding level) and the embedding corresponding to "a certain scenic spot" in the input constitute two pairs of positive sample pairs; while the embedding corresponding to "location entity type 1" in the instruction and the embedding corresponding to "is" in the input, as well as the embedding of "location entity type 2" and the embedding corresponding to "nearby" in the input can constitute two pairs of negative sample pairs. Positive sample pairs and negative sample pairs play a guiding role in the training process, helping the nested entity recognition model to conduct comparative learning.

[0034] Furthermore, existing positive sample pairs can be slightly modified (such as synonym replacement, sentence transformation, etc.) to generate more positive sample pairs. Similarly, existing negative sample pairs can be slightly modified to generate more negative sample pairs, which helps to increase the generalization ability of the model. In addition, in addition to determining positive and negative sample pairs based on sample data, external knowledge bases (such as knowledge graphs, etc.) can be used to find sentences or relationships related to entity types, and positive sample pairs can be constructed based on this. At the same time, sentences that do not contain specified entity relationships or attributes are screened out from the external knowledge base to construct negative sample pairs. The embodiments of the present invention do not specifically limit this.

[0035] The method provided by the embodiment of the present invention can obtain accurate recognition results by inputting recognition instructions and text to be recognized into a nested entity recognition model, and the recognition instructions are used to describe the nested entity recognition task, so that the present invention is highly flexible and can adapt to different nested entity recognition needs. Since the nested entity recognition model is based on sample data, positive sample pairs and negative sample pairs, and is jointly trained with the nested entity recognition task and the contrastive learning task as training tasks, the model can better learn the characteristics and laws of the nested entities. By introducing the contrastive learning task, the representation ability of the nested entity recognition model can be enhanced, thereby improving the accuracy of recognition, so that the model can more accurately recognize the nested entities in the text, and effectively reduce the situation of over-extraction or under-extraction.

[0036] Based on the above embodiments, Figure 2 is a schematic diagram of the process of training the nested entity recognition model provided by the present invention, such as Figure 2 As shown, the training steps of the nested entity recognition model include: Step 210: Obtain sample data, and determine positive sample pairs and negative sample pairs based on the sample data.

[0037] Specifically, sample data can be obtained using existing annotated data sets or through manual annotation. After the sample data is determined, positive sample pairs and negative sample pairs can be determined based on these data. These sample pairs are used to guide the model to learn how to distinguish relevant and irrelevant information. Among them, in the nested entity recognition task, the positive sample pair is a pair consisting of the embedding corresponding to the entity type defined by the sample instruction and the embedding corresponding to the real entity in the sample text. The negative sample pair is a pair consisting of the embedding corresponding to the entity type defined by the sample instruction and the embedding corresponding to the edge information in the sample text. Here, edge information refers to information in the sample text that is irrelevant to the entity type or related to the entity type, but does not actually conform to the nested relationship. By accurately obtaining sample data and determining positive and negative sample pairs, the model can be effectively guided to learn how to identify and distinguish nested entities, thereby improving the recognition ability of the model.

[0038] Step 220: Determine a contrast loss based on the positive sample pair and the negative sample pair.

[0039] Specifically, contrastive loss is a loss function used for metric learning. It aims to optimize model parameters so that similar samples are closer in feature space and dissimilar samples are farther apart. In the nested entity recognition task, contrastive loss is used to guide the model to learn how to better distinguish positive and negative sample pairs, improve the ability to perceive the boundaries of edge information, and thus effectively alleviate the problem of over- or under-sampling of nested entities.

[0040] After determining the positive sample pairs and the negative sample pairs, the embedding vector distance between the positive sample pairs and the embedding vector distance between the negative sample pairs can be calculated. The distance can be Euclidean distance, cosine distance, etc. In order to distinguish between positive sample pairs and negative sample pairs, the model usually sets a threshold. According to the embedding vector distance between positive samples, the embedding vector distance between negative sample pairs and the threshold, the contrast loss can be calculated.

[0041] Step 230, input the sample instructions and sample text in the sample data into the initial model, obtain the predicted recognition result output by the initial model, and determine the recognition loss based on the predicted recognition result and the sample recognition result in the sample data.

[0042] Specifically, the initial model refers to the initial state of the model before the training process begins. In the embodiment of the present invention, the initial model can be an untrained neural network model, or a pre-trained large model that has been trained on a large-scale data set, and the embodiment of the present invention does not specifically limit this.

[0043] After obtaining the sample data, the sample instruction and its corresponding sample text can be input into the initial model to obtain the predicted recognition result output by the model. Here, the predicted recognition result refers to the output result obtained by the initial model after receiving the sample instruction and sample text through forward propagation calculation. This result is calculated by the model based on the current parameters and input data, and may not be accurate. Therefore, the model will calculate the difference between the predicted recognition result and the sample recognition result, and set a loss function based on the difference. The loss function is usually a metric that can reflect the difference between the predicted result and the true result, such as cross entropy loss, mean square error loss, etc. Finally, the model will calculate the loss value based on the set loss function and difference. This loss value is the recognition loss. Here, the recognition loss refers to the loss value calculated by the model during the training process based on the difference between the predicted recognition result and the sample recognition result. This loss value is used to guide the update of the model parameters so that the model can more accurately recognize nested entities in subsequent training.

[0044] Step 240: Based on the recognition loss and the contrast loss, perform parameter iteration on the initial model to obtain the nested entity recognition model.

[0045] Specifically, in order to enhance the representation ability of the model, a contrastive learning task is introduced in the model training process, and the contrastive loss is used as an auxiliary loss function to jointly train the model. Therefore, after the contrastive loss and recognition loss are calculated, the contrastive loss and recognition loss can be weighted and summed to form a total loss function. Here, the purpose of weighting is to assign different importance to different types of losses according to task requirements and data characteristics.

[0046] Next, the model uses the back-propagation algorithm to calculate the gradient of the total loss function with respect to the model parameters. Here, the gradient is a vector that indicates the direction in which the loss function value decreases fastest under the current parameter values. By calculating the gradient, the model can understand how to adjust the parameters to reduce the loss. With the gradient information, the model can use optimization algorithms (such as stochastic gradient descent SGD, Adam, etc.) to update the parameters.

[0047] The above steps will be repeated until a certain stopping condition is met. The stopping condition can be reaching a preset number of iterations, the loss function value drops below a certain threshold, or the performance of the model on the validation set no longer improves. During the iteration process, the model will gradually learn how to extract useful features from sample data and accurately identify nested entities, thereby obtaining a nested entity recognition model.

[0048] Based on the above embodiment, in step 210, obtaining sample data includes: Step 211, collecting nested entity related data.

[0049] Specifically, nested entity related data refers to text data containing nested entities (i.e., one entity contains another entity inside), which is essential for training models to identify nested entities. In order to collect nested entity related data, it can be achieved through existing public data sets, or by writing a web crawler program to crawl text data containing nested entities from web pages, news, social media and other platforms. The embodiments of the present invention do not specifically limit this.

[0050] Step 212, fuzzy deduplication is performed on the nested entity related data, and the deduplication data is cleaned to obtain initial nested entity data.

[0051] Specifically, fuzzy deduplication refers to the deduplication of possible duplicate data during the data collection process. Due to the diversity of data sources, there may be similar or identical text data. Fuzzy deduplication compares the similarity of text data and regards data with higher similarity as duplicate data and performs deduplication, which helps reduce data redundancy and improve data quality. Data cleaning refers to the preprocessing of collected data to eliminate errors, inconsistencies, and redundancies in the data.

[0052] After fuzzy deduplication and cleaning of the collected nested entity related data, high-quality initial nested entity data can be obtained. Here, the initial nested entity data is the basic data that meets the requirements of the nested entity recognition task after data collection, fuzzy deduplication and cleaning.

[0053] Furthermore, step 212 specifically includes: Step 2121, based on a text similarity measurement algorithm and / or a clustering algorithm, deduplicate the nested entity related data.

[0054] Here, a text similarity algorithm is a method for calculating the similarity between two or more texts. For example, the text similarity algorithm may include edit distance, pre-similarity, etc. When deduplicating nested entity related data, a selected text similarity algorithm (such as edit distance, cosine similarity, etc.) may be used to calculate the similarity between each pair of data items, and determine whether the data items are repeated based on a set similarity threshold. If the similarity between two data items exceeds the threshold, they are considered as duplicate data and deduplicated.

[0055] Clustering algorithms are a method of dividing a data set into several groups or clusters so that the data points within the same cluster are similar to each other, while the data points between different clusters are quite different. For example, clustering algorithms can include K-means clustering, hierarchical clustering, etc. Clustering algorithms can also be used to remove duplicates of nested entity-related data. Through clustering algorithms, similar data items can be clustered into the same cluster, and then a representative item is selected in each cluster as the only data item to achieve deduplication.

[0056] Step 2122, based on preset filtering rules, clean the deduplicated data, wherein the preset filtering rules include at least one of language-based filtering rules, metric-based filtering rules, and keyword-based filtering rules.

[0057] Here, when cleaning the deduplicated data, we mainly filter dirty data, data with format problems, and data with unqualified content. Specifically, we can design a set of filtering rules to eliminate low-quality data, thereby improving data quality. These filtering rules can include language-based filtering rules, metric-based filtering rules, keyword-based filtering rules, etc. Through various filtering rules, we can filter out poor-quality data.

[0058] Among them, language-based filtering rules refer to filtering based on the language characteristics of the text. For example, you can set rules to filter out texts that contain specific grammatical errors or spelling errors, or filter out texts that do not conform to a specific language style (such as texts that are too colloquial or too formal). Metric-based filtering rules refer to using a certain metric to evaluate the quality or relevance of the text and filtering based on a set threshold. For example, you can use metrics such as text length, sentence complexity, and keyword density to filter out low-quality text. Keyword-based filtering rules refer to filtering based on whether the text contains specific keywords or phrases. For example, you can set rules to filter out texts that contain irrelevant keywords or negative keywords, or only keep texts that contain specific topic keywords.

[0059] Step 213: construct sample data based on the initial nested entity data.

[0060] Specifically, after obtaining high-quality initial nested entity data in step S212, sample data can be constructed based on these data. When constructing sample data, the design of sample instructions is crucial. The instructions need to clearly define the goals and requirements of the nested entity recognition task. For example, the instructions can specify the entity types to be extracted (such as names of people, places, names of institutions, etc.), and explain that there is a nested relationship between these entity types. In addition, the instructions can also specify the format of the output, such as using specific tags (such as <<>>) to identify the extracted entities. Next, it is necessary to select appropriate sentences from the high-quality data as input according to the requirements of the instructions. These sentences should contain the entity types specified in the instructions, and there is a nested relationship between these entity types. For each input sentence, it is necessary to construct the corresponding output according to the requirements of the instructions and the actual situation. The output should accurately identify the nested entities in the input sentence and follow the format specified in the instructions. It should be understood that the construction of sample data can be achieved through an existing generative large model, or it can be obtained through manual construction, and the embodiment of the present invention does not specifically limit this.

[0061] In addition, in order to enhance the generalization ability of the model, it is necessary to construct diverse nested entity sample data. This can be achieved by selecting different sentences, adjusting the combination of entity types, changing the output format, etc. Diversified data can help the model better learn the characteristics and rules of nested entities, thereby improving recognition performance.

[0062] Based on the above embodiment, in step 210, determining positive sample pairs and negative sample pairs based on the sample data includes: Step 214: extract features of each entity type in the sample instruction to obtain an embedding vector of each entity type.

[0063] Specifically, the entity type in the sample instruction refers to the entity type to be extracted or recognized that is explicitly specified in the instruction. These entity types are usually pre-defined according to task requirements and are used to guide the model to recognize and extract corresponding entities in specific texts.

[0064] Feature extraction for each entity type in the sample instructions can be achieved through natural language processing technology. For example, you can use pre-trained word embedding models (such as Word2Vec, BERT, etc.) to convert the name of the entity type into a high-dimensional vector representation. These vectors can capture the semantic relationship between entity types. You can also directly integrate the embedding layer in the initial model to extract features for each entity type through the embedding layer. In addition, you can also extract features based on the position, attributes and other information of the entity type in the knowledge graph to enrich the representation of the entity type. The embedding vector of each entity type refers to a numerical representation of the entity type. This representation can encode the semantic information of the entity type into a vector in a high-dimensional space. The vector can capture the semantic similarities and differences between entity types, thereby playing a key role in contrastive learning.

[0065] Step 215 , performing feature extraction on each entity and each edge information in the sample text to obtain an embedding vector of each entity and an embedding vector of each edge information.

[0066] Specifically, the entities in the sample text refer to the entity instances that appear in the text and are required to be extracted by the instruction. For example, in the instruction "Please extract the location entity type 1 and location entity type 2 in the following sentence", if the sample text is "I am a child who grew up near a certain scenic spot in City A", then "City A" and "a certain scenic spot" are the entities in the sample text. The marginal information refers to the additional information related to these entities but not required to be extracted by the instruction, such as other non-target entities or attributes in the text.

[0067] The process of extracting features from each entity and edge information in the sample text is similar to that of extracting features from entity types, but it focuses more on extracting specific entity instances and related information from the text. For example, you can use a pre-trained word embedding model or integrate an embedding layer in the initial model to extract entity and edge information from the text and convert it into a high-dimensional vector representation.

[0068] It can be understood that the embedding vectors of each entity and each edge information refer to the high-dimensional vector representation obtained by converting each entity and each edge information in the sample text through feature extraction technology. In contrastive learning, these embedding vectors are compared with the embedding vector of the entity type. If the semantic similarity between the two is high (that is, the distance between the vectors is small), it means that the entity matches the entity type defined in the instruction, thus forming a positive sample pair. On the contrary, if the semantic similarity between the two is low (that is, the distance between the vectors is large), it may constitute a negative sample pair.

[0069] Step 216, determining a positive sample pair based on an embedding vector of any entity type and an embedding vector of an entity corresponding to the any entity type, wherein the entity corresponding to the any entity type is determined based on the sample recognition result; Step 217: determine a negative sample pair based on the embedding vector of any entity type and the embedding vector of any edge information.

[0070] Specifically, after extracting the embedding vectors (i.e., embeddings) of each entity type, each entity, and each edge information, the entity corresponding to each entity type can be determined based on the sample recognition results, so that the embedding of the entity type and the embedding of the corresponding entity can be combined into a positive sample pair, and the embedding of the entity type and the embedding of the edge information can be combined into a negative sample pair.

[0071] Based on any of the above embodiments, step 220 specifically includes: Step 221, determining a first distance based on the embedding vector of the entity type and the embedding vector of the entity in the positive sample pair; Step 222, determining a second distance based on the embedding vector of the entity type and the embedding vector of the edge information in the negative sample pair; Step 223: Determine the contrast loss based on the first distance and the second distance.

[0072] Specifically, after determining the positive sample pairs and the negative sample pairs, the distance between the positive sample pairs can be calculated based on the embedding of the entity type and the embedding of the corresponding entity in the positive sample pair using a certain distance measurement method (such as cosine similarity, Euclidean distance, etc.), that is, the first distance is obtained. At the same time, the distance between the negative sample pairs is calculated based on the embedding of the entity type and the embedding of the edge information in the negative sample pair, that is, the second distance is obtained. Subsequently, based on the first distance and the second distance, the contrast loss can be calculated through the contrast loss function.

[0073] Figure 3 is a schematic diagram of the positive and negative sample pairs provided by the present invention, such as Figure 3As shown in the figure, the embedding corresponding to "location entity type 1" in the instruction and the embedding corresponding to "city A" in the input, as well as the embedding corresponding to "location entity type 2" and the embedding corresponding to "a certain scenic spot" in the input, respectively constitute two pairs of positive sample pairs; while the embedding corresponding to "location entity type 1" in the instruction and the embedding corresponding to "is" in the input, as well as the embedding of "location entity type 2" and the embedding corresponding to "nearby" in the input constitute two pairs of negative sample pairs. By calculating the distance between the positive sample pairs and the distance between the negative sample pairs, the contrast loss can be determined. The goal of the contrast loss is to narrow the distance between the entity type embedding and the corresponding entity embedding, and to push the distance between the entity type embedding and the corresponding edge information embedding.

[0074] Based on any of the above embodiments, the initial model is a pre-trained large model, and the pre-trained large model refers to a model pre-trained based on a training data set, and the pre-trained large model adopts a Decoder-Only architecture.

[0075] Specifically, a pre-trained large model refers to a model trained on a large-scale data set. These models are usually composed of billions of parameters and can perform well on various tasks. At present, the pre-trained large models can be roughly divided into three network architectures: Encoder-Only, Decoder-Only and Encoder-Decoder. The pre-trained large model of the embodiment of the present invention adopts the Decoder-Only architecture. Here, the main feature of the Decoder-Only architecture is that it only contains the decoder part and there is no encoder. This architecture is particularly suitable for generation tasks such as text generation, dialogue systems, etc. The decoder is usually composed of multiple layers of Transformer decoder blocks stacked together, each block containing a self-attention mechanism and a feedforward neural network.

[0076] It is understandable that in the joint training process of the pre-trained large model, the contrastive loss can be used as an auxiliary target for model training, and together with the main task loss of the model (i.e., recognition loss), it constitutes the total loss. Since the contrastive loss is calculated based on the distance between embeddings, and the embeddings are generated by the embedding layer in the model or the pre-trained embedding model, the gradient of the contrastive loss will indirectly affect the parameters of the embedding layer (if the embedding layer is trainable) and other parameters in the model related to the generation of the embeddings. When the model parameters are updated, the embeddings extracted in the next iteration will also change, which in turn affects the calculation of the contrastive loss. Therefore, the contrastive loss is dynamically involved in the model training process, and it constantly adjusts the parameters according to the current model state to optimize the representation ability of the model.

[0077] Based on any of the above embodiments, Figure 4 is a flow chart of a nested entity recognition method based on contrastive learning enhanced large model provided by the present invention, such as Figure 4 As shown, the method includes: Step S1, data collection and cleaning First, collect and organize the data related to nested entities, including tasks such as names of people, places, and institutions. Secondly, perform fuzzy deduplication on the collected nested entity data and stock data to reduce data redundancy. Common deduplication methods include deduplication using text similarity measurement methods (such as edit distance, cosine similarity, etc.) and deduplication using clustering algorithms. Nested entity data requires manual sampling inspection to ensure that the logic is correct, thereby improving data quality. Finally, data cleaning is performed on the collected data, mainly filtering dirty data, format problems, and data with unqualified content. The conventional approach is to design a set of filtering rules to eliminate low-quality data, thereby improving data quality. Commonly used filtering rules include language-based filtering rules, metric-based filtering rules, and keyword-based filtering rules. Filter poor quality data through various filtering rules.

[0078] Step S2: Nested entity supervised fine-tuning data construction Based on the high-quality data obtained in step S1, construct supervised fine-tuning data for nested entities, which mainly consists of three parts: instruction, input, and output. The specific example is as follows: instruction: Please extract the location entity type 1 and location entity type 2 in the following sentence, where the entity types are nested relationships, and output the entities in the format <<>>; input: I am a child who grew up near a scenic spot in City A; output: The location entity type 1 is <<City A>>, and the location entity type 2 is <>. Among them, the instruction is used to define the nested entity recognition task, the input is the sentence to be recognized, and the output is the answer.

[0079] Step S3, design positive and negative sample pairs for contrastive learning Based on step S2, generate supervised fine-tuning data for diverse nested entities. Then, positive and negative sample pairs can be designed for contrastive learning. Among them, the embeddings corresponding to the entity type 1 and entity type 2 to be extracted defined by the instruction and the embeddings corresponding to the real entities in the input are positive sample pairs, and the embeddings corresponding to the entity type 1 and entity type 2 to be extracted defined by the instruction and the embeddings corresponding to the marginal information in the input are negative sample pairs. Narrow the distance between the entity type embeddings and the embeddings of the corresponding real entities, and widen the distance between the entity type embeddings and the embeddings of the corresponding marginal information.

[0080] Step S4, model training and inference Pre-trained large models, simply put, are models trained on large-scale datasets. These models usually consist of billions of parameters and can perform well on various tasks. Currently, pre-trained large models can be roughly divided into three network architectures: Encoder-Only, Decoder-Only, and Encoder-Decoder. The pre-trained model in the embodiment of the present invention adopts the Decoder-Only architecture. The experiments are verified based on pre-trained large models of 13B and 65B. First, prepare supervised fine-tuning data for nested entities according to step S2. After the data preparation is completed, contrastive learning is carried out according to the construction of positive and negative sample pairs in S3 to enhance the representation ability of the large model. Based on steps S2 and S3, only a small number of samples are used (because the pre-trained large model has knowledge ability), and the contrastive loss is used as an auxiliary function to jointly train with the large model. After the model training is completed, model inference is performed. The large model enhanced by contrastive learning improves the boundary perception ability for marginal information, can effectively alleviate the problem of over-extracting or under-extracting nested entities, and improves the nested entity recognition ability.

[0081] Based on any of the above embodiments, Figure 5 is a schematic diagram of the structure of the nested entity recognition device provided by the present invention, such as Figure 5 As shown, the device comprises: A determination unit 510, configured to determine a recognition instruction and a text to be recognized, wherein the recognition instruction is used to describe a nested entity recognition task; The recognition unit 520 is used to input the recognition instruction and the text to be recognized into a nested entity recognition model to obtain a recognition result output by the nested entity recognition model; Among them, the nested entity recognition model is obtained by joint training based on sample data, positive sample pairs and negative sample pairs, with nested entity recognition tasks and contrastive learning tasks as training tasks. The sample data includes sample instructions and sample text and sample recognition results corresponding to the sample instructions. The positive sample pairs and the negative sample pairs are used to guide the nested entity recognition model to perform contrastive learning.

[0082] The device provided by the embodiment of the present invention can obtain accurate recognition results by inputting recognition instructions and text to be recognized into a nested entity recognition model, and the recognition instructions are used to describe the nested entity recognition task, so that the present invention is highly flexible and can adapt to different nested entity recognition needs. Since the nested entity recognition model is based on sample data, positive sample pairs and negative sample pairs, and is obtained by joint training with the nested entity recognition task and the contrastive learning task as training tasks, the model can better learn the characteristics and laws of the nested entities. By introducing the contrastive learning task, the representation ability of the nested entity recognition model can be enhanced, thereby improving the accuracy of recognition, so that the model can more accurately recognize the nested entities in the text, and effectively reduce the situation of over-extraction or under-extraction.

[0083] Based on any of the above embodiments, the device further includes a model training unit, which specifically includes: An acquisition unit, configured to acquire sample data and determine positive sample pairs and negative sample pairs based on the sample data; a contrast loss determining unit, configured to determine a contrast loss based on the positive sample pair and the negative sample pair; a recognition loss determining unit, configured to input the sample instructions and sample text in the sample data into an initial model, obtain a predicted recognition result output by the initial model, and determine the recognition loss based on the predicted recognition result and the sample recognition result in the sample data; A parameter iteration unit is used to perform parameter iteration on the initial model based on the recognition loss and the contrast loss to obtain the nested entity recognition model.

[0084] Based on any of the above embodiments, the acquisition unit includes a sample pair determination subunit, and the sample pair determination subunit is used to: Extracting features of each entity type in the sample instruction to obtain an embedding vector of each entity type; Performing feature extraction on each entity and each edge information in the sample text to obtain an embedding vector of each entity and an embedding vector of each edge information; Determine a positive sample pair based on an embedding vector of any entity type and an embedding vector of an entity corresponding to the any entity type, wherein the entity corresponding to the any entity type is determined based on the sample recognition result; A negative sample pair is determined based on the embedding vector of any entity type and the embedding vector of any edge information.

[0085] Based on any of the above embodiments, the contrast loss determination unit is specifically used for: Determining a first distance based on the embedding vector of the entity type and the embedding vector of the entity in the positive sample pair; Determining a second distance based on the embedding vector of the entity type and the embedding vector of the edge information in the negative sample pair; Based on the first distance and the second distance, a contrast loss is determined.

[0086] Based on any of the above embodiments, the acquisition unit includes a data acquisition subunit, and the data acquisition subunit specifically includes: A data collection subunit, used to collect data related to nested entities; A data processing subunit, configured to perform fuzzy deduplication on the nested entity related data and clean the deduplicated data to obtain initial nested entity data; The data construction subunit is used to construct sample data based on the initial nested entity data.

[0087] Based on any of the above embodiments, the data processing subunit is specifically used for: Deduplication of the nested entity related data is performed based on a text similarity measurement algorithm and / or a clustering algorithm; The deduplicated data is cleaned based on preset filtering rules, wherein the preset filtering rules include at least one of language-based filtering rules, metric-based filtering rules, and keyword-based filtering rules.

[0088] Based on any of the above embodiments, the initial model is a pre-trained large model, and the pre-trained large model refers to a model pre-trained based on a training data set, and the pre-trained large model adopts a Decoder-Only architecture.

[0089] Figure 6An example of a physical structure diagram of an electronic device is shown in FIG. Figure 6 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the nested entity recognition method, which includes: determining a recognition instruction and a text to be recognized, wherein the recognition instruction is used to describe a nested entity recognition task; inputting the recognition instruction and the text to be recognized into a nested entity recognition model to obtain a recognition result output by the nested entity recognition model; wherein the nested entity recognition model is obtained by joint training based on sample data, positive sample pairs and negative sample pairs, with the nested entity recognition task and the contrastive learning task as training tasks, wherein the sample data includes a sample instruction and a sample text and a sample recognition result corresponding to the sample instruction, and the positive sample pair and the negative sample pair are used to guide the nested entity recognition model to perform contrastive learning.

[0090] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0091] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the nested entity recognition method provided by the above-mentioned methods, and the method includes: determining a recognition instruction and a text to be recognized, wherein the recognition instruction is used to describe a nested entity recognition task; inputting the recognition instruction and the text to be recognized into a nested entity recognition model to obtain a recognition result output by the nested entity recognition model; wherein the nested entity recognition model is based on sample data, positive sample pairs and negative sample pairs, and is obtained by joint training with nested entity recognition tasks and comparative learning tasks as training tasks, and the sample data includes sample instructions and sample text and sample recognition results corresponding to the sample instructions, and the positive sample pairs and the negative sample pairs are used to guide the nested entity recognition model to perform comparative learning.

[0092] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the nested entity recognition method provided by the above-mentioned methods, the method comprising: determining a recognition instruction and a text to be recognized, the recognition instruction being used to describe a nested entity recognition task; inputting the recognition instruction and the text to be recognized into a nested entity recognition model to obtain a recognition result output by the nested entity recognition model; wherein the nested entity recognition model is obtained by joint training based on sample data, positive sample pairs and negative sample pairs, with nested entity recognition tasks and comparative learning tasks as training tasks, the sample data comprising sample instructions and sample text and sample recognition results corresponding to the sample instructions, the positive sample pairs and the negative sample pairs being used to guide the nested entity recognition model to perform comparative learning.

[0093] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0094] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A nested entity recognition method, characterized in that: include: Determine a recognition instruction and a text to be recognized, wherein the recognition instruction is used to describe a nested entity recognition task; Inputting the recognition instruction and the text to be recognized into a nested entity recognition model to obtain a recognition result output by the nested entity recognition model; Among them, the nested entity recognition model is obtained by joint training based on sample data, positive sample pairs and negative sample pairs, with nested entity recognition tasks and contrastive learning tasks as training tasks. The sample data includes sample instructions and sample text and sample recognition results corresponding to the sample instructions. The positive sample pairs and the negative sample pairs are used to guide the nested entity recognition model to perform contrastive learning.

2. The nested entity recognition method according to claim 1, characterized in that: The training steps of the nested entity recognition model include: Acquire sample data, and determine positive sample pairs and negative sample pairs based on the sample data; Determining a contrast loss based on the positive sample pair and the negative sample pair; Inputting sample instructions and sample text in the sample data into an initial model to obtain a predicted recognition result output by the initial model, and determining a recognition loss based on the predicted recognition result and a sample recognition result in the sample data; Based on the recognition loss and the contrast loss, parameters of the initial model are iterated to obtain the nested entity recognition model.

3. The nested entity recognition method according to claim 2, characterized in that: The determining of positive sample pairs and negative sample pairs based on the sample data includes: Extracting features of each entity type in the sample instruction to obtain an embedding vector of each entity type; Performing feature extraction on each entity and each edge information in the sample text to obtain an embedding vector of each entity and an embedding vector of each edge information; Determine a positive sample pair based on an embedding vector of any entity type and an embedding vector of an entity corresponding to the any entity type, wherein the entity corresponding to the any entity type is determined based on the sample recognition result; A negative sample pair is determined based on the embedding vector of any entity type and the embedding vector of any edge information.

4. The nested entity recognition method according to claim 3, characterized in that: The determining the contrast loss based on the positive sample pair and the negative sample pair includes: Determining a first distance based on the embedding vector of the entity type and the embedding vector of the entity in the positive sample pair; Determining a second distance based on the embedding vector of the entity type and the embedding vector of the edge information in the negative sample pair; Based on the first distance and the second distance, a contrast loss is determined.

5. The nested entity recognition method according to claim 2, characterized in that: The obtaining of sample data comprises: Collect data about nested entities; Performing fuzzy deduplication on the nested entity related data, and cleaning the deduplicated data to obtain initial nested entity data; Based on the initial nested entity data, sample data is constructed.

6. The nested entity recognition method according to claim 5, characterized in that: The fuzzy deduplication of the nested entity related data and cleaning the deduplication data include: Deduplication of the nested entity related data is performed based on a text similarity measurement algorithm and / or a clustering algorithm; The deduplicated data is cleaned based on preset filtering rules, wherein the preset filtering rules include at least one of language-based filtering rules, metric-based filtering rules, and keyword-based filtering rules.

7. The nested entity recognition method according to any one of claims 2 to 6, characterized in that: The initial model is a pre-trained large model, which refers to a model pre-trained based on a training data set, and the pre-trained large model adopts a Decoder-Only architecture.

8. A nested entity recognition device, characterized in that: include: A determination unit, used to determine a recognition instruction and a text to be recognized, wherein the recognition instruction is used to describe a nested entity recognition task; A recognition unit, used for inputting the recognition instruction and the text to be recognized into a nested entity recognition model to obtain a recognition result output by the nested entity recognition model; Among them, the nested entity recognition model is obtained by joint training based on sample data, positive sample pairs and negative sample pairs, with nested entity recognition tasks and contrastive learning tasks as training tasks. The sample data includes sample instructions and sample text and sample recognition results corresponding to the sample instructions. The positive sample pairs and the negative sample pairs are used to guide the nested entity recognition model to perform contrastive learning.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the nested entity recognition method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the nested entity recognition method according to any one of claims 1 to 7 is implemented.

Images