Named entity recognition method combining large model and dynamic sample

Abstract

The invention discloses a named entity recognition method in combination with a large model and dynamic examples. The method comprises the following steps: S1, constructing a dynamic example library; s2, constructing a basic cue word: defining a business entity annotation background, a rule and an output specification, and constructing a structured cue word in an XML (Extensible Markup Language) style; s3, recalling a dynamic sample; s4, training corpus construction, namely entity recognition corpus construction and recognition reasoning corpus construction; s5, selecting a large model; s6, identifying the large model, integrating the instance d, recalling the sample set R and the basic cue word, inputting the integrated instance d, recalling the sample set R and the basic cue word into the trained model, and outputting a final entity identification result; and S7, API service packaging: recalling the dynamic sample, and packaging the prompt word and the selected large model into AP I service for convenient calling. According to the method, the problem of bad case identification can be rapidly repaired, powerful support is provided for rapid landing of new field and new demand expansion, and the manpower and time cost of data annotation is effectively reduced.

Description

Technical Field

[0001] The invention belongs to the technical field of data processing, and in particular is a named entity recognition method combining a large model and dynamic examples. Background Art

[0002] In the field of natural language processing, named entity recognition (NER) is a crucial task, and its core goal is to accurately identify entities with specific meanings in texts, such as names of people and places. Currently, the NER task is mainly based on model fine-tuning training, such as deep model BERT, large model LLAMA series, etc., which is achieved by fine-tuning training on large-scale annotated datasets. This type of method performs well in specific fields and datasets and can provide high recognition accuracy.

[0003] Defects of the prior art:

[0004] 1. Data dependence: Model recognition performance depends largely on large-scale labeled data, which requires not only high-quality annotations but also wide coverage of scenarios to ensure that the model has good recognition capabilities. However, collecting and labeling this data takes a lot of time and manpower;

[0005] 2. Low generalization. When the model processes new fields or new types of entities, it needs to collect data again and conduct training, which increases the difficulty of development and maintenance and limits the model's ability to adapt to new fields and new needs.

[0006] 3. Difficult to repair. When these models encounter bad examples during actual application, they often need to collect data again and conduct training, which makes it impossible to quickly repair and update them. Summary of the invention

[0007] In view of the shortcomings of the existing technology, the present invention provides a named entity recognition solution that combines a large model and dynamic samples, which can quickly fix the problem of bad example recognition, provide strong support for the rapid implementation of new fields and new demand expansion, and effectively reduce the manpower and time costs of data annotation.

[0008] The technical solution of the present invention is as follows: a method for named entity recognition combining a large model and dynamic examples, the steps are as follows:

[0009] S1. Build a dynamic sample library;

[0010] S2. Construct basic prompt words: clarify the business entity annotation background, rules, and output specifications, and construct XML-style structured prompt words;

[0011] S3, dynamic sample recall;

[0012] S4, training corpus construction, including entity recognition corpus construction and recognition reasoning corpus construction;

[0013] S5, large model selection;

[0014] S6, large model recognition, integrating instance d, recall sample set R, and basic prompt words into the trained model, and outputting the final recognition entity result;

[0015] S7, API service encapsulation, encapsulates dynamic sample recall, prompt words, and selected large models into API services for easy calling.

[0016] Preferably, the specific steps of constructing the dynamic sample library in step S1 are as follows:

[0017] 1) Continuously collect common expression data and problem data for annotation to build a representative sample data set S = {s1, s2, ..., s n}; where data s i Use JSON format to store various information;

[0018] 2) Store the entire sample dataset S on the zilliz cloud platform.

[0019] Preferably, the data s in step S1 i The key-value pairs of each information stored in JSON format are as follows:

[0020] The following points:

[0021] a. Embedding vector, which converts the search content into a 1024-dimensional vector representation through the text2vec-large-chinese model;

[0022] b. text data, complete data with context;

[0023] c. label tag, output result in JSON format of the labeled entity;

[0024] d. COT thinking chain, reasoning chain of entity recognition.

[0025] Preferably, the specific steps of the step S3 dynamic sample recall are as follows:

[0026] 1) All the content to be retrieved in instance d L = {l1,l2,...,l m}Then the text2vec-large-chinese model is used to convert the vector into a 1024-dimensional vector representation to obtain the matrix V = {v1, v2, ..., v m};

[0027] 2) Search based on the zilliz cloud platform and issue recall v i(i=1,2,……m) TOP1 similar examples r in the sample data set S whose cosine distance similarity is between 0.7-1.1 (the similarity of completely identical content will be higher than 1, so the upper limit is temporarily set to 1.1) i (i=1,2,……m);

[0028] 3) All similar examples c i After removing duplicates, the recall sample set R = {r1, r2, ..., r k}(k<=m).

[0029] Preferably, the specific steps of constructing the entity recognition corpus in step S4 are as follows:

[0030] 1) Collect a certain amount of data sets with scene coverage N = {n1, n2, ..., n s};

[0031] 2) For data n i (i=1,2,……s) construct fusion prompt word p i (i=1,2,……s), including the following three content data: data n i 、Data i Obtain the recalled sample set R through dynamic sample recall i , basic prompt words;

[0032] 3) Change p i (i=1,2,……s) are successively recognized by the doubao-pro-4k large model to obtain the recognition data u i (i=1,2,……s);

[0033] 4) Identify the data u i The doubao-pro-4k large model is used to test and judge again, and finally the correct identification data set N1 and the identification data set N2 to be confirmed are obtained;

[0034] 5) Manually verify the unidentified dataset N2 and integrate the correctly identified dataset N1 to obtain the final entity training corpus T1 = {t 11 ,t 12 ,……,t 1s}.

[0035] Preferably, the specific steps of identifying the construction of the inference corpus in step S4 are as follows:

[0036] 1) Select data with different difficulty levels from the correct recognition data set N1 and the unrecognized recognition data set N2 to form a data set W = {w1, w2, ..., w l};

[0037] 2) The data wi (i=1,2,……l) The inference data o is obtained by constructing the inference thinking chain through the doubao-pro-4k large model i (i=1,2,……l);

[0038] 3) The inference data o i The doubao-pro-4k large model is used to test and judge again, and finally the correct inference data set O1 and the inference data set O2 to be confirmed are obtained;

[0039] 4) Manually verify the inference dataset O2 to be confirmed, and integrate the inference correct dataset O1 to obtain the final inference training corpus T2 = {t 21 ,t 22 ,……,t 2s}.

[0040] Preferably, the specific steps of selecting the large model in step S5 are as follows:

[0041] 1) Comparison base selection, select llama3.1-8b, qwen2.5-7b, llama3-8b, llama2-7b and other base models for comparison training;

[0042] 2) Model training: Use entity training corpus T1 and reasoning training corpus T2 to fine-tune the base model respectively;

[0043] 3) Model evaluation: Use the validation set to evaluate each comparison model, calculate indicators such as accuracy, recall, and F1 score, and select the best trained model for final adoption.

[0044] The present invention can quickly fix the problem of identifying bad examples, provide strong support for the rapid implementation of new fields and new demand expansion, and effectively reduce the manpower and time costs of data annotation. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0046] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the invention, so the present invention is not limited by the specific implementation disclosed below.

[0047] Example 1

[0048] A named entity recognition method combining large models and dynamic examples, such as Figure 1 As shown, the following steps are included:

[0049] S1. Build a dynamic sample library:

[0050] 1. Continuously collect common expression data and problem data for annotation to build a representative sample data set S = {s1, s2, ..., s n}. Among them, data s i The JSON format is used to store various information, and its key-value pairs involve the following aspects of knowledge:

[0051] (1) Embedding vector: the search content is converted into a 1024-dimensional vector representation through the text2vec-large-chinese model;

[0052] (2) text data, complete data with context;

[0053] (3) label tag, output results in JSON format of the labeled entity;

[0054] (4)cot thinking chain, reasoning chain of entity recognition.

[0055] 2. Store the entire sample dataset S on the zilliz cloud platform.

[0056] S2. Construct basic prompt words:

[0057] Clarify the business entity annotation background, rules, and output specifications, and construct XML-style structured prompt words.

[0058] S3, Dynamic sample recall:

[0059] 1. All the content to be retrieved in instance d L = {l1,l2,...,l m}Then the text2vec-large-chinese model is used to convert the vector into a 1024-dimensional vector representation to obtain the matrix V = {v1, v2, ..., v m};

[0060] 2. Search based on the zilliz cloud platform and issue a recall v i (i=1,2,……m) In the sample data set S, the cosine distance similarity is between 0.7 and 1.1 (the similarity of completely identical content will be higher than 1, so the upper limit is temporarily set to

[0061] 1.1) The top 1 similar examples between i (i=1,2,……m);

[0062] 3. All similar examples ci After removing duplicates, the recall sample set R = {r1, r2, ..., r k}(k<=m).

[0063] S4. Construction of training corpus:

[0064] 3. Entity recognition corpus construction:

[0065] (1) Collect a certain amount of data sets with scene coverage N = {n1, n2, ..., n s};

[0066] (2) For data n i (i=1,2,……s) construct fusion prompt word p i (i=1,2,……s), including the following three items of content data:

[0067] ①Data n i ;

[0068] ②Data n i Obtain the recalled sample set R through dynamic sample recall i ;

[0069] ③Basic prompt words;

[0070] (3) i (i=1,2,……s) are successively recognized by the doubao-pro-4k large model to obtain the recognition data u i (i=1,2,……s);

[0071] (4) Identify the data u i The doubao-pro-4k large model is used to test and judge again, and finally the correct identification data set N1 and the identification data set N2 to be confirmed are obtained;

[0072] (5) Manually verify the unidentified dataset N2 and integrate the correctly identified dataset N1 to obtain the final entity training corpus T1 = {t 11 ,t 12 ,……,t 1s};

[0073] 4. Identification and Reasoning Corpus Construction

[0074] (1) Select data with different difficulty levels from the correct recognition data set N1 and the unrecognized recognition data set N2 to form a data set W = {w1, w2, ..., w l};

[0075] (2) Data w i(i=1,2,……l) The inference data o is obtained by constructing the inference thinking chain through the doubao-pro-4k large model i (i=1,2,……l);

[0076] (3) The inference data o i The doubao-pro-4k large model is used to test and judge again, and finally the correct inference data set O1 and the inference data set O2 to be confirmed are obtained;

[0077] (4) Manually verify the inference unconfirmed data set O2 and integrate the inference correct data set O1 to obtain the final inference training corpus T2 = {t 21 ,t 22 ,……,t 2s};

[0078] S5, Large Model Selection:

[0079] 5. Comparison base selection, select llama3.1-8b, qwen2.5-7b, llama3-8b, llama2-7b and other base models for comparison training;

[0080] 6. Model training: Use the entity training corpus T1 and the reasoning training corpus T2 to fine-tune the base model respectively.

[0081] 7. Model evaluation: Use the validation set to evaluate each comparison model, calculate indicators such as accuracy, recall rate, and F1 score, and select the best trained model for final adoption.

[0082] S6, large model recognition, integrates instance d, recalled sample set R, and basic prompt words into the trained model and outputs the final entity recognition result.

[0083] S7, API service encapsulation, encapsulates dynamic sample recall, prompt words, and selected large models into API services for easy calling.

[0084] Example 2

[0085] The dynamic sample library of the present invention can be constructed by using other vector database platforms for storage as needed, such as Pinecone, Weaviate, Redis, etc., and can also be stored by using other database platforms, such as neo4j, excel, etc.; the sample paradigm can be adjusted as needed, such as reducing the thinking chain, increasing the thinking chain corresponding to the standard reasoning line, increasing the relevant background knowledge, etc.

[0086] The vectorization of the present invention can adopt other vector models as needed, such as word2vec, m3e, bge, etc., and can self-train the vector model in the field when necessary; the vector dimension can be adjusted as needed, such as 128, 768, 1536, etc.

[0087] Example 3

[0088] The dynamic sample recall of the present invention adjusts the recall similarity threshold range as needed, such as 0.5-1.1, 0.6-0.9, 0.5-0.8, etc.; adjusts the recall of appropriate TOPN similar samples as needed, such as N is 1, 5, 10, 20, etc.; and can use other retrieval methods or a combination of several retrieval methods for recall as needed, such as: knowledge graph full-text index and vector index, keyword retrieval, BM25 retrieval, BM25 and vector index combination, etc.

[0089] The similarity judgment of the present invention may adopt other similarity calculation methods, such as Euclidean distance, inner product, etc., as needed.

[0090] Example 4

[0091] The training corpus construction of the present invention can adjust the content of the training corpus as needed, which can be reduced to only fine-tuning the entity recognition data, or can be further expanded to supplement relevant knowledge, enhance data and other content for training.

[0092] The large model selection of the present invention can adopt other base large models as needed, such as Qwen2.5 series, Qwen2 series, LLAM3 series, Phi-3.5 series, GPT-3 series, etc.; deep learning models can be adopted as needed, such as BERT, Transform, LSTM, etc.; subsequent reinforcement training can be supplemented as needed.

[0093] The large model recognition of the present invention can directly adopt existing open source large models, such as Qwen, Kimi, llama, etc., as needed, and directly realize entity recognition through high-quality prompt words combined with samples.

[0094] It should be noted that the embodiments mentioned above illustrate rather than limit the present invention, and those skilled in the art will be able to design alternative embodiments without departing from the scope of the claims. Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention, so the scope of protection of the present invention shall be subject to the scope defined by the claims.

Claims

1. A method for named entity recognition combining a large model and dynamic examples, characterized in that: The steps are as follows: S1. Build a dynamic sample library; S2. Construct basic prompt words: clarify the business entity annotation background, rules, and output specifications, and construct XML-style structured prompt words; S3, dynamic sample recall; S4, training corpus construction, including entity recognition corpus construction and recognition reasoning corpus construction; S5, large model selection; S6, large model recognition, integrating instance d, recall sample set R, and basic prompt words into the trained model, and outputting the final recognition entity result; S7, API service encapsulation, encapsulates dynamic sample recall, prompt words, and selected large models into API services for easy calling.

2. The method for named entity recognition combining a large model and dynamic examples according to claim 1, characterized in that: The specific steps of step S1 to construct a dynamic sample library are as follows: 1) Continuously collect common expression data and problem data for annotation to build a representative sample data set S = {s1, s2, ..., s n }; where data s i Use JSON format to store various information; 2) Store the entire sample dataset S on the zilliz cloud platform.

3. The method for named entity recognition combining a large model and dynamic examples according to claim 2, characterized in that: In step S1, the data s i The key-value pairs for storing various information in JSON format involve the following contents: a. Embedding vector, which converts the search content into a 1024-dimensional vector representation through the text2vec-large-chinese model; b. text data, complete data with context; c. label tag, output result in JSON format of the labeled entity; d. COT thinking chain, reasoning chain of entity recognition.

4. The method for named entity recognition combining a large model and dynamic examples according to claim 1, characterized in that: The specific steps of the step S3 dynamic sample recall are as follows: 1) All the content to be retrieved in instance d L = {l1,l2,...,l m }Then the text2vec-large-chinese model is used to convert the vector into a 1024-dimensional vector representation to obtain the matrix V = {v1, v2, ..., v m }; 2) Search based on the zilliz cloud platform and issue recall v i (i=1,2,……m) TOP1 similar examples r in the sample data set S whose cosine distance similarity is between 0.7-1.1 (the similarity of completely identical content will be higher than 1, so the upper limit is temporarily set to 1.1) i (i=1,2,……m); 3) All similar examples c i After removing duplicates, the recall sample set R = {r1, r2, ..., r k }(k<=m).

5. The method for named entity recognition combining a large model and dynamic examples according to claim 1, characterized in that: The specific steps of constructing the entity recognition corpus in step S4 are as follows: 1) Collect a certain amount of data sets with scene coverage N = {n1, n2, ..., n s }; 2) For data n i (i=1,2,……s) construct fusion prompt word p i (i=1,2,……s), including the following three content data: data n i 、Data i Obtain the recalled sample set R through dynamic sample recall i , basic prompt words; 3) Change p i (i=1,2,……s) are successively recognized by the doubao-pro-4k large model to obtain the recognition data u i (i=1,2,……s); 4) Identify the data u i The doubao-pro-4k large model is used to test and judge again, and finally the correct identification data set N1 and the identification data set N2 to be confirmed are obtained; 5) Manually verify the unidentified dataset N2 and integrate the correctly identified dataset N1 to obtain the final entity training corpus T1 = {t 11 ,t 12 ,……,t 1s }.

6. The method for named entity recognition combining a large model and dynamic examples according to claim 1, characterized in that: The specific steps of identifying the construction of the inference corpus in step S4 are as follows: 1) Select data with different difficulty levels from the correct recognition data set N1 and the unrecognized recognition data set N2 to form a data set W = {w1, w2, ..., w l }; 2) The data w i (i=1,2,……l) The inference data o is obtained by constructing the inference thinking chain through the doubao-pro-4k large model i (i=1,2,……l); 3) The inference data o i The doubao-pro-4k large model is used to test and judge again, and finally the correct inference data set O1 and the inference data set O2 to be confirmed are obtained; 4) Manually verify the inference dataset O2 to be confirmed, and integrate the inference correct dataset O1 to obtain the final inference training corpus T2 = {t 21 ,t 22 ,……,t 2s }.

7. The method for named entity recognition combining a large model and dynamic examples according to claim 1, characterized in that: The specific steps of the large model selection in step S5 are as follows: 1) Comparison base selection: l lama3.1-8b, qwen2.5-7b, l lama3-8b, l lama2-7b and other base models were selected for comparison training; 2) Model training: Use entity training corpus T1 and reasoning training corpus T2 to fine-tune the base model respectively; 3) Model evaluation: Use the validation set to evaluate each comparison model, calculate indicators such as accuracy, recall, and F1 score, and select the best trained model for final adoption.

Images