Large model named entity recognition method and system based on representative sample selection and context enhancement

Abstract

The invention provides a large model named entity recognition method based on representative sample selection and context enhancement, which comprises a representative sample selection module, an entity knowledge construction module, a dynamic context selection module, a large model calling module and an iterative feedback optimization module, according to representative sample selection, samples with representativeness and information diversity are automatically selected from unlabeled data for labeling through a sample screening strategy based on clustering, entity description integration aims at each entity type, a plurality of high-quality instances are extracted from labeled samples, and standardized entity definition or description prompts are constructed. According to the dynamic context selection, for to-be-recognized text content, a context example most relevant to a target text is dynamically selected from a historical annotation sample or a description set through a semantic similarity retrieval mechanism to serve as auxiliary prompt input, and the adaptability and generalization ability of LLM in a complex or variable scene are improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence natural language processing, and in particular to a large-model named entity recognition method and system based on representative sample selection and context enhancement. Background Art

[0002] With the rapid development of large-scale pre-trained language models (LLMs) in the field of natural language processing, named entity recognition (NER), as a core subtask in information extraction, plays a vital role in practical applications such as financial risk control, clinical medicine, and knowledge graph construction. However, its generalization and robustness are still insufficient when faced with cross-domain applications, data distribution shifts, and novel entity types. Existing NER methods usually rely on manual random selection of training samples for labeling. This method cannot fully cover the semantic diversity and typical features in the data, which in turn affects the model's ability to recognize rare entities and complex scenarios. The model usually uses fixed entity definitions in the inference stage, resulting in a lack of semantic understanding of specific or ambiguous entity types. Most current NER processes are single-round static inference, lacking a summary and feedback mechanism for recognition errors, and cannot achieve continuous optimization and adaptive adjustment of model performance.

[0003] The present invention solves three existing challenging problems: (1) How to effectively select representative and information-diverse samples from large-scale unlabeled corpora, reducing the cost of manual labeling while improving the coverage and practicality of training data; (2) How to guide large language models to understand and distinguish semantically ambiguous or domain-specific entity types without relying on fixed entity definitions, thereby improving the model's discrimination ability in complex scenarios; (3) How to build a closed-loop named entity recognition process so that the model can continuously learn from erroneous results and self-optimize, breaking through the limitations of traditional static reasoning models in generalization ability and continuous improvement. Summary of the Invention

[0004] In order to solve the problems of high manual labeling cost, dependence of entity definition on manual experience and poor model adaptability in the existing LLM-based NER tasks, the present invention discloses a large-model named entity recognition method and system based on representative sample selection and context enhancement, and proposes a multi-module integrated prompt optimization method including representative sample selection, entity knowledge construction, dynamic context selection and iterative feedback optimization, which systematically optimizes the large language model input from multiple dimensions from data selection, prompt construction to model feedback.

[0005] The present invention adopts the following technical solutions to achieve the above-mentioned purpose:

[0006] This paper proposes a large-model named entity recognition method based on representative sample selection and context enhancement. The method mainly includes a representative sample selection module, an entity knowledge construction module, a dynamic context selection module, a large model calling module, and an iterative feedback optimization module. The specific contents of each module are as follows:

[0007] (1) Representative sample selection module

[0008] The representative sample selection module aims to select representative and diverse samples from large-scale unlabeled data, reducing redundant annotations and improving data utilization and annotation efficiency. Through processes such as text encoding, cluster analysis, and diversity calculation, this module effectively balances representativeness and diversity, outputting a batch of core samples for initial annotation. Specifically, it includes the following six steps:

[0009] Step 1: The representative sample selection module first encodes the input text set, converts each text into a vector representation using the Sentence-BERT model, and constructs a high-dimensional semantic embedding matrix to preserve the semantic structure and similarity information between texts.

[0010] Step 2: The module performs cluster analysis on the encoded vector data, using the HDBSCAN clustering algorithm by default. This algorithm automatically determines the number of clusters and handles noise, enabling it to identify latent semantic categories from unsupervised text vectors and providing a foundation for representativeness calculations.

[0011] Step 3: The module calculates the centroid of all sample embedding vectors in each non-noise cluster (i.e., the label is not -1) for subsequent evaluation of sample representativeness. This process obtains the cluster center vector by taking the average of the sample vectors in each cluster.

[0012] Step 4: The module traverses all samples, calculates their distance to the cluster center, and then calculates a representativeness score. Samples with smaller distances are more representative of the semantic center of the cluster and have higher representativeness scores. The score results are used to guide the "coverage" of sample selection.

[0013] Step 5: During each round of sample selection, the module calculates the minimum distance between each candidate sample and the currently selected sample set. This distance serves as the basis for the diversity score, encouraging the selection of text that is far from existing samples and enhancing the semantic diversity of the sample set.

[0014] Step 6: The module calculates a final score for each sample based on its representativeness and diversity scores. This combined score is weighted using an adjustable balance factor to balance the two objectives of representativeness and diversity. Finally, the module iteratively selects the highest-scoring samples and outputs the dataset for initial manual annotation.

[0015] Through the above process, the representative sample selection module can screen out sample sets with the greatest information value and balanced structural distribution from large-scale unlabeled corpus, laying a solid foundation for subsequent prompt construction and knowledge mining, while significantly reducing the cost and workload of manual labeling.

[0016] (2) Entity knowledge construction module

[0017] The Entity Knowledge Building Module aims to generate refined semantic descriptions for each named entity type, improving the large language model's ability to understand various entities. This module is particularly suitable for addressing issues such as ambiguity, blurred boundaries, and overlapping labels in fields like medicine. This module constructs a library of entity definitions that can be inserted into prompts by performing structured extraction of entity information from annotated samples, constructing prompts, and generating a large model. Specifically, it includes the following six steps:

[0018] Step 1: The entity knowledge construction module first reads the annotated sample file, traverses the text and entity annotations, identifies all entity types, and extracts several examples for each entity type. Each example contains the original entity text and its context (the context is a text window of approximately 50 characters before and after the entity), and is marked with symbols to help the large language model understand the context of the entity.

[0019] Step 2: To prevent data redundancy and keep the input concise, the module filters the examples extracted for each entity type, sets a maximum number of examples, and uses a random sampling strategy to retain some representative context samples when the sample size is too large to ensure that the semantic features of each type of entity are covered but not overloaded.

[0020] Step 3: The module constructs a standardized prompt template based on the entity type and its examples. The prompt contains the model's task instructions (e.g., "Please generate a description for the following entity type") and several contextual annotation examples for each entity type. This template is written to a file in plain text format, providing the input for subsequent large language model calls.

[0021] Step 4: The module uses an external API (such as the DeepSeek API) to call the large language model, passing the prompt template as input text and setting parameters such as temperature and maximum output length to obtain the entity type description text generated by the model based on the example. To ensure stability, a low temperature (such as 0.1) is used in the call to improve the consistency and accuracy of the output description.

[0022] Step 5: The module performs a structured analysis of the natural language response returned by the large language model, automatically identifying the key-value pairs contained therein, extracting the semantic definition corresponding to each entity type into a standard JSON structure, and saving the result as an entity knowledge base file (e.g., entity_descriptions.json). This structured definition can be directly used in subsequent prompt construction, improving the model's semantic guidance capabilities when identifying ambiguous entity types.

[0023] Step 6: After completing the initial knowledge construction, the entity knowledge construction module allows the prompt text of the entity type to be reconstructed based on typical error samples in the subsequent model prediction results, and calls the large language model in combination with the new examples to correct or expand the definition expression of the entity.

[0024] Through the above steps, the entity knowledge construction module can transform the entity distribution information in large-scale annotated samples into compact and clear semantic definitions, significantly improving the model's discrimination ability, context adaptability and cross-sample generalization performance when processing domain entities.

[0025] (3) Dynamic context selection module

[0026] The Dynamic Context Selection module aims to improve the large language model's ability to discern named entity boundaries and categories. By dynamically retrieving the most relevant context samples for each piece of text to be recognized, it constructs targeted prompt input before model inference, enhancing the model's adaptability to semantic scenarios and its ability to identify entity types. This module combines semantic similarity calculation and vector retrieval techniques to build a prompt generation mechanism based on task context. Specifically, it includes the following four steps:

[0027] Step 1: Let the training sample set be: D train ={(x1,y1),(x2,y2),...(x N ,y N )}where x i represents the i-th training text, y i Denotes the set of labeled entities. Let the test sample set be: D test ={x′1,x′2,...,x′ M} represents the set of unlabeled text to be recognized. These samples will be used to build a semantic vector library and perform similarity retrieval between samples. The module loads the labeled training sample set and the test sample set to be predicted, providing the necessary materials for subsequent similarity matching and prompt construction.

[0028] Step 2: The module calls the pre-trained sentence vector model to encode the training and test texts, converting the original text in the training and test samples into high-dimensional semantic vector representations, retaining the semantic structure information, and facilitating subsequent semantic retrieval. The vector encoding function is Then the vector representation of the training set: H train ={h1, h2,..., h N}, The vector representation of the test set: H test ={h′1, h′2,..., h′ M}, where d is the dimension of the sentence vector,

[0029] Step 3: For each test sample text to be recognized, the module calculates the cosine similarity between its semantic vector and all semantic vectors in the training samples: And sorts them in descending order according to the similarity, and selects the top K training samples with the highest similarity TopK i (sim(h′ j , h i )). These selected samples are regarded as the context support information of the current test statement, and are used to provide references for similar expression structures or entity distributions.

[0030] Step 4: The module formats the entities in the retrieved similar training samples. By marking the entity positions in each training sample, the original text is transformed into an annotated format text using a structure such as <entity type>entity text< / entity type>, and several Few-shot NER task examples are constructed as the prompt part of the model input.

[0031] (4) Large model call module

[0032] This module aims to call a large language model (such as GPT, GLM, DeepSeek, etc.) to perform the Named Entity Recognition (NER) task based on the prompt input after entity knowledge construction and dynamic context enhancement. The module performs a structured analysis on the model output results, extracts standardized entity information, including entity text, category, and position in the original text, and finally generates entity annotation results that can be used for downstream processing. It mainly consists of the following four key technical steps:

[0033] Step 1: Based on the entity type definitions, example corpora, and error experience summaries provided by the entity knowledge construction module, dynamic context selection module, and iterative feedback optimization module, the module organizes the model input content using a unified Prompt template.

[0034] Step 2: The module submits the above Prompt input to a preset large language model (such as OpenAI GPT, DeepSeek Chat, ChatGLM, etc.) through the API interface, and sets the following parameters:

[0035] 1. Model name: such as gpt-3.5-turbo, deepseek-chat

[0036] 2.Temperature: Set to a lower value (such as 0.1) to improve output consistency

[0037] 3. Max tokens: Set the upper limit of output length to ensure coverage of all entities

[0038] 4. Format constraints: Use guidance prompts to require model output to follow a specified annotation format

[0039] Step 3: The module performs structural analysis on the output text of the large model and identifies all token structures that conform to the regular expression: <(\w+?)>(.*?)< / \1> For each matching result, the following three types of information are extracted:

[0040] 1. Entity type (label)

[0041] 2. Entity

[0042] 3. Original text position (start, end): The offset is calculated by the position where the entity text first appears in the original text

[0043] The output results are uniformly stored in a JSON structure to facilitate evaluation and downstream task calls.

[0044] (5) Iterative feedback optimization module

[0045] This module aims to build a closed loop of prediction-verification-analysis-prompt update in named entity recognition tasks. By analyzing the error patterns of large language models in entity recognition tasks, it dynamically optimizes prompt engineering and entity knowledge definition to achieve continuous improvement in model performance. Based on an iterative execution mechanism, the module automatically evaluates the discrepancies between model prediction output and manual annotation, identifies common errors, and combines with external LLMs (such as DeepSeek) to generate targeted improvement suggestions, which are then used to update the next round of recognition prompts. The process includes the following six steps:

[0046] Step 1: Model output evaluation and error analysis

[0047] The module first compares the model-predicted entities with the manually annotated entities for each test sample, and counts three typical types of errors:

[0048] 1. Missing entities: entities that exist in manual annotations but are not recognized by the model;

[0049] 2. Mislabeled entities: entities that are recognized by the model but do not exist in the manual annotations;

[0050] 3. Boundary or type error: The entity is recognized but the boundary is inaccurate or the type is wrong.

[0051] By representing entities as triplets (label, start_offset, end_offset), calculating set intersection and difference, and evaluating the model's precision, recall, and F1 score:

[0052]

[0053] Step 2: The module counts the entity recognition performance of all test samples by entity type, calculates the total number of entities of each type, the number of correct recognitions, the number of missed labels, the number of mislabeled entities, and the recall rate, and identifies the most frequently erroneous entity types.

[0054] Step 3: The module automatically generates a structured summary of the evaluation results and calls the DeepSeek API to request it to generate a detailed error analysis and next round of annotation guidance. The analysis includes:

[0055] 1. Worst / best performing entity type;

[0056] 2. Statistics of missed labels / mislabeled labels / boundary errors;

[0057] 3. Short entity recognition problems, type confusion and other patterns;

[0058] 4. Recommend improvement measures and rule strengthening.

[0059] Step 4: The module extracts a structured JSON object from the LLM analysis results as an "annotation guide" in the following format:

[0060] {"Entity Definition":{"Entity Type":"Entity Description"},"Annotation Rules":[],"Boundary Identification":"","Focus":[],"Priority":{"Entity Type 1":Specific Entity 1,"Entity Type 2":Specific Entity 2}}

[0061] Step 5: The extracted "annotation guidelines" will be incorporated into the next round of prompts, alongside entity definitions and examples, to form a more comprehensive prompt input structure. The system uses these prompts to supplement definitions and examples of frequently erroneous entity types, annotation rules and entity boundaries, and recognition priorities and considerations to improve the accuracy of the next round of model recognition.

[0062] Beneficial effects of the present invention:

[0063] 1. Representative sample selection: Through a clustering-based sample screening strategy, representative and diverse samples are automatically selected from unlabeled data for labeling. This can significantly reduce labor costs while ensuring data coverage and effectiveness, alleviating the redundancy problem in the traditional labeling process.

[0064] Entity Description Integration: For each entity type, we extract several high-quality instances from annotated samples to construct standardized entity definitions or descriptions. These descriptions can be integrated into the LLM input as prior knowledge, improving the model's ability to distinguish domain terms, similar types, and ambiguous entities, enhancing recognition accuracy.

[0065] 3. Dynamic context selection: For the text content to be recognized, the semantic similarity retrieval mechanism dynamically selects the context examples most relevant to the target text from historical annotation samples or description sets as auxiliary prompt input, improving the adaptability and generalization ability of LLM in complex or changing scenarios.

[0066] 4. Iterative feedback optimization builds a closed-loop mechanism between model prediction and manual verification, automatically collects recognition errors, analyzes error types through language models, and optimizes entity descriptions and summarizes error experiences accordingly, thereby adaptively improving the overall performance of the system.

[0067] 5. In addition, there is a potential synergistic enhancement effect among the various modules of this method: the clustering results can assist in more accurate context retrieval, and the feedback analysis can guide the dynamic adjustment of entity descriptions and error experiences, ultimately forming a continuously optimized closed-loop system, which improves the robustness of LLM in NER tasks.

[0068] 6. The present invention focuses on improving the generalization and continuous optimization capabilities of large language models for small-sample entity recognition tasks, emphasizes sample selection and Prompt optimization mechanisms, and has a wide range of applications. It introduces Sentence-BERT and HDBSCAN to ensure the representativeness and diversity of examples, automatically generates entity definitions through LLM and stores them in a structured manner, and uses LLM to analyze error patterns and dynamically optimize prompt content to achieve self-evolution of prompt templates. Based on real labeled data, it selects an efficient initial training set through a representative sample selection module, uses an entity knowledge construction module to generate structured definitions, and uses a dynamic context selection module to retrieve Top-K similar samples for each text to be recognized and construct prompts. It then cooperates with the large model calling module to complete the NER task, and it iteratively optimizes the prompt content and annotation rules through the feedback evaluation module. It does not rely on ontology word bags or domain-specific knowledge graphs, and is suitable for NER tasks with pan-domain, multi-label, and multi-round optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 , Schematic diagram of the named entity recognition method based on large language model based on representative sample selection and context enhancement;

[0070] Figure 2 ,Representative sample selection module flow chart;

[0071] Figure 3,Entity knowledge construction module flow chart;

[0072] Figure 4 Dynamic context selection module flow chart;

[0073] Figure 5 Flowchart of large model calling module;

[0074] Figure 6 Flowchart of the iterative feedback optimization module. DETAILED DESCRIPTION

[0075] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are intended only to illustrate the present invention and are not intended to limit the scope of the present invention. It should be noted that the terms "front," "rear," "left," "right," "up," and "down" used in the following description refer to directions in the accompanying drawings, and the terms "inward" and "outward" refer to directions toward or away from the geometric center of a particular component, respectively.

[0076] like Figure 1 As shown in the figure, a large-model named entity recognition method based on representative sample selection and context enhancement is proposed. The specific implementation case is as follows:

[0077] S1: Representative sample selection module

[0078] The representative sample selection module aims to select a small number of samples with high information density and balanced structure distribution from the unlabeled text collection as the initial prompt examples for the subsequent large language model Few-shot named entity recognition task. Figure 2 As shown in the figure, this module integrates semantic vector encoding, cluster analysis, representativeness estimation and diversity control technologies to achieve pre-sample screening of representative annotations. Specifically, it includes the following steps:

[0079] S101: The representative sample selection module first receives the input text collection and calls the Sentence-BERT model to encode each text, mapping the natural language text into a high-dimensional semantic vector. embeddings = self.sbert.encode(texts, show_progress_bar = True). This semantic embedding matrix preserves the semantic structure between samples, providing a basis for subsequent clustering and sample selection.

[0080] S102: The module applies the HDBSCAN clustering algorithm to the encoded vector matrix H to automatically identify semantic subclusters in the text: cluster_labels = self.clusterer.fit_predict(embeddings). HDBSCAN supports density-based adaptive clustering and can automatically filter noise points (labeled as -1), thereby constructing potential semantic categories in an unsupervised setting.

[0081] S103: For each valid cluster (i.e., label not equal to -1), the module calculates its semantic centroid to estimate the representativeness of each member sample to the cluster. The centroid calculation adopts the Euclidean mean method. The specific operation is as follows: cluster_points = embeddings[labels == label]

[0082] centers[label] = np.mean(cluster_points,axis=0), all cluster centroids will be used to guide the representative scoring of subsequent samples.

[0083] S104: The module traverses all samples and calculates the Euclidean distance between them and the centroid of their cluster. The smaller the distance, the more the sample covers the core content of the semantic cluster. The module constructs a representativeness score array, representativeness_scores, and uses the "center proximity" of each sample to guide sample coverage selection.

[0084] S105: In each round of sampling, the module calculates h for each candidate sample based on the currently selected sample set S. i The minimum cosine distance to any vector in the set S is used as the diversity indicator, divi = min h j ∈s(1-cos(h i , h j )) is implemented as follows: distances=pairwise_distances(embeddings,selected_embeddings); min_distances=np.min(distances, axis=1); scores=min_distances / np.sum(min_distances)

[0085] S106: The module performs a weighted fusion of the representativeness score and the diversity score to calculate the final selection score of the sample: score i =(1-λ)·rep i +λ·div i, where λ∈[0,1] is an adjustable balance factor. Finally, the module uses a greedy strategy to iteratively select the sample with the highest score and exclude the selected samples until the set number n_select is met. The implementation is as follows: final_scores = (1-balance_factor)*representativeness_scores + balance_factor*diversity_scores; next_idx = np.argmax(final_scores); selected_indices.append(next_idx)

[0086] The selected samples are written to the selected_samples.jsonl file for use by subsequent modules.

[0087] Through S101 to S106, the representative sample selection module constructs a high-quality initial annotated sample set that takes into account both representativeness and diversity, providing a data basis for the construction of large language model prompts and significantly reducing the cost of manual annotation.

[0088] S2: Entity knowledge building blocks, e.g. Figure 3 As shown in Figure 2, this module is used to generate clear and accurate semantic definitions for each type of named entity, enhancing the model's ability to understand entity categories.

[0089] S201: The entity knowledge construction module first reads the annotated sample file selected_samples.jsonl and parses the text and entities fields within each sample to identify all entity types. For each entity, the original text (entity_text) and its context window (default: a text fragment within 50 characters before and after the entity) are extracted. The entity occurrences are explicitly annotated in the context using the [] symbol to help the model identify the entity's contextual characteristics.

[0090] S202: The module screens all contextual annotation examples extracted for each entity type, setting a maximum number of examples (30 by default). If the sample size exceeds the limit, a random sampling strategy is used to retain a representative number of contextual instances, ensuring semantic coverage while controlling the length of the prompts. The screened entity examples will serve as the core content for subsequent template construction.

[0091] S203: The module constructs a uniformly formatted prompt template text based on the filtered entity types and context examples. This text includes the task description, the name of each entity type, the context example number, the annotation content, and the context annotation. It is written uniformly in plain text to the file llm_entity_description_prompt.txt. This prompt template is used to construct input content during the large language model invocation phase.

[0092] S204: This module calls an external large language model (such as the DeepSeek API) using the Prompt constructed in S203 as input, setting parameters such as a temperature of 0.1 and a maximum number of output tokens of 2000 to retrieve the definition text for each entity type output by the model. This call follows the conversational API protocol, organizing prompts in a structured message format to improve output consistency and description accuracy.

[0093] S205: After receiving the natural language output from the large language model, the module automatically extracts the entity type descriptions presented as key-value pairs and parses them into a standardized JSON object. This parsing method supports format adaptation, adapting to syntax fluctuations in different outputs. The parsed JSON structure becomes the entity definition knowledge base and is written to the file entity_descriptions.json.

[0094] S206: The entity knowledge construction module supports continuous updates based on an iterative feedback mechanism. When the model encounters problems with new entity recognition types or invalidating existing definitions in subsequent entity recognition tasks, the system automatically collects these typical error samples, reconstructs the prompt, and submits it to the main model with the newly added context examples to generate improved entity definitions, enabling dynamic updates and adaptive expansion of the knowledge base.

[0095] In summary, steps S201 to S206 constitute an entity knowledge construction process that automatically extracts entity context from annotated samples, builds prompts, generates definitions, and dynamically optimizes and updates them, effectively enhancing the semantic understanding and prompt adaptability of large language models for named entity categories.

[0096] S3: Dynamic context selection module, such as Figure 4 As shown in the figure, this module is used to dynamically construct a few-shot prompt containing semantically similar samples and entity definitions for each text to be recognized, thereby improving the recognition context adaptability.

[0097] S301: The module first loads the labeled training set selected_samples.jsonl and the unlabeled test set test_samples.jsonl, extracts the original text of each sample, and stores them as the train_texts and test_texts lists respectively.

[0098] S302: The module calls SentenceTransformer to load the specified sentence vector model (such as "paraphrase-multilingual-MiniLM-L12-v2"), encodes the training text and the test text respectively, and generates the high-dimensional sentence vector matrices train_embeddings and test_embeddings for subsequent cosine similarity calculation and vector retrieval.

[0099] S303: For each test sample, the module calls cosine_similarity to calculate the similarity matrix between its vector and all sample vectors in the training set, sorts them from high to low similarity, and selects the top K samples as the "context support set" of the current test text for Few-shot prompting.

[0100] S304: The module formats the entities of the retrieved similar training samples through the add_entity_tags() function, embeds them into the original text in the form of <entity type>entity text< / entity type>, and concatenates them with the entity type description as the Few-shot prompt input; finally, it concatenates the prompt with the current test sample to construct a complete Prompt, which is saved to the ner_prompts / directory for subsequent large language model calls.

[0101] S4: The large model entity recognition module, such as Figure 5 As shown, this module uses a large language model to perform the NER task on the constructed Prompt and outputs a structured annotation result.

[0102] S401: The module first integrates the input elements generated by the previous modules, including: the entity definitions entity_descriptions.json produced by the entity knowledge construction module, the similar context samples similar_samples selected by the dynamic context selection module, and the error prompts summarized by the iterative feedback module (such as common misannotations, entity boundary rules), to construct a unified format Prompt, including: instruction description (such as "Please annotate the following text in the <tag>entity< / tag> format"), entity type definition (a concise description for each type of entity), Few-shot examples (labeled training texts), and the test text to be recognized.

[0103] S402: The module calls a pre - set large - language model (such as OpenAI GPT - 3.5, DeepSeek, ChatGLM, etc.) through the API, submits the constructed Prompt as the input text, and sets the following parameters to control the model behavior: model: the name of the model, such as "gpt - 3.5 - turbo", "deepseek - chat"; temperature: set to 0.1 to improve output consistency; max_tokens: set the maximum number of tokens (such as 2000) to ensure a complete output; messages: construct the dialogue input using {"role":"user","content":prompt}.

[0104] S403: The module analyzes the returned model annotation results, identifies all entity - tag structures that conform to the <entity type> entity < / entity type> pattern, and extracts label (entity category), entity (original entity text), start_offset (the position where it first appears in the original test text), and end_offset (the end position of the entity, calculated as start+len(entity)).

[0105] S5: Iterative feedback optimization module, such as Figure 6 As shown, after each round of large - language model prediction of named entities, this module systematically analyzes the differences between its output results and manual annotations, identifies typical error samples, and uses the analysis results to dynamically update entity definitions and annotation rules, constructing a closed - loop mechanism for continuous optimization of model performance.

[0106] S501: The module first conducts entity - level prediction - truth comparison for each test sample. Represents the entity as a triple (label, start, end), and uses set operations to count: correctly identified (TP), missed labels (FN), mis - labels (FP), and calculates entity - level metrics: <0000 (14]]

[0107] S502: The module uses analyze_errors(all_evaluation_results) to calculate the number of samples correctly identified, missed, and mis - labeled for each type of entity, and based on this, calculates the recall rate and F1 for each entity category, identifying the entity types that are most likely to be mis - handled.

[0108] S503: According to the evaluation results of S501 and S502, the module constructs a structured Prompt, calls a large - language model such as DeepSeek, and generates two parts of content:

[0109] Detailed analysis (natural language): statistics on missed / mislabeled / boundary errors, poor entity types, short entity recognition issues, confusion patterns, and targeted annotation improvement suggestions

[0110] Annotation Guide (structured JSON), including entity definition supplements, annotation rules, boundary identification, key points, and priorities

[0111] S504: The module uses extract_json_from_response(response) to extract the JSON segment from the model return text and parse it to generate a new round of "annotation guide" objects

[0112] S505: The module automatically splices the "annotation guide" generated by S504 into the next round of prompts, so that the new round of prompts not only includes: entity definition, example context, and test text, but also includes error experience summary, reinforcement of entity rules, and focus on difficult type guidance, thereby forming a prompt evolution mechanism with self-correction capabilities, and continuously improving model recognition accuracy and generalization capabilities.

[0113] The technical means disclosed in the solution of the present invention are not limited to the technical means disclosed in the above-mentioned embodiment, but also include technical solutions composed of any combination of the above technical features.

Claims

1. A large-model named entity recognition method based on representative sample selection and context enhancement, characterized by: The following steps are involved: Step 1: Select representative and diverse samples from large-scale unlabeled data; Step 2: Generate a refined semantic description for each named entity type. By performing structured extraction of entity information from the annotated samples, constructing hints, and generating a large model, a library of entity definitions that can be inserted into hints is constructed. Step 3: By dynamically retrieving the most relevant context samples for each piece of text to be recognized, we construct targeted prompt input before model inference. We use semantic similarity calculation and vector retrieval technology to build a prompt generation mechanism based on the task context. Step 4: Input the prompts enhanced with entity knowledge and dynamic context into the large language model, perform the named entity recognition task, and parse the output results to extract standardized entity information; Step 5: Build a closed loop of prediction-verification-analysis-prompt update in the named entity recognition task. By analyzing the model error pattern, iteratively optimize the prompt engineering and entity knowledge definition, automatically evaluate the prediction results, identify errors, and combine with the external LLM to generate improvement suggestions for updating the next round of recognition prompt content.

2. The large-model named entity recognition method based on representative sample selection and context enhancement according to claim 1, characterized in that: The step 1 specifically includes the following steps: Step 11: First, encode the input text collection and use the Sentence-BERT model to convert each text into a vector representation. A high-dimensional semantic embedding matrix is ​​constructed to preserve the semantic structure and similarity information between texts. Step 12: The module performs cluster analysis on the encoded vector data, using the HDBSCAN clustering algorithm by default; Step 13: Calculate the centroid of all sample embedding vectors in each non-noise cluster, that is, all sample embedding vectors whose labels are not -1, for subsequent evaluation of sample representativeness. This process obtains the cluster center vector by taking the average of the sample vectors in each cluster. Step 14: Traverse all samples and calculate their distance to the cluster center, and then calculate the representativeness score. The smaller the distance, the more representative the sample is of the semantic center of the cluster, and the higher its representativeness score. The score result is used to guide the "coverage" of sample selection. Step 15: In each round of sample selection, calculate the minimum distance between each candidate sample and the currently selected sample set; Step 16: Calculate the final score for each sample based on the representativeness score and diversity score; The comprehensive score is weighted and integrated through an adjustable balance factor to balance the two goals of "representativeness" and "diversity"; finally, the module iteratively selects several samples with the highest scores and outputs a data set for initial manual annotation.

3. The large-model named entity recognition method based on representative sample selection and context enhancement according to claim 1, characterized in that: The step 2 specifically includes the following steps: Step 21: First, read the annotated sample file, traverse the text and entity annotation information, identify all entity types, and extract several examples for each entity type. Each example contains the original entity text and its context information, and is marked with symbols to help the large language model understand the context in which the entity appears. Step 22: Filter the extracted examples for each entity type, set a maximum number of examples, and use a random sampling strategy to retain some representative context samples when the sample size is too large to ensure that the semantic features of each entity type are covered but not overloaded; Step 23: Construct a standardized prompt template text based on the entity type and its examples; the template text is written to a file in plain text format to provide an input basis for subsequent large language model calls; Step 24: The module uses an external API to call the large language model, passes the prompt template as input text, and sets the temperature and maximum output length parameters to obtain the entity type description text generated by the model based on the example; Step 25: Perform structured parsing on the natural language response returned by the large language model, automatically identifying the key-value pair format contained therein, extracting the semantic definition corresponding to each entity type into a standard JSON structure, and saving the result as an entity knowledge base file; Step 26: After completing the initial knowledge construction, it is allowed to reconstruct the prompt text of the entity type based on typical error samples in the subsequent model prediction results, call the large language model in combination with the newly added examples, and correct or expand the definition expression of the entity.

4. The large-model named entity recognition method based on representative sample selection and context enhancement according to claim 1, characterized in that: The step 3 specifically includes the following steps: Step 31: Let the training sample set be: D train ={(x1,y1),(x2,y2),...(x N ,y N )}, where x i represents the i-th training text, y i Denotes the set of labeled entities; let the test sample set be: D text ={x′1,x′2,...,x′ M } represents the set of unlabeled texts to be identified; these samples will be used to build a semantic vector library and implement similarity retrieval between samples; the module loads the labeled training sample set and the test sample set to be predicted, providing necessary materials for subsequent similarity matching and prompt construction; Step 32: The module calls the pre-trained sentence vector model to encode the training and test texts, converting the original text in the training samples and test samples into high-dimensional semantic vector representations, retaining the semantic structure information, and facilitating subsequent semantic retrieval; the vector encoding function is Then the training set vector represents: H train ={h1, h2, ..., h N }, Test set vector representation: H test ={h′1,h′2,...,h′ M }, Where d is the dimension of the sentence vector; Step 33: For each test sample text to be recognized, the module calculates the cosine similarity between its semantic vector and all semantic vectors in the training samples: And sort them from high to low according to the similarity, select the top K training samples with the highest similarity i (sim(h′ j , h i )); These selected samples are regarded as the contextual support information of the current test sentence and are used to provide references for similar expression structures or entity distributions; Step 34: Format the entities in the retrieved similar training samples; by marking the entity positions of each training sample, construct several Few-shot NER task examples as prompts for the model input.

5. The large-model named entity recognition method based on representative sample selection and context enhancement according to claim 1, characterized in that: The step 4 specifically includes the following steps: Step 41: Based on the entity type definition, sample corpus, and error experience summary, use a unified prompt template to organize the model input content; Step 42: Submit the prompt input to the preset large language model through the API interface; Step 43: The module performs structured parsing on the large model output text and identifies all tag structures that conform to the regular expression.

6. The large-model named entity recognition method based on representative sample selection and context enhancement according to claim 1, characterized in that: The step 5 specifically includes the following steps: Step 1: Model output evaluation and error analysis. First, compare the model-predicted entities with the manually labeled entities for each test sample and count three typical errors: (1) Missing entities: entities that exist in manual annotations but are not recognized by the model; (2) Mislabeled entities: entities recognized by the model but not present in the manual annotations; (3) Boundary or type errors: the entity is recognized but the boundary is inaccurate or the type is wrong; By representing entities as triplets (label, start_offset, end_offset), calculating set intersection and difference, and evaluating the model's precision, recall, and F1 score: Step 2: Analyze the entity recognition performance of all test samples by entity type, calculate the total number of entities, the number of correct recognitions, the number of missed labels, the number of mislabeled entities, and the recall rate for each type of entity, and identify the most frequently mislabeled entity types; Step 3: Automatically generate a structured summary prompt based on the evaluation results, and call the DeepSeek API to request it to generate a detailed error analysis and next round of annotation guidelines; Step 4: The module extracts a structured JSON object from the LLM analysis results as an "annotation guide". Step 5: The extracted "annotation guide" will be used as part of the new round of prompts, added to the entity definition and examples to form a more complete prompt input structure.

7. A large-model named entity recognition system based on representative sample selection and context enhancement, characterized by: It includes representative sample selection module, entity knowledge construction module, dynamic context selection module, large model calling module and iterative feedback optimization module. Representative Sample Selection Module: This module is used to automatically select representative and semantically diverse few-shot prompt samples from large-scale unlabeled datasets as the initial sample set before input to the large language model; Entity Knowledge Construction Module: This module aims to automatically construct semantic descriptions covering various entities from labeled samples, improving the large language model's ability to identify and generalize label categories; Dynamic Context Selection Module: Before the model performs named entity recognition (NER) tasks, this module dynamically selects the top-K most similar context samples for each predicted text through semantic retrieval. It then constructs a prompt input that integrates entity definitions and examples, ensuring the large language model's ability to discern entity boundaries and understand complex contexts. Large model calling module: This module is used to build a unified prompt input template, call the specified large language model to perform named entity recognition tasks, and perform structured parsing and extraction of model output results; The iterative feedback optimization module automatically compares the model output with manual annotations and performs error analysis after each round of named entity recognition, identifying performance shortcomings and common error patterns. It then uses the large language model to generate the next round of optimization strategies, enabling Prompt's continuous self-evolution.

Images