A few-shot contrastive augmentation fine-tuning method and system based on a large language model

Abstract

The present application relates to a kind of few sample contrast enhancement fine-tuning methods and systems based on large language model, for the named entity recognition in recruitment text;The method comprises: the original recruitment text is cleaned and format conversion, generates the input sample in line with natural language instruction format;In the case where the labeled sample is insufficient, construct positive and negative sample pairs to enhance the recognition ability of model to entity class and boundary;LoRA fine-tuning technology is used to update the low-rank parameters of pre-trained large language model, and combined with 4bit quantization training to reduce the consumption of computing resources;In the inference process of pre-trained large language model, through multi-dimensional joint confidence evaluation mechanism, the confidence of four dimensions of entity level, length, type and context is integrated, and the low-confidence recognition result is filtered after dynamic weighted normalization processing.The present application is suitable for recruitment recommendation, talent matching and other downstream tasks, with the advantages of high recognition accuracy, low training cost, strong system robustness and the like.

Description

Technical Field

[0001] This invention relates to the field of computer network information technology, and in particular to a few-sample contrast enhancement fine-tuning method and system based on a large language model. Background Technology

[0002] Recruitment, as an important component of human resource management, is widely used in various fields such as corporate talent screening, university career guidance, and talent market analysis.

[0003] With the development of natural language processing technology, named entity recognition (NER) has gradually become a key technology in recruitment text information extraction, covering the entire process from job description parsing to entity extraction of skills, education, experience, etc. Among related technologies, a preliminary system for structured processing of recruitment texts has been constructed through the collaborative operation of models such as BiGRU-ATT-CRF and RSBC with traditional rule systems. Specifically, this system includes key stages such as data collection, entity annotation, model training, and inference optimization. Large language models, with their rich semantic representation capabilities, provide new possibilities for cross-domain entity recognition. However, existing NER methods directly adopt traditional models or zero-shot hint strategies without fully considering the unstructured characteristics and domain specificity of recruitment texts. This may lead to problems such as blurred entity boundary recognition, scarce labeled samples, and unstable model output. Furthermore, the lack of domain-adaptive fine-tuning mechanisms may result in significant deviations in recognition results in practical applications, thus affecting the accuracy of job profile construction and the effectiveness of recruitment recommendation systems.

[0004] Large-scale language models, with their rich pre-training corpus resources and powerful reasoning capabilities, have been widely applied in various industries. However, the recruitment recommendation field, characterized by its commercial sensitivity and user privacy, rarely constructs accurate job profile maps. The cross-domain entity annotation involved is difficult and costly. Summary of the Invention

[0005] The present invention aims to at least partially solve one of the technical problems in the related art.

[0006] This invention proposes a few-sample contrastive enhancement fine-tuning method based on a large language model, expanding the performance boundaries of large language model fine-tuning, fully stimulating its autonomous learning ability, and further eliminating the bias of large model pre-training and fine-tuning training through multi-dimensional joint rule filtering. Compared with other entity extraction models in the field, this invention has the best performance in extraction accuracy, resource consumption, and annotation difficulty.

[0007] Another objective of this invention is to propose a few-sample contrast enhancement fine-tuning system based on a large language model.

[0008] To achieve the above objectives, this invention proposes a few-sample contrastive enhancement fine-tuning method based on a large language model for named entity recognition in recruitment texts, comprising the following steps:

[0009] In response to a job posting text input request, the original job posting text is preprocessed to generate an input sample that conforms to the natural language instruction format;

[0010] When the number of labeled samples is insufficient, positive and negative sample pairs are constructed. Positive sample pairs are generated by randomly replacing labeled entities in the same entity category, while negative sample pairs are generated by inserting real labeled entities in non-labeled positions, replacing real labeled positions with entities of non-uniform type, or randomly deleting some labeled entities.

[0011] A large language model is used for fine-tuning, and a pre-trained large language model is loaded through the LoRA fine-tuning training module.

[0012] During the inference process of the pre-trained large language model, the confidence of the identified entities is evaluated. The confidence evaluation includes entity-level confidence, entity-length confidence, entity-type confidence, and context confidence. The confidence of each dimension is dynamically weighted and normalized to obtain the final entity confidence score.

[0013] The final entity confidence score is compared with a preset filtering threshold. Based on the comparison result, the entity is output to the model inference and output module for constructing a job profile.

[0014] The few-sample contrast enhancement fine-tuning method based on a large language model in this invention may also have the following additional technical features:

[0015] In one embodiment of the present invention, the preprocessing of the original recruitment text to generate input samples conforming to the natural language instruction format includes:

[0016] The original recruitment text is separated into structured and unstructured content, retaining the unstructured descriptive parts;

[0017] Convert traditional NER annotation format into natural language format;

[0018] Align entity types to unify different annotation formats into predefined entity types.

[0019] In one embodiment of the present invention, the conversion of the traditional NER annotation format into a natural language format includes:

[0020] Based on internet industry technology reports, e-books, and papers, a specialized vocabulary for industry recruitment was constructed, and the original recruitment text data was used as the corpus input for entity recognition and job profile construction.

[0021] The NER model, based on a proprietary lexicon and fine-tuned with few samples, performs entity recognition on the unstructured original recruitment text data, identifies entities of knowledge, skills and personal characteristics, and performs data cleaning and sample filtering preprocessing according to job categories.

[0022] Based on the preprocessing results, structured labeled data and high-quality job profiles are output.

[0023] In one embodiment of the present invention, the step of constructing positive and negative sample pairs when the number of labeled samples is insufficient includes:

[0024] In positive sample generation, entities within the same entity category are randomly replaced;

[0025] In negative sample generation, real labeled entities are inserted into unlabeled locations, real labeled entities are replaced with non-uniform type entities, or some labeled entities are randomly deleted.

[0026] Positive and negative sample pairs are used to train the model to identify the categories and boundaries of entities.

[0027] In one embodiment of the present invention, in generating positive samples, randomly replacing entities within the same entity category includes:

[0028] The text is divided into the original input X = {x1, x2, ..., x...} n The entity l = {l1, l2, ..., l} is labeled with the following structure: m};

[0029] Construct a real sample set L from all entities of the same category in the same dataset. type =

[0030] {L1,L2,…,L i};

[0031] When generating positive samples, the entity l1 at the labeled position is randomly replaced with l2, where l1, l2 ∈ L. type That is, random replacement is performed in the same labeled vocabulary, the replacement probability of each label of each sample is set to p, and (X,L2) is set as an enhanced version of (X,L1), and the two form a pair of positive samples.

[0032] In one embodiment of the present invention, negative sample generation includes inserting real labeled entities at unlabeled locations, replacing real labeled locations with entities of non-uniform type, or randomly deleting some labeled entities, including:

[0033] During the negative sample generation process, unlabeled positions are randomly replaced with real labels, and entities of non-uniform type l1 are randomly replaced with l2 at the original real label positions, where l1∈L type1 ,l2∈Ltype2 Alternatively, the number of labeled samples in the real sample l can be randomly deleted and used to form negative sample pairs.

[0034] In one embodiment of the present invention, loading the pre-trained large language model through the LoRA fine-tuning training module includes:

[0035] Load the pre-trained large language model and freeze the original parameters;

[0036] Low-rank matrices are introduced into the attention layer and MLP layer of the model, and the model parameters are updated by training the low-rank matrices.

[0037] A 4-bit quantization training method is adopted, and the objective is jointly optimized using cross-entropy loss and InfoNCE loss. Cross-entropy loss is used for entity recognition task, and InfoNCE loss is used for contrastive learning.

[0038] In one embodiment of the present invention, a low-rank matrix is ​​introduced into the attention layer and MLP layer of the model, and the model parameters are updated by training the low-rank matrix, including:

[0039] Based on the inherent low-rank characteristic of large models, a bypass matrix is ​​added to simulate full parameter fine-tuning. Let the pre-trained model and training parameters be denoted as... After the new fine-tuning task, the parameters become In model fine-tuning, the original parameters are frozen directly, and the incremental difference ΔW is the amount of change to be learned; each linear layer applying LoRA has a corresponding low-rank matrix. and r is the rank of the low-rank matrix, and α is the scaling factor:

[0040]

[0041] The fine-tuning parameters are concentrated in the attention layer and the MLP layer; the default values ​​for the target fine-tuning module are q_proj, k_proj, and v_proj in Attn; and down_proj, up_proj, and gate_proj in MLP; among which, Here, is the output layer weight matrix, and RotaryEmb is the rotation position encoding function, whose formula is shown below:

[0042]

[0043] MLP=down(up(x)×SiLU(gate(x)))

[0044] The q_proj layer of the l-th decoder layer, Where r is the rank of the low-rank matrix, and α is the scaling factor.

[0045] In one embodiment of the present invention, a 4-bit quantization training method is employed, and a joint optimization objective of cross-entropy loss and InfoNCE loss is used, wherein the cross-entropy loss is used for entity recognition tasks, and the InfoNCE loss is used for contrastive learning, including:

[0046] The base loss is selected by calculating the cross-entropy loss function of the target task. Among them, P(w n |w1,w2,…,w n-1 ) indicates that given the preceding text w1, w2, ..., w n-1 Under the given conditions, predict the probability of the current word; then, in conjunction with the contrastive learning task, use InfoNCE loss as the joint optimization objective, where B is the batch size, τ is the temperature parameter, sim(·) is the cosine similarity function, and α = 0.1 is the weight of the contrastive loss; the formula is as follows:

[0047]

[0048]

[0049] When loading model-related components, the PEFT configuration is first loaded from the specified adapter model path. The autoregressive language model is then loaded using AutoModelForCausalLM. Afterwards, the LoRA adapter model is loaded onto the base model using the from_pretrained method, completing the initial integration of the fine-tuned model and the base model. For the q_proj layer of the l-th decoder layer, its effective parameter matrix... Basic model parameter matrix LoRA update volume The sum, the formula is as follows:

[0050]

[0051] The adapter model merging function merges fine-tuned models at specified checkpoints as needed, based on the training status at different checkpoints. The parameter merging function calls the merge_and_unload method to merge the parameters of the LoRA adapter into the base model and unloads the adapter, resulting in a model containing the fine-tuned parameters. The model is then set to evaluation mode, the special layers used during training are turned off, and the merged model and the word segmenter are saved to the specified output path. The maximum segment size is set to 5GB, and the model is saved again.

[0052] In one embodiment of the present invention, the confidence assessment of the identified entities includes:

[0053] Calculate the entity-level confidence score by summing the predicted probabilities of each character in the entity based on the token-level probabilities output by the word segmenter, using position-increasing weights.

[0054] Calculate the entity length confidence score and evaluate it based on the degree of matching between the entity length and the preset length distribution;

[0055] Calculate the entity type confidence score and evaluate it based on the degree of matching between the entity type and the preset type distribution;

[0056] Calculate context confidence based on the frequency of an entity’s occurrence in the context and the diversity of its surrounding words;

[0057] The confidence scores of the four dimensions are dynamically weighted and normalized to form the final entity confidence score, which is used to filter out low-confidence identification results.

[0058] To achieve the above objectives, another aspect of the present invention proposes a few-sample contrast enhancement fine-tuning system based on a large language model, comprising:

[0059] The recruitment text preprocessing module is used to receive the original recruitment text input, and perform cleaning, word segmentation and annotation alignment to generate training samples that conform to the natural language instruction format.

[0060] The contrast enhancement data generation module is used to construct positive and negative sample pairs under the condition of few samples. Positive sample pairs are generated by randomly replacing labeled entities in the same entity category, and negative sample pairs are generated by inserting real labeled entities in non-labeled positions, replacing real labeled positions with entities of non-uniform type, or randomly deleting some labeled entities.

[0061] The LoRA fine-tuning training module is used to load pre-trained large language models, freeze the original parameters, train only low-rank matrices, and train using 4-bit quantization.

[0062] The multidimensional joint confidence filtering module is used to evaluate the confidence of the identified entities after model inference. The confidence evaluation includes entity-level confidence, entity-length confidence, entity-type confidence, and context confidence. The confidence of each dimension is dynamically weighted and normalized to form the final entity confidence score.

[0063] The model reasoning and output module is used to output the identified entity information based on the final entity confidence score. If the score is greater than a preset filtering threshold, the identified entity information is used to construct a job profile.

[0064] The few-sample contrast enhancement fine-tuning method and system based on a large language model in this invention not only solves the problem of data annotation difficulties by enhancing positive and negative sample pairs of a small number of real samples, but also further stimulates the entity recognition task's ability to learn the representation domain of entity boundaries and types. Furthermore, the method of combining low-rank fine-tuning with multi-dimensional joint filtering can fully integrate the advantages of large-model semantic understanding and rule learning.

[0065] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0066] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0067] Figure 1 A flowchart illustrating a few-sample contrast enhancement fine-tuning method based on a large language model, provided for embodiments of this application;

[0068] Figure 2 Flowchart of the LLM-based few-shot contrast enhancement fine-tuning recruitment information extraction model provided in this application embodiment;

[0069] Figure 3 This is a diagram illustrating the architecture of the LLM-based few-shot contrast enhancement fine-tuning recruitment information extraction model provided in this application embodiment.

[0070] Figure 4 These are recruitment information extraction prompts provided in the embodiments of this application;

[0071] Figure 5 This application provides an architecture diagram of a data comparison enhancement instruction fine-tuning module.

[0072] Figure 6 This is a diagram illustrating the architecture of the multidimensional joint confidence filtering module in an embodiment of this application.

[0073] Figure 7 This is a structural diagram of a few-sample contrast enhancement fine-tuning system based on a large language model, provided in an embodiment of this application. Detailed Implementation

[0074] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0075] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0076] The following description, with reference to the accompanying drawings, describes a few-sample contrast enhancement fine-tuning method and system based on a large language model, according to an embodiment of the present invention.

[0077] Figure 1 This is a flowchart of a few-shot contrastive enhancement fine-tuning method based on a large language model according to an embodiment of the present invention, used for named entity recognition in recruitment text, such as... Figure 1 The steps shown are as follows:

[0078] S1, in response to the job posting text input request, preprocesses the original job posting text to generate an input sample that conforms to the natural language instruction format;

[0079] S2, When the number of labeled samples is insufficient, construct positive and negative sample pairs. Positive sample pairs are generated by randomly replacing labeled entities in the same entity category, and negative sample pairs are generated by inserting real labeled entities in non-labeled positions, replacing real labeled positions with entities of non-uniform type, or randomly deleting some labeled entities.

[0080] S3 uses a large language model for fine-tuning, and loads the pre-trained large language model through the LoRA fine-tuning training module.

[0081] S4. During the inference process of the pre-trained large language model, the confidence of the identified entities is evaluated. The confidence evaluation includes entity-level confidence, entity-length confidence, entity-type confidence, and context confidence. The confidence of each dimension is processed by dynamic weighted normalization to obtain the final entity confidence score.

[0082] S5. Compare the final entity confidence score with the preset filtering threshold, and output the entity to the model inference and output module according to the comparison result for use in constructing job profiles.

[0083] Furthermore, the preprocessing of the original recruitment text to generate input samples conforming to natural language instruction formats includes:

[0084] The original recruitment text is separated into structured and unstructured content, retaining the unstructured descriptive parts;

[0085] Convert traditional NER annotation format into natural language format;

[0086] Align entity types to unify different annotation formats into predefined entity types.

[0087] Furthermore, the step of constructing positive and negative sample pairs when the number of labeled samples is insufficient includes:

[0088] In positive sample generation, entities within the same entity category are randomly replaced;

[0089] In negative sample generation, real labeled entities are inserted into unlabeled locations, real labeled entities are replaced with non-uniform type entities, or some labeled entities are randomly deleted.

[0090] Positive and negative sample pairs are used to train the model to identify the categories and boundaries of entities.

[0091] Furthermore, the loading of the pre-trained large language model via the LoRA fine-tuning training module includes:

[0092] Load the pre-trained large language model and freeze the original parameters;

[0093] Low-rank matrices are introduced into the attention layer and MLP layer of the model, and the model parameters are updated by training the low-rank matrices.

[0094] A 4-bit quantization training method is used to reduce memory usage and computational resource consumption during the training process;

[0095] The optimization objective is jointly optimized using cross-entropy loss and InfoNCE loss, where cross-entropy loss is used for entity recognition tasks and InfoNCE loss is used for contrastive learning.

[0096] Furthermore, the confidence assessment of the identified entities includes:

[0097] Calculate the entity-level confidence score by summing the predicted probabilities of each character in the entity based on the token-level probabilities output by the word segmenter, using position-increasing weights.

[0098] Calculate the entity length confidence score and evaluate it based on the degree of matching between the entity length and the preset length distribution;

[0099] Calculate the entity type confidence score and evaluate it based on the degree of matching between the entity type and the preset type distribution;

[0100] Calculate context confidence based on the frequency of an entity’s occurrence in the context and the diversity of its surrounding words;

[0101] The confidence scores of the four dimensions are dynamically weighted and normalized to form the final entity confidence score, which is used to filter out low-confidence identification results.

[0102] Understandably, this invention, based on a pre-trained large language model for general semantic understanding, enhances positive and negative sample pairs from a representation learning perspective when dealing with limited labeled samples. It utilizes a joint loss function of contrastive learning and entity recognition to improve the accuracy of entity recognition boundaries, solving the challenge of extracting information from limited samples and low-quality labeled data in vertical fields such as recruitment. Furthermore, it avoids model overfitting by using multi-dimensional rule-based joint confidence filtering independent of large model training. Ultimately, it achieves high-precision named entity recognition with limited labeling and resources on multiple public datasets and private recruitment job postings. Figure 2 As shown.

[0103] This invention mainly consists of three stages: preprocessing, low-rank fine-tuning with few samples, and pre-trained model inference. Specifically, it can be divided into the following five steps: data preprocessing, large model hinting engineering, data contrast enhancement, LoRA fine-tuning training, and joint confidence filtering. The specific process is as follows... Figure 3 As shown.

[0104] Data preprocessing stage:

[0105] The raw recruitment data processed in the data preprocessing stage is mainly divided into structured text and unstructured descriptions. The former, such as (educational requirement: Bachelor's degree, recruiting company: Company A, gender requirement: male), presents discrete values, while the latter is mainly presented in the form of job responsibilities and job requirements. Therefore, this invention mainly focuses on keyword extraction of information in the unstructured text portion, aiming to build readable job profiles for downstream applications. Annotation prioritizes convenience and accuracy of prompts, combining industry-specific thesaurus and manual verification.

[0106] Few-shot annotation. Taking the internet industry as an example, the proprietary vocabulary is based on proprietary terms from publicly available internet technical reports, e-books, dissertations, journals, and other materials, creating a proprietary vocabulary for industry recruitment. Using automatic word segmentation combined with manual annotation and verification, entity annotation is performed across multiple dimensions, including proper nouns, technical hotspots, and individual comprehensive abilities. For unstructured information, corresponding job profiles are extracted, and entity types are labeled as "SKL" and "PER," signifying knowledge, skills, and other personal characteristics. Example: "text: Familiar with database principles, proficient in using more than one common database product, standard Mandarin. answer: [('database','SKL'),('Mandarin','PER')]". Other processing instructions are as follows: I. Number of training samples. Although existing research only uses zero-shot NER methods with no fine-tuning of prompt words, their prompt words are often designed for proprietary datasets, resulting in limited transferability and robustness. Therefore, this invention incorporates few-shot training, significantly improving domain extensibility and annotation accuracy, and its effectiveness has been verified on multiple Chinese NER datasets. To balance accuracy, diversity, and efficiency in entity labeling, the recruitment dataset is fine-tuned with a small number of samples. The training and test sets are sampled in a 1:1 ratio, with a maximum of 2000 samples. For public datasets with fewer than 300 test samples, a 1:1 to 1:2 ratio is used for sampling. Insufficient labeled samples cannot drive saturated training of a large model. II. Sample Filtering Method. This invention uses the job title or resume's intended job category as the classification basis for each recruitment post. Structured descriptions are removed, empty labeled samples are eliminated, and duplicate labeled entities are verified. The original recruitment data is obtained through web scraping from third-party online recruitment platforms.

[0107] Annotation type alignment. During the validity verification process, in addition to constructing a private dataset for campus recruitment tasks, it is also necessary to verify the validity of this invention on existing public datasets. Therefore, data preprocessing of the public datasets is required. In the existing NER public datasets, annotation formats such as IOB annotation, BIO annotation, CoNLL-2003 format, and JSON format are mainly used. This format is not applicable to the streaming question and answer of generative large models. The tab stops, line breaks, and escape characters contained in it are highly redundant, affecting the context understanding of the large model. Therefore, it is necessary to change it to a natural language format with coherent semantics for instruction fine-tuning. Taking the following sentence as an example, this sentence is from the Resume NER Chinese dataset: "大 / tB-EDU / n学 / t E-EDU / n毕 / t O / n业 / t O / n, / t O / n高 / t B-TITLE / n级 / t M-TITLE / n经 / t M-TITLE / n济 / t M-TITLE / n师 / t E-TITLE / n。 / t O", which should be adjusted to the format of "text: 大学毕业,高级经济师。answer: [('大学','EDU'),('高级经济师','TITLE')]". [[ID= 1]] [[ID= 2]]

[0108] [[ID= 3]]Prompt engineering stage: [[ID= 4]] [[ID= 5]]

[0109] [[ID= 6]]In this invention, the instruction tuning method is mainly used, that is, the model is trained through data in the form of natural language conversations. Factors such as instruction description, quantity, and semantics will directly affect the training effect of the model. The instruction contains: the definition of the question and answer subject, the task type, task question and answer examples, and additional supplementary information. Extracting job recruitment information is classified as a sequence annotation task in traditional NLP research, which is not consistent with the underlying idea of generative large models. Therefore, if overly long and personalized descriptions are given in the instruction, it will reduce the accuracy of the model, and the answers tend to be diverse. [[ID= 7]] [[ID= 8]]

[0110] [[ID= 9]]As [[ID= 10]] Figure 4 [[ID= 11]]shown in the prompt words, although very detailed, the actual training effect is not ideal. Its extraction results tend to be long sentence descriptions rather than keyword information, such as: "('项目研发需求分析 / 原型设计','SKL'),('项目开发风险点','SKL')". Therefore, in this invention, for entity recognition tasks, a method of streamlining instructions is adopted, eliminating large-scale descriptions of the question and answer topic and entity type, and merging them into short sentence descriptions that clarify the task focus. This can, to a certain extent, strengthen the accuracy of the output instructions in a fixed format and avoid low test effects due to the diversity of the generation dimensions of large models. And in the public datasets for multi-type entity recognition, additional prompt word texts are added to explain the detailed Chinese interpretations of each annotation prompt type. [[ID= 12]] [[ID= 13]]

[0111] [[ID= 14]]Data contrast enhancement stage:

[0112] To improve the training performance on few-shot datasets and enhance the boundary and category recognition capabilities of general skill entities, this invention proposes a contrastive learning-based data augmentation strategy, such as... Figure 5 As shown. Specifically, for a given text, the original input X = {x1, x2, ..., x...} is divided into... n The entity l = {l1, l2, ..., l} is labeled with the following structure: m First, construct a real sample set L from all entities l of the same category in the same dataset. type ={L1,L2,…,L i When generating positive samples, it is only necessary to randomly replace the entity l1 at the labeled position with l2, where l1, l2 ∈ L. type This involves performing random replacements within a lexicon with identical annotations. The replacement probability for each annotation in each sample is set to p, and (X, L2) is defined as an enhanced version of (X, L1). These two pairs form a positive sample. Similarly, in the negative sample generation process, we randomly replace unlabeled positions with real annotations and randomly replace non-uniform type entities l1 with l2 at the original real annotation positions, where l1 ∈ L... type1 ,l2∈L type2 Alternatively, a certain number of labeled samples can be randomly removed from the real sample l and used to form negative sample pairs. A diagram illustrating data enhancement is shown below. Figure 4 As shown in the upper left part.

[0113] Based on the original input sample pairs, operations such as word segmentation and label setting prepare a suitable data format for subsequent model training. The text is then embedded into a uniform-length multidimensional embedding using a large-scale model word segmenter. Let the input text be x. input The target text is x target The complete prompt text is x full =x input +x target The `tokenize` function is called to perform tokenization on the complete prompt text. The resulting sequence of input IDs for the tokenized complete prompt text is `input_ids`. full =[id full,1 ,id full,2 ,…,id full,k To better locate instruction examples, and to exclude contextual parts that do not need to be predicted, the adjusted label sequence is... adjusted :

[0114] labels adjusted = [-n]×L input +labels full [L input :]

[0115] Where n = 100 represents the instruction length of the context portion that does not require prediction.

[0116] LoRA fine-tuning training phase:

[0117] Fine-tuning is a training method developed since the rapid rise of large language models to adapt them to specific downstream tasks. Specifically, commercial L0-level pre-trained large language models have a massive number of parameters. Leveraging the rich language representations learned on large-scale data, they bring unprecedented semantic understanding and text generation capabilities to natural language processing tasks. Fine-tuning techniques aim to specifically adjust general-purpose large models using domain-specific data. For example, Qwen2 uses the same tokenizer based on byte-level byte-pair encoding, and its vocabulary contains 151,643 regular lexical units and 3 control lexical units. Considering training cost, compliance, and model performance, the 7B model is of moderate size, possesses strong domain adaptability and basic performance, and can effectively complete the entity extraction fine-tuning task in the campus recruitment domain of this invention. The architecture diagram of the data comparison enhancement instruction fine-tuning module is shown below. Figure 5 As shown in the yellow area on the right.

[0118] This invention employs Low-Rank Adaptation (LoRA) technology, which adapts to new tasks by learning the parameters of a low-rank matrix without altering the original model weights. This significantly reduces the number of trainable parameters and saves on model training costs. Furthermore, LoRA adds virtually no inference latency because the adapter weights can be merged with the base model. Based on the inherent low-rank characteristic of large models, a bypass matrix is ​​added to simulate full-parameter fine-tuning. Let the pre-trained model and training parameters be denoted as... After the new fine-tuning task, the parameters become In model fine-tuning, the original parameters are frozen directly, and the incremental difference ΔW is the amount of change to be learned. Each linear layer applying LoRA has a corresponding low-rank matrix. and r is the rank of the low-rank matrix, and α is the scaling factor.

[0119]

[0120] The fine-tuning parameters are mainly concentrated in the attention layer and the MLP layer. The default values ​​for the target fine-tuning module are q_proj, k_proj, and v_proj in Attn; and down_proj, up_proj, and gate_proj in MLP. Here, is the output layer weight matrix, and RotaryEmb is the rotation position encoding function, whose formula is shown below:

[0121]

[0122] MLP=down(up(x)×SiLU(gate(x)))

[0123] Taking the q_proj layer of the l-th decoder layer as an example, Where r is the rank of the low-rank matrix, typically 8 or 16. After testing, the default value r = 8 was set. Similarly, other layers also have corresponding values. And so on. Meanwhile, α is used as a scaling factor, with a default value of α = 32.

[0124] 4-bit quantization is configured for low-precision training, further reducing memory usage and accelerating computation. For training and evaluation, the base loss is selected using the cross-entropy loss function calculated from the target task. Among them, P(w n |w1,w2,…,w n-1 ) indicates that given the preceding text w1, w2, ..., w n-1 Under the given conditions, predict the probability of the current word. Furthermore, for the contrastive learning task, the InfoNCE loss is used as the joint optimization objective, where B is the batch size, τ is the temperature parameter, sim(·) is the cosine similarity function, and α = 0.1 is the weight of the contrastive loss. The formula is as follows:

[0125]

[0126] The PEFT library is used to efficiently fine-tune parameters. When loading model-related components, the PEFT configuration, which contains information about the fine-tuned model, is first loaded from the specified adapter model path. The autoregressive language model is loaded using AutoModelForCausalLM, and then the LoRA adapter model is loaded onto the base model using the from_pretrained method, completing the initial integration of the fine-tuned model and the base model. Taking the q_proj layer of the l-th decoder layer as an example, its effective parameter matrix... Basic model parameter matrix LoRA update volume The sum, the formula is as follows:

[0127]

[0128] The adapter model merging function can merge fine-tuned models at specified checkpoints as needed, based on the training status at different checkpoints. Parameter merging is implemented by calling the `merge_and_unload` method to merge the parameters of the LoRA adapter into the base model and unload the adapter, resulting in a complete model containing the fine-tuned parameters. The model is then set to evaluation mode, and special layers used during training are disabled to ensure the model remains stable for subsequent use. Finally, the merged model and the word segmenter are saved to the specified output path, with a maximum fragment size of 5GB, to meet different storage and usage requirements. The LoRA adapter is approximately 400MB in size, and the merged model is approximately 14GB in size, demonstrating a storage compression ratio of less than 2.9%, significantly saving storage resources.

[0129] Multidimensional joint confidence filtering stage:

[0130] Multi-dimensional joint confidence filtering aims to perform secondary verification on the extraction model fine-tuned with few samples based on multi-dimensional rules. Inspired by traditional rule-based methods, it adds entity filtering after the output of the large model, effectively improving the accuracy of entity recognition. As mentioned in the previous section on large model prompts and various fine-tuning methods, the compliance of the fine-tuned large model with instructions is not consistently high, and its output may deviate from the prescribed format to a certain extent. This stems from the strong semantic understanding ability of the large model itself and the emphasis on answer richness during pre-training. Richness is an important evaluation metric for the output of the pre-trained large model, and therefore, in principle, it cannot be completely solved by fine-tuning. While traditional rule-based algorithms have drawbacks such as cumbersome rule writing and large dictionary requirements, multi-dimensional joint filtering can be added after fine-tuning, and the combined weights can be dynamically updated by combining context frequency and historical statistical information during multiple rounds of confidence calculation. This module considers multi-dimensional information more comprehensively and can adaptively adjust weights and filtering thresholds. The multi-dimensional joint confidence filtering module is as follows: Figure 6 As shown.

[0131] Specifically, multidimensional confidence calculation is divided into four dimensions: entity-level confidence, entity-length confidence, entity-type confidence, and contextual confidence.

[0132] Each component of the entity-level confidence score originates from the token-level confidence score output by the token segmenter of the pre-trained large model, based on the BPE (Byte-Pair Encoding) algorithm. The token segmenter converts the token IDs output by the model back into text, then determines whether they are valid tokens, returning the log odds for each step in the generation process for all valid tokens. Assuming the vocabulary size is V, at step t, the log odds vector output by the model is z. t =[zt1 ,z t2 ,…,z tV ], where z ti This represents the score for predicting the i-th word in the vocabulary at step t. Next, to obtain the prediction probability for each token, the log odds are processed using the Softmax function. For the log odds vector z at step t... t After processing by the Softmax function, the resulting probability vector p t =[p t1 ,p t2 ,…,p tV ], where p ti This represents the probability of predicting the i-th word in the vocabulary at step t, calculated using the following formula:

[0133]

[0134] After obtaining the probability of each token in the vocabulary, the weight information of the entire entity is combined using position-increasing weights. Here, pi is the predicted probability of the i-th character in the entity, n is the length of the entity, and 1 + 0.1 × i is the position weight, with subsequent characters having higher weights (e.g., the first character has a weight of 1.0, the second has a weight of 1.1, and so on).

[0135]

[0136] Entity length and entity type confidence depend on prior knowledge of the selected dataset. For example, the MSRA dataset has three categories: People (PER), Locations (LOC), and Organizations (ORG), and entity lengths are typically 2-4 characters. The initialization of these two confidence scores can be weighted according to data characteristics and experience. The following shows the weighted entity length and category settings for the MSRA dataset:

[0137]

[0138] Context-based confidence is assessed by calculating the frequency of an entity's occurrences within its context, combined with the diversity of surrounding words. This is to prevent duplicate entities from being counted as two separate labels in some datasets, where they appear in different contexts. Specifically, `entity_freq` is the number of times the entity appears in the context, `context_word_diversity` is the number of different words within the window surrounding the entity, and `window_size = 2` (a window containing two words before and after the entity).

[0139]

[0140] The four dimensions above are dynamically weighted and normalized to form the final entity confidence score, which is the multidimensional joint confidence score. Initial weights are assigned to each dimension: weights = {token: 0.7, length: 0.1, type: 0.1, context: 0.1}. Then, the weights are adjusted based on historical statistical data, primarily targeting the number of correct predictions and the total number of predictions for each entity type and length. This updates the weight values ​​for these two dimensions to better learn the labeled distribution of the dataset and the recognition performance of the large model fine-tuning. Statistical data is updated after each prediction.

[0141] weights[type]←weights[type]×(1+type accuracy )

[0142]

[0143] weights[length]←weights[length]×(1+length accuracy )

[0144]

[0145] The updated weights are normalized again to ensure a sum of 1, and finally the entity multidimensional joint confidence score is calculated. Finally, the default filtering threshold for this module is set to 0.8.

[0146]

[0147] ensemble_confidence

[0148] =weights[entity]×entity_conf+weights[length]

[0149] ×length_conf+weights[type]×type_conf+weights[context]

[0150] ×context_conf

[0151] The multi-dimensional joint confidence score filtering module comprehensively considers character-level prediction and concatenation, entity structure, and contextual information, avoiding the limitations of a single dimension. It adjusts weights based on historical performance, enabling the model to adapt to the recognition difficulty of entities of different types and lengths. New confidence dimensions (such as entity location and part-of-speech tagging) can be easily added. The weight table and calculation formula can be adjusted according to specific domain requirements, demonstrating good scalability. Extensive experiments on multiple datasets have shown that applying this module, based on fine-tuning of a large model, significantly improves recognition quality.

[0152] The few-shot contrast enhancement fine-tuning method based on a large language model according to embodiments of the present invention enhances the model's ability to recognize entity boundaries by constructing positive and negative sample pairs, reduces training costs by combining low-rank fine-tuning (LoRA) technology, and introduces a multi-dimensional joint confidence filtering mechanism to improve the accuracy and robustness of entity recognition. This achieves high-precision, low-resource, and strong generalization recognition results in recruitment text information extraction tasks. It effectively addresses the challenges of entity recognition tasks in the recruitment field and is applicable to various application scenarios such as university career guidance, corporate recruitment platforms, and talent database construction.

[0153] To achieve the above embodiments, such as Figure 7 As shown, this embodiment also provides a few-shot contrast enhancement fine-tuning system 10 based on a large language model, including:

[0154] The recruitment text preprocessing module 100 is used to receive the original recruitment text input, and perform cleaning, word segmentation and annotation alignment to generate training samples that conform to the natural language instruction format.

[0155] The contrast enhancement data generation module 200 is used to construct positive and negative sample pairs under the condition of few samples. Positive sample pairs are generated by randomly replacing labeled entities in the same entity category, and negative sample pairs are generated by inserting real labeled entities in non-labeled positions, replacing real labeled positions with entities of non-uniform type, or randomly deleting some labeled entities.

[0156] LoRA fine-tuning training module 300 is used to load pre-trained large language models and freeze the original parameters, train only low-rank matrices, and train using 4-bit quantization.

[0157] The multidimensional joint confidence filtering module 400 is used to evaluate the confidence of the identified entities after model inference. The confidence evaluation includes entity-level confidence, entity-length confidence, entity-type confidence, and context confidence. The confidence of each dimension is dynamically weighted and normalized to form the final entity confidence score.

[0158] The model reasoning and output module 500 is used to output the identified entity information based on the final entity confidence score, if it is greater than a preset filtering threshold, for constructing a job profile.

[0159] Furthermore, the recruitment text preprocessing module 100 further includes:

[0160] The structured content removal unit is used to identify and remove structured fields from recruitment texts;

[0161] The annotation alignment unit is used to convert traditional NER annotation format into natural language format and to perform unified mapping of entity types.

[0162] Furthermore, the contrast enhancement data generation module 200 further includes:

[0163] The positive sample generation unit is used to randomly replace labeled entities in the same entity category to generate enhanced positive sample pairs;

[0164] The negative sample generation unit is used to insert real labeled entities in non-labeled locations, replace real labeled entities with non-uniform type entities in real labeled locations, or randomly delete some labeled entities to generate enhanced negative sample pairs.

[0165] Furthermore, the LoRA fine-tuning training module 300 further includes:

[0166] The low-rank matrix training unit is used to introduce low-rank matrices into the attention layer and MLP layer of the model, and to achieve efficient updating of model parameters by training the low-rank matrix.

[0167] The quantization training unit is used to employ 4-bit quantization training to reduce memory usage and computational resource consumption during the training process.

[0168] The joint loss optimization unit is used to combine cross-entropy loss and InfoNCE loss for model training, thereby improving the model's ability to recognize entity boundaries.

[0169] Furthermore, the multidimensional joint confidence filtering module 400 further includes:

[0170] The entity-level confidence calculation unit is used to perform a weighted summation of the predicted probabilities of each character in the entity based on the token-level probabilities output by the word segmenter, using position-increasing weights.

[0171] The entity length confidence calculation unit is used to evaluate the degree of matching between the entity length and the preset length distribution.

[0172] The entity type confidence calculation unit is used to evaluate the degree of matching between the entity type and the preset type distribution.

[0173] The context confidence calculation unit is used to evaluate an entity based on its frequency of occurrence in the context and the diversity of its surrounding words.

[0174] The dynamic weighted normalization unit is used to process the confidence scores of the four dimensions through dynamic weighted normalization to form the final entity confidence score, which is used to filter out low confidence recognition results.

[0175] The few-shot contrast enhancement fine-tuning system based on a large language model according to embodiments of the present invention enhances the model's ability to recognize entity boundaries by constructing positive and negative sample pairs, reduces training costs by combining low-rank fine-tuning (LoRA) technology, and introduces a multi-dimensional joint confidence filtering mechanism to improve the accuracy and robustness of entity recognition. This achieves high-precision, low-resource, and strong generalization recognition results in recruitment text information extraction tasks. It effectively addresses the challenges of entity recognition tasks in the recruitment field and is suitable for various application scenarios such as university career guidance, corporate recruitment platforms, and talent database construction.

[0176] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0177] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A few-shot contrastive enhancement fine-tuning method based on a large language model for named entity recognition in recruitment texts, characterized in that, Includes the following steps: In response to a job posting text input request, the original job posting text is preprocessed to generate an input sample that conforms to the natural language instruction format; When the number of labeled samples is insufficient, positive and negative sample pairs are constructed. Positive sample pairs are generated by randomly replacing labeled entities in the same entity category, while negative sample pairs are generated by inserting real labeled entities in non-labeled positions, replacing real labeled positions with entities of non-uniform type, or randomly deleting some labeled entities. A large language model is used for fine-tuning, and a pre-trained large language model is loaded through the LoRA fine-tuning training module. The loading of the pre-trained large language model via the LoRA fine-tuning training module includes: loading the pre-trained large language model and freezing the original parameters; introducing low-rank matrices into the model's attention layer and MLP layer, and updating the model parameters by training the low-rank matrices; employing 4-bit quantization training and using a joint optimization objective of cross-entropy loss and InfoNCE loss, where cross-entropy loss is used for entity recognition tasks and InfoNCE loss is used for contrastive learning; introducing low-rank matrices into the model's attention layer and MLP layer, and updating the model parameters by training the low-rank matrices, includes: based on the inherent low-rank characteristics of the large model, adding a side-channel matrix to simulate full parameter fine-tuning, denoted as the pre-trained model and training parameters as... After the new fine-tuning task, the parameters become In model fine-tuning, the original parameters are frozen directly, and the incremental difference is... This represents the amount of change that needs to be learned; each linear layer applying LoRA has a corresponding low-rank matrix. and , It is the rank of a low-rank matrix. It is the scaling factor: The fine-tuning parameters are concentrated in the attention layer and the MLP layer; the default value of the target fine-tuning module is... middle , , ;as well as middle , , ;in, It is the output layer weight matrix. This is the rotation position encoding function, and its formula is shown below: No. Decoder layer layer, , ,in It is the rank of a low-rank matrix. As a scaling factor; when loading model-related components, first load the PEFT configuration from the specified adapter model path, use AutoModelForCausalLM to load the autoregressive language model, and then use the from_pretrained method to load the LoRA adapter model onto the base model, completing the initial combination of the fine-tuned model and the base model; for the first... Decoder layer Layer, its effective parameter matrix Basic model parameter matrix LoRA update volume The sum, the formula is as follows: The adapter model merging process merges fine-tuned models at specified checkpoints as needed, based on the training status at different checkpoints. Parameter merging calls the `merge_and_unload` method to merge the parameters of the LoRA adapter into the base model and unload the adapter, resulting in a model containing the fine-tuned parameters. The model is then set to evaluation mode, special layers used during training are disabled, and the merged model and tokenizer are saved to the specified output path with a maximum fragment size of 5GB. The model is saved again. During model inference, confidence is evaluated for identified entities. This confidence evaluation includes entity-level confidence, entity-length confidence, entity-type confidence, and context confidence. The confidence scores for each dimension are obtained through dynamic weighted normalization. The final entity confidence score is compared with a preset filtering threshold. Based on the comparison result, the entity is output to the model inference and output module for constructing a job profile.

2. The method according to claim 1, characterized in that, The preprocessing of the original recruitment text to generate input samples conforming to natural language instruction format includes: The original recruitment text is separated into structured and unstructured content, retaining the unstructured descriptive parts; Convert traditional NER annotation format into natural language format; Align entity types to unify different annotation formats into predefined entity types.

3. The method according to claim 2, characterized in that, The process of converting traditional NER annotation format into natural language format includes: Based on internet industry technology reports, e-books, and papers, a specialized vocabulary for industry recruitment was constructed, and the original recruitment text data was used as the corpus input for entity recognition and job profile construction. The NER model, based on a proprietary lexicon and fine-tuned with few samples, performs entity recognition on the unstructured original recruitment text data, identifies entities of knowledge, skills and personal characteristics, and performs data cleaning and sample filtering preprocessing according to job categories. Based on the preprocessing results, structured labeled data and high-quality job profiles are output.

4. The method according to claim 1, characterized in that, When the number of labeled samples is insufficient, constructing positive and negative sample pairs includes: In positive sample generation, entities within the same entity category are randomly replaced; In negative sample generation, real labeled entities are inserted into unlabeled locations, real labeled entities are replaced with non-uniform type entities, or some labeled entities are randomly deleted. Positive and negative sample pairs are used to train the model to identify the categories and boundaries of entities.

5. The method according to claim 4, characterized in that, In positive sample generation, entities within the same entity category are randomly replaced, including: Divide the text into raw input. and labeled entities ; All entities of the same category in the same dataset Construct a real sample set ; When generating positive samples, randomly replace the entities at the labeled locations. for in This means performing random replacements within a lexicon with the same annotations, setting the replacement probability for each annotation in each sample to be [value missing]. ,set up for An enhanced version of that, together the two form a pair of positive samples.

6. The method according to claim 4, characterized in that, In negative sample generation, methods include inserting real labeled entities into unlabeled locations, replacing real labeled locations with entities of non-uniform types, or randomly deleting some labeled entities, including: During the negative sample generation process, unlabeled positions are randomly replaced with real labels, and entities of non-uniform types are randomly replaced in the original real label positions. for ,in Or randomly delete real samples The number of labeled items in the sample is used to form negative sample pairs.

7. The method according to claim 1, characterized in that, A 4-bit quantization training method is employed, and the objective is jointly optimized using cross-entropy loss and InfoNCE loss. Cross-entropy loss is used for entity recognition, and InfoNCE loss is used for contrastive learning, including: The base loss is selected by calculating the cross-entropy loss function of the target task. ,in, Indicates that in a given Under the given conditions, predict the probability of the current word; then, in combination with the comparative learning task, use InfoNCE loss as the joint optimization objective, where B is the batch size, τ is the temperature parameter, and sim ( ) is the cosine similarity function. These are the weights for the contrastive loss; the formula is as follows: ) 。 8. The method according to claim 1, characterized in that, The confidence assessment of the identified entities includes: Calculate the entity-level confidence score by summing the predicted probabilities of each character in the entity based on the token-level probabilities output by the word segmenter, using position-increasing weights. Calculate the entity length confidence score and evaluate it based on the degree of matching between the entity length and the preset length distribution; Calculate the entity type confidence score and evaluate it based on the degree of matching between the entity type and the preset type distribution; Calculate context confidence based on the frequency of an entity’s occurrence in the context and the diversity of its surrounding words; The confidence scores of the four dimensions are dynamically weighted and normalized to form the final entity confidence score, which is used to filter out low-confidence identification results.

9. A few-shot contrast enhancement fine-tuning system based on a large language model, characterized in that, include: The recruitment text preprocessing module is used to receive the original recruitment text input, and perform cleaning, word segmentation and annotation alignment to generate training samples that conform to the natural language instruction format. The contrast enhancement data generation module is used to construct positive and negative sample pairs under the condition of few samples. Positive sample pairs are generated by randomly replacing labeled entities in the same entity category, and negative sample pairs are generated by inserting real labeled entities in non-labeled positions, replacing real labeled positions with entities of non-uniform type, or randomly deleting some labeled entities. The LoRA fine-tuning training module is used to load pre-trained large language models, freeze the original parameters, train only low-rank matrices, and train using 4-bit quantization. The LoRA fine-tuning training module is also used for: loading pre-trained large language models, including: loading the pre-trained large language model and freezing the original parameters; introducing low-rank matrices into the model's attention layer and MLP layer, and updating model parameters by training the low-rank matrices; employing 4-bit quantization training and using a joint optimization objective of cross-entropy loss and InfoNCE loss, where cross-entropy loss is used for entity recognition tasks and InfoNCE loss is used for contrastive learning; introducing low-rank matrices into the model's attention layer and MLP layer, and updating model parameters by training the low-rank matrices, including: adding a side-channel matrix to simulate full parameter fine-tuning based on the inherent low-rank characteristics of the large model, denoted as [pre-trained model and training parameters are not specified in the original text]. After the new fine-tuning task, the parameters become In model fine-tuning, the original parameters are frozen directly, and the incremental difference is... This represents the amount of change that needs to be learned; each linear layer applying LoRA has a corresponding low-rank matrix. and , It is the rank of a low-rank matrix. It is the scaling factor: The fine-tuning parameters are concentrated in the attention layer and the MLP layer; the default value of the target fine-tuning module is... middle , , ;as well as middle , , ;in, It is the output layer weight matrix. This is the rotation position encoding function, and its formula is shown below: No. Decoder layer layer, , ,in It is the rank of a low-rank matrix. As a scaling factor; when loading model-related components, first load the PEFT configuration from the specified adapter model path, use AutoModelForCausalLM to load the autoregressive language model, and then use the from_pretrained method to load the LoRA adapter model onto the base model, completing the initial combination of the fine-tuned model and the base model; for the first... Decoder layer Layer, its effective parameter matrix Basic model parameter matrix LoRA update volume The sum, the formula is as follows: The adapter model merging function merges fine-tuned models at specified checkpoints as needed, based on the training status at different checkpoints. The parameter merging function calls the merge_and_unload method to merge the parameters of the LoRA adapter into the base model and unloads the adapter, resulting in a model containing the fine-tuned parameters. The model is then set to evaluation mode, the special layers used during training are disabled, and the merged model and the word segmenter are saved to the specified output path with a maximum fragment size of 5GB. The model is then saved again. The multidimensional joint confidence filtering module is used to evaluate the confidence of the identified entities after model inference. The confidence evaluation includes entity-level confidence, entity-length confidence, entity-type confidence, and context confidence. The confidence of each dimension is dynamically weighted and normalized to form the final entity confidence score. The model reasoning and output module is used to output the identified entity information based on the final entity confidence score. If the score is greater than a preset filtering threshold, the identified entity information is used to construct a job profile.

10. The system according to claim 9, characterized in that, The recruitment text preprocessing module further includes: The structured content removal unit is used to identify and remove structured fields from recruitment texts; The annotation alignment unit is used to convert traditional NER annotation format into natural language format and to perform unified mapping of entity types.

11. The system according to claim 9, characterized in that, The contrast enhancement data generation module further includes: The positive sample generation unit is used to randomly replace labeled entities in the same entity category to generate enhanced positive sample pairs; The negative sample generation unit is used to insert real labeled entities in non-labeled locations, replace real labeled entities with non-uniform type entities in real labeled locations, or randomly delete some labeled entities to generate enhanced negative sample pairs.

12. The system according to claim 9, characterized in that, The LoRA fine-tuning training module further includes: The low-rank matrix training unit is used to introduce low-rank matrices into the attention layer and MLP layer of the model, and to achieve efficient updating of model parameters by training the low-rank matrix. The quantization training unit is used to employ 4-bit quantization training to reduce memory usage and computational resource consumption during the training process. The joint loss optimization unit is used to combine cross-entropy loss and InfoNCE loss for model training, thereby improving the model's ability to recognize entity boundaries.

13. The system according to claim 9, characterized in that, The multidimensional joint confidence filtering module further includes: The entity-level confidence calculation unit is used to perform a weighted summation of the predicted probabilities of each character in the entity based on the token-level probabilities output by the word segmenter, using position-increasing weights. The entity length confidence calculation unit is used to evaluate the degree of matching between the entity length and the preset length distribution. The entity type confidence calculation unit is used to evaluate the degree of matching between the entity type and the preset type distribution. The context confidence calculation unit is used to evaluate an entity based on its frequency of occurrence in the context and the diversity of its surrounding words. The dynamic weighted normalization unit is used to process the confidence scores of the four dimensions through dynamic weighted normalization to form the final entity confidence score, which is used to filter out low confidence recognition results.

Images