A named entity recognition method and device and model training method and device thereof

Abstract

This application discloses a named entity recognition method and apparatus for model training. The model training method includes: establishing a multi-source heterogeneous training dataset, entity coverage vectors, and association relationships between identifiers in the source datasets; loading and freezing all parameters of a pre-trained large language model as a base model; constructing a shared low-rank adapter and an entity-specific low-rank adapter; training the base model based on the association relationships and the shared low-rank adapter to obtain a shared adapter; training the base model based on the association relationships and the entity-specific low-rank adapter to obtain an entity-specific adapter for each entity type; and saving the trained adapters and the base model to obtain a pre-trained named entity recognition model. Using this application, with a small number of parameters, the negative transfer problem caused by joint training with multi-source heterogeneous data is solved, improving the recognition accuracy for low-resource entity types and enabling flexible customization and incremental updates of model deployment.

Description

Technical Field

[0001] This application relates to the field of machine learning technology, and in particular to a named entity recognition method and apparatus for training its model. Background Technology

[0002] Named entity recognition (NER) is a key technology in natural language processing, aiming to automatically identify and classify meaningful entity units from unstructured text. In professional fields, especially in healthcare and biomedicine, this technology is the cornerstone for building domain knowledge graphs and realizing intelligent information retrieval and decision support.

[0003] In practical applications, training a robust and accurate named entity recognition model requires utilizing labeled datasets from multiple institutions and for different purposes. These datasets naturally differ in annotation standards, entity type systems, and text styles, constituting a typical scenario of multi-source heterogeneous data. Developers hope to collaboratively leverage this data to improve the model's generalization ability and the comprehensiveness of entity coverage.

[0004] In related technologies, to address model training in scenarios with multiple data sources, the goal is to employ multi-task learning, efficient parameter fine-tuning, and cue-based learning.

[0005] However, multi-task learning, which trains different datasets as different tasks, struggles to effectively handle differences at the entity type level. Efficient parameter fine-tuning fails to adequately consider the inherent differences between datasets and entity types. When faced with multi-source data with vastly different annotation standards and entity distributions, its adaptation process is prone to confusion, making it difficult to achieve optimal performance on all datasets simultaneously and failing to fundamentally solve the negative transfer problem. Cue learning guides the model by designing cue templates, but its effectiveness is limited in complex medical entity recognition tasks. Summary of the Invention

[0006] This application provides a named entity recognition method and apparatus for model training. To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or describe the scope of protection of these embodiments. Its sole purpose is to present some concepts in a simple form as a prelude to the detailed description that follows.

[0007] In a first aspect, embodiments of this application provide a method for training a named entity recognition model, the method comprising: Establish the association between the multi-source heterogeneous training dataset, entity coverage vector, and source dataset identifier; the entity coverage vector is used to represent the entity types contained in the multi-source heterogeneous training dataset. Load and freeze all parameters of the pre-trained large language model as the base model; Construct a shared low-rank adapter and an entity-specific low-rank adapter; the shared low-rank adapter is used to capture general knowledge across datasets and entity types, while the entity-specific low-rank adapter is used to capture recognition patterns for specific entity types. Based on the association relationship and the shared low-rank adapter, the adapter is trained in combination with the base model to obtain the shared adapter; Based on the association relationship and the entity-specific low-rank adapter, the adapter is trained in combination with the base model to obtain the entity-specific adapter for each entity type. Save the shared adapter, the entity-specific adapter for each entity type, and the base model to obtain a pre-trained named entity recognition model.

[0008] Optionally, establish the association between multi-source heterogeneous training datasets, entity overlay vectors, and source dataset identifiers, including: Obtain multiple heterogeneous training datasets from multiple sources; Scan and count the entity type indicators contained in each multi-source heterogeneous training dataset to obtain the entity coverage vector of each multi-source heterogeneous training dataset; Obtain the source dataset identifier for each multi-source heterogeneous training dataset; For each multi-source heterogeneous training dataset, associate its corresponding entity coverage vector with the source dataset identifier, as the association relationship between the multi-source heterogeneous training dataset, entity coverage vector, and source dataset identifier.

[0009] Optionally, based on the association relationship and the shared low-rank adapter, and combined with the base model, an adapter is trained to obtain a shared adapter, including: Inject a shared low-rank adapter into the preset first key projection layer of the base model; The parameters of the injected shared low-rank adapter are instantiated into trainable first low-rank matrices and second low-rank matrices to obtain the first initialization model; The current training sample is retrieved from multiple heterogeneous training datasets from multiple sources. Input the current training sample into the first initialized model to calculate the weighted loss of the current training sample; The parameters of the first low-rank matrix and the second low-rank matrix are updated using a weighted loss. Continue executing the step of traversing the current training samples from multiple multi-source heterogeneous training datasets until the training count reaches a preset number of rounds, resulting in a shared adapter that encodes general entity recognition knowledge across all datasets.

[0010] Optionally, calculate the weighted loss for the current training samples, including: Get the current source dataset identifier of the current training sample; Retrieve the current entity coverage vector corresponding to the current source dataset identifier from the association relationships; Calculate the dynamic scalar weights of the current training sample based on the current entity coverage vector; Perform machine learning forward propagation on the current training samples to quantify the original cross-entropy loss; The weighted loss of the current training sample is generated based on dynamic scalar weights, cross-entropy loss, and a preset entity perception mask loss function.

[0011] Optionally, the calculation expression for the dynamic scalar weights is as follows:

[0012] in, It is the first Dynamic scalar weights for each training sample These are the basic weight hyperparameters. It is 0.3. This represents the total number of global entity types. Used to identify datasets Does it include entity type? , =1 indicates that the dataset contains entity types. , =0 indicates that the dataset does not contain entity types. , This refers to the number of unique entity types that appear in all datasets; The default expression for the entity-aware masking loss function is:

[0013] in, For weighted loss, It is the original cross-entropy loss, used to measure the model's performance on the dataset. The difference between the predicted results and the actual labels.

[0014] Optionally, based on the association relationship and the entity-specific low-rank adapter, adapter training is performed in conjunction with the base model to obtain an entity-specific adapter for each entity type, including: To freeze the shared adapter, the gradient computation of the shared adapter is disabled, based on freezing all parameters of the pre-trained large language model. Determine the target entity type; the target entity type is each preset entity type; From the association relationships, filter out the multi-source heterogeneous training datasets that indicate the existence of target entity types in the entity coverage vectors, so as to retain the multi-source heterogeneous training datasets containing target entity types as a subset of training data dedicated to target entity types; In the pre-defined second key projection layer of the base model, inject an entity-specific low-rank adapter; The injected entity-specific low-rank adapter is instantiated into trainable third and fourth low-rank matrices to obtain the second initialization model. The second initialization model is trained based on a subset of training data specific to the target entity type, resulting in an entity-specific adapter for each entity type.

[0015] Optionally, the second initialization model is trained based on a subset of training data specific to the target entity type, including: Input a subset of training data specific to the target entity type into the second model to perform forward propagation in machine learning on the subset of training data and calculate the original second cross-entropy loss. Stop iterating when the second cross-entropy loss reaches its minimum; or, If the second cross-entropy loss has not reached its minimum, update the parameters of the third and fourth low-rank matrices, and continue to execute the step of inputting a subset of training data specific to the target entity type into the second model until the second cross-entropy loss reaches its minimum, at which point the iteration stops.

[0016] Secondly, embodiments of this application provide a named entity recognition method, the method comprising: Load a pre-trained named entity recognition model, which includes a shared adapter, entity-specific adapters for each entity type, and a base model. The base model is obtained by freezing all parameters of the pre-trained large language model. The shared adapter and entity-specific adapter are trained using the aforementioned named entity recognition model training method. Receive a named entity recognition request, which carries the actual entity type and the text to be recognized for the current inference task. From the entity-specific adapters for each entity type, select the target entity-specific adapter that matches each actual entity type of the current inference task; The parameters of the base model, each target entity-specific adapter, and the shared adapter are combined to generate the final target model used for inference. The text to be recognized is input into the target model for forward propagation calculation, and the output is each identified entity fragment in the text and its corresponding entity type label.

[0017] Thirdly, embodiments of this application provide a named entity recognition model training apparatus, the apparatus comprising: The association establishment module is used to establish the association between multi-source heterogeneous training datasets, entity coverage vectors, and source dataset identifiers; entity coverage vectors are used to represent the types of entities contained in multi-source heterogeneous training datasets. The large language model processing module is used to load and freeze all parameters of a pre-trained large language model as the base model. The adapter building module is used to build shared low-rank adapters and entity-specific low-rank adapters; the shared low-rank adapter is used to capture general knowledge across datasets and entity types, and the entity-specific low-rank adapter is used to capture recognition patterns for specific entity types. The shared low-rank adapter fine-tuning module is used to train the adapter based on the association relationship, the shared low-rank adapter, and the base model to obtain the shared adapter. The entity-specific low-rank adapter fine-tuning module is used to train adapters based on the association relationship, the entity-specific low-rank adapter, and the base model to obtain the entity-specific adapter for each entity type. The Named Entity Recognition Model Generation Module is used to save the shared adapter, the entity-specific adapter for each entity type, and the base model to obtain a pre-trained Named Entity Recognition Model.

[0018] Fourthly, embodiments of this application provide a named entity recognition device, the device comprising: The loading module is used to load a pre-trained named entity recognition model. The named entity recognition model includes a shared adapter, entity-specific adapters for each entity type, and a base model. The base model is obtained by freezing all parameters of the pre-trained large language model. The shared adapter and entity-specific adapter are trained using the named entity recognition model training method described above. The receiving module is used to receive named entity recognition requests, which carry the actual requirements of the current inference task and the text to be recognized. The adapter selection module is used to select a target entity-specific adapter that matches the actual needs of the current inference task from the entity-specific adapters for each entity type. The fusion module is used to fuse the base model, the target entity-specific adapter, and the shared adapter to generate the final target model for inference. The recognition result output module is used to input the text to be recognized into the target model for forward propagation calculation, and output each recognized entity fragment in the text and its corresponding entity type label.

[0019] The technical solutions provided in this application embodiment may include the following beneficial effects: In this embodiment, on the one hand, the entity richness of each dataset is dynamically quantified through entity coverage vectors, and an entity-aware mask loss function is designed based on this. This allows for the allocation of higher weights to samples with more comprehensive entity coverage during shared adapter training, guiding the model to learn common features across datasets. Simultaneously, an entity-specific low-rank adapter is trained independently for each entity type, enabling it to learn proprietary patterns only from data containing that entity type, avoiding interference between different data types. This design decouples general knowledge from typed knowledge, effectively mitigating negative transfer and maintaining robust performance across multiple heterogeneous datasets. On the other hand, a modular adapter structure allows the shared adapter and each entity-specific adapter to be trained, stored, and loaded independently. During the inference phase, a weighted fusion mechanism dynamically combines adapters for the required entity types, achieving plug-and-play flexible deployment. When a new entity type is added, only the independent adapter for that type needs to be trained, without modifying existing modules. Furthermore, adapter weights can be adjusted online, quickly responding to changes in business requirements.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0022] Figure 1 This is a flowchart illustrating a named entity recognition model training method provided in an embodiment of this application; Figure 2 This is a schematic diagram of a two-stage entity perception low-rank adaptation training method provided in this application; Figure 3 This is a flowchart illustrating a named entity recognition method provided in an embodiment of this application; Figure 4 This is a schematic diagram of an entity-aware adapter fine-tuning architecture provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a named entity recognition model training device provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of a named entity recognition device provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0023] The following description and accompanying drawings fully illustrate specific embodiments of this application to enable those skilled in the art to practice them.

[0024] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0025] In the following description, when referring to the accompanying drawings, the same numbers in different drawings denote the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0026] In the description of this application, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances. Furthermore, in the description of this application, unless otherwise stated, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0027] This application provides a named entity recognition method and apparatus for training its model, addressing the problems existing in the aforementioned related technologies. In the embodiments of this application, on the one hand, entity richness of each dataset is dynamically quantified using entity coverage vectors, and an entity-aware mask loss function is designed based on this. This allows for the allocation of higher weights to samples with more comprehensive entity coverage during shared adapter training, guiding the model to learn common features across datasets. Simultaneously, entity-specific low-rank adapters are trained independently for each entity type, enabling it to learn proprietary patterns only from data containing that entity type, avoiding interference between different data types. This design decouples general knowledge from typified knowledge, effectively mitigating negative transfer and maintaining robust performance across multiple heterogeneous datasets. On the other hand, a modular adapter structure allows shared adapters and entity-specific adapters to be trained, stored, and loaded independently. During the inference phase, adapters for the required entity types are dynamically combined through a weighted fusion mechanism, achieving flexible, plug-and-play deployment. When adding a new entity type, only the independent adapter for that type needs to be trained, without modifying existing modules. Furthermore, adapter weights can be adjusted online, quickly responding to changes in business requirements. The following describes the process in detail using exemplary embodiments.

[0028] This application provides a named entity recognition method and apparatus for training its model, to address the problems existing in the aforementioned related technologies. The following will be discussed in conjunction with the appendix... Figure 1 - Appendix Figure 4 This application provides a detailed description of the named entity recognition model training method provided in its embodiments. This method can be implemented using a computer program and can run on a named entity recognition model training device based on the von Neumann architecture. This computer program can be integrated into applications or run as a standalone utility application.

[0029] Please see Figure 1 This is a flowchart illustrating a named entity recognition model training method provided in an embodiment of this application. Figure 1 As shown, the method in this application embodiment may include the following steps: S101, Establish the association between the multi-source heterogeneous training dataset, entity coverage vector, and source dataset identifier; the entity coverage vector is used to represent the entity types contained in the multi-source heterogeneous training dataset; In this context, a multi-source heterogeneous training dataset refers to a collection of training data originating from two or more different data sources, differing in data structure, annotation standards, or content domain. An entity coverage vector quantitatively represents the set of entity types contained in a dataset. Each dimension of this vector corresponds to a predefined entity type, and its value indicates whether that entity type exists in the dataset. A source dataset identifier is a symbol or code used to uniquely identify and distinguish each independent dataset in the multi-source heterogeneous training dataset. The association refers to the logical mapping established between the multi-source heterogeneous training dataset, its corresponding entity coverage vector, and its source dataset identifier, used to index the entity type information contained within the dataset during model training based on the dataset identifier.

[0030] In some embodiments of this application, the specific process of establishing the association between multi-source heterogeneous training datasets, entity coverage vectors, and source dataset identifiers includes: acquiring multiple multi-source heterogeneous training datasets; scanning and counting the entity type indicators contained in each multi-source heterogeneous training dataset to obtain the entity coverage vector of each multi-source heterogeneous training dataset; acquiring the source dataset identifier of each multi-source heterogeneous training dataset; and for each multi-source heterogeneous training dataset, associating its corresponding entity coverage vector and source dataset identifier as the association between the multi-source heterogeneous training dataset, entity coverage vector, and source dataset identifier.

[0031] In this context, entity type indicators refer to symbols, labels, or codes used to mark or identify specific entity types in a multi-source heterogeneous training dataset. These indicators are the fundamental elements that constitute the entity coverage vector. By scanning and statistically analyzing these indicators, the set of entity types contained in the dataset can be determined.

[0032] In one possible implementation, the following parameters are maintained for each dataset 𝒟ᵢ: unique identifier idᵢ, entity coverage vector cᵢ, and dataset weights. .

[0033] For example, joint training can be performed using three medical text datasets: Dataset A: CLEF eHealth 2013, containing disease, drug, and symptom entities; Dataset B: i2b2 / VA 2010, containing question, test, and treatment entities; Dataset C: n2c2 2018 contains entities for drugs, symptoms, and adverse reactions.

[0034] By converting the three datasets to a unified JSON format and establishing relationships between them, we can obtain: Dataset A: Entity coverage ["disease", "drug", "symptom"], weight 1.0; Dataset B: Entity coverage ["problem", "test", "treatment"], weight 1.0; Dataset C: Entity coverage ["drug", "symptom", "adverse reaction"], weight 0.8.

[0035] S102, Load and freeze all parameters of the pre-trained large language model as the base model; Among them, a pre-trained large language model is a deep neural network model that has been pre-trained using unsupervised learning based on a massive general corpus. A base model refers to a model instance that is loaded as a feature extractor or computational skeleton during the training process of a specific task (such as named entity recognition), and whose original pre-trained parameters are set to a non-updateable state.

[0036] In one possible implementation, a pre-trained large-scale language model (such as Qwen2-7B-Instruct) is used as the base model, and all its parameters (denoted as W0) are frozen to maintain the model's general language understanding ability and prevent catastrophic forgetting.

[0037] S103, construct a shared low-rank adapter and an entity-specific low-rank adapter; wherein, the shared low-rank adapter is used to capture general knowledge across datasets and across entity types, and the entity-specific low-rank adapter is used to capture recognition patterns for specific entity types; In one possible implementation, the shared low-rank adapter is defined as:

[0038] in, and It is a trainable low-rank matrix. (Rank parameter) provides sufficient expression capacity. (Scaling factor), controls the extent of adaptation. For being composed of real numbers, possessing lines and A set of columns of matrices. It is the rank, which controls the dimension of low-rank decomposition. It is the output feature dimension, corresponding to the length of the output vector of a certain linear layer in the base model (such as the Query, Key, Value, Output projection layers in Attention).

[0039] The model weights of the base model can be updated as follows:

[0040] Among them, the shared low-rank adapter is adapted to the key layers (query, key, value, output, gate, up, down projection layers) of the base model and is trained for two rounds using all training data.

[0041] In one possible implementation, a dedicated small LoRA adapter is created for each entity type (e.g., disease, drug, symptom, treatment, etc.) to capture the recognition pattern of that specific entity type. Each entity-specific low-rank adapter can be defined as:

[0042] in, and For entity type The trainable first low-rank matrix and the second low-rank matrix, (Smaller rank) Optimize parameter efficiency. Each entity-specific low-rank adapter is trained for one epoch only on the query and value projection layers, using filtering to include only data of the corresponding entity type.

[0043] S104. Based on the association relationship and the shared low-rank adapter, the adapter is trained in combination with the basic model to obtain the shared adapter. In some embodiments of this application, the specific process of obtaining a shared adapter by training an adapter based on the association relationship and a shared low-rank adapter in conjunction with the base model includes: injecting a shared low-rank adapter into a preset first key projection layer of the base model; instantiating the parameters of the injected shared low-rank adapter into trainable first low-rank matrices and second low-rank matrices to obtain a first initialization model; traversing the current training samples from multiple multi-source heterogeneous training datasets; inputting the current training samples into the first initialization model to calculate the weighted loss of the current training samples; updating the parameters of the first low-rank matrix and the second low-rank matrix using the weighted loss; continuing to execute the step of traversing the current training samples from multiple multi-source heterogeneous training datasets until the number of training rounds reaches a preset number, thereby obtaining a shared adapter that encodes general entity recognition knowledge across all datasets.

[0044] The pre-selected first key projection layer refers to a specific linear transformation layer in the neural network architecture of the base model, which is used to inject the adapter. It usually corresponds to the query, key, value, or output projection layer in the self-attention mechanism, as well as the up or down projection layer in the feedforward network. For example, (query, key, value, output, gate, up, down projection layer).

[0045] Specifically, the process of calculating the weighted loss of the current training sample includes: obtaining the current source dataset identifier of the current training sample; querying the current entity coverage vector corresponding to the current source dataset identifier from the association relationship; calculating the dynamic scalar weights of the current training sample based on the current entity coverage vector; performing machine learning forward propagation on the current training sample to quantify the original cross-entropy loss; and generating the weighted loss of the current training sample based on the dynamic scalar weights, cross-entropy loss, and a preset entity-aware mask loss function.

[0046] Specifically, the expression for calculating dynamic scalar weights is as follows:

[0047] in, It is the first Dynamic scalar weights for each training sample These are the basic weight hyperparameters. It is 0.3. This represents the total number of global entity types. Used to identify datasets Does it include entity type? =1 indicates that the dataset contains entity types. , =0 indicates that the dataset does not contain entity types. , This refers to the number of unique entity types that appear in all datasets; The default expression for the entity-aware masking loss function is:

[0048] in, For weighted loss, It is the original cross-entropy loss, used to measure the model's performance on the dataset. The difference between the predicted results and the actual labels.

[0049] In one possible implementation, the pre-trained base model is loaded and all parameters are frozen. The shared low-rank adapter (r=16, α=32) is initialized. All training datasets are loaded, a dataset ID and entity overlay vector are added to each sample, the shared adapter is trained for 2 epochs using entity-aware mask loss, and the trained shared adapter is saved.

[0050] S105, Based on the association relationship and the entity-specific low-rank adapter, the adapter is trained in combination with the base model to obtain the entity-specific adapter for each entity type; In some embodiments of this application, the specific process of obtaining an entity-specific adapter for each entity type by training adapters based on association relationships and entity-specific low-rank adapters in conjunction with the base model includes: freezing the gradient calculation of the shared adapter by disabling it on the basis of freezing all parameters of the pre-trained large language model; determining the target entity type; the target entity type is each preset entity type; filtering out multi-source heterogeneous training datasets from the association relationships that indicate the existence of the target entity type in the entity coverage vector, so as to retain the multi-source heterogeneous training datasets containing the target entity type as a subset of training data specific to the target entity type; injecting entity-specific low-rank adapters into the preset second key projection layer of the base model; instantiating the injected entity-specific low-rank adapters into trainable third low-rank matrices and fourth low-rank matrices to obtain a second initialization model; training the second initialization model according to the subset of training data specific to the target entity type to obtain an entity-specific adapter for each entity type.

[0051] The preset second key projection layer refers to the linear transformation layer in the base model used to inject entity-specific low-rank adapters. This layer can be the same as the preset first key projection layer, or it can be other specific projection layers in the model used to carry out parameter updates for specific tasks. For example, query and value projection layers.

[0052] The filtering expression for the multi-source heterogeneous training dataset is as follows: Dⱼ = {s∈D|c_{id(s),j}=1}; where Dⱼ is the filtered dataset used to train the entity-specific adapter for the j-th entity type. s∈𝒟 indicates that element s in the set is a sample from the original total training dataset𝒟. c_{id(s)} is a filtering function that returns a unique identifier from the source dataset to which sample s belongs, with id as its unique identifier.

[0053] Specifically, the process of training the second initialization model based on the specific training data subset for the target entity type includes: inputting the specific training data subset for the target entity type into the second model to perform forward propagation in machine learning on the specific training data subset and calculate the original second cross-entropy loss; stopping the iteration when the second cross-entropy loss reaches its minimum; or, if the second cross-entropy loss has not reached its minimum, updating the parameters of the third low-rank matrix and the fourth low-rank matrix, and continuing to execute the step of inputting the specific training data subset for the target entity type into the second model until the second cross-entropy loss reaches its minimum, and then stopping the iteration.

[0054] In one possible implementation, the base model and the shared adapter are frozen. For each entity type eⱼ: the entity-specific adapter is initialized (r=8, α=16), the training data is filtered to keep only the samples containing entity type eⱼ, the entity adapter is trained for 1 epoch using the filtered data, the entity adapter is saved, and the entity configuration information (entity type list, dataset overlay mapping, etc.) is saved.

[0055] S106, save the shared adapter, the entity-specific adapter for each entity type, and the base model to obtain a pre-trained named entity recognition model.

[0056] For example Figure 2 As shown, Figure 2This is a schematic diagram of a two-stage entity-aware low-rank adaptation training method provided in this application. In a medical data scenario, firstly, in the first stage, a shared low-rank adapter injected into a pre-trained large language model is trained using a combined dataset containing multiple entity types. During this process, all parameters of the pre-trained large language model remain frozen, and the output is calculated only through forward propagation. Backpropagation is performed using a preset entity-aware mask loss function to update the parameters of the shared low-rank adapter, enabling it to learn cross-domain general semantic representations. Subsequently, the second stage, namely the independent training of entity-specific adapters, is entered. In this stage, the combined dataset is decomposed into dedicated training subsets such as disease data, drug data, and symptom data, and a corresponding entity-specific adapter (such as a disease adapter, drug adapter, and symptom adapter) is injected into each subset. When training the specific entity adapter, the parameters of both the pre-trained large language model and the shared low-rank adapter remain frozen. The output of the specific entity is calculated only through forward propagation, and backpropagation is performed using a mask loss function filtered by the corresponding entity to update only the parameters of the current target entity-specific adapter. Through the above two-stage training, a composite model with both general semantic understanding ability and specific entity recognition accuracy is finally obtained.

[0057] In one possible implementation, the data obtained in step S101 can be divided into a training set and a validation set (9:1 ratio). The model configuration is as follows: base model: Qwen2-7B-Instruct, shared adapter: r=16, α=32, dropout=0.05, entity adapter: r=8, α=16, dropout=0.05, mask loss parameter: β=0.3.

[0058] Training the shared adapter: Batch size: 1 (8 gradient accumulation steps), Learning rate: 2×10⁻ 5 Training epochs: 2 epochs; Using all training data, applying entity-aware mask loss; Training time: Approximately 4 hours (NVIDIA A100 GPU).

[0059] Training entity adapters: Disease adapter: uses samples containing diseases in dataset A; Drug adapter: uses samples containing drugs in datasets A and C; Symptom adapter: uses samples containing symptoms in datasets A and C; Learning rate: 1×10⁻ 5 Training cycles: 1 epoch; Training time per adapter: approximately 30-45 minutes.

[0060] In this embodiment, on the one hand, the entity richness of each dataset is dynamically quantified through entity coverage vectors, and an entity-aware mask loss function is designed based on this. This allows for the allocation of higher weights to samples with more comprehensive entity coverage during shared adapter training, guiding the model to learn common features across datasets. Simultaneously, an entity-specific low-rank adapter is trained independently for each entity type, enabling it to learn proprietary patterns only from data containing that entity type, avoiding interference between different data types. This design decouples general knowledge from typed knowledge, effectively mitigating negative transfer and maintaining robust performance across multiple heterogeneous datasets. On the other hand, a modular adapter structure allows the shared adapter and each entity-specific adapter to be trained, stored, and loaded independently. During the inference phase, a weighted fusion mechanism dynamically combines adapters for the required entity types, achieving plug-and-play flexible deployment. When a new entity type is added, only the independent adapter for that type needs to be trained, without modifying existing modules. Furthermore, adapter weights can be adjusted online, quickly responding to changes in business requirements.

[0061] Please see Figure 3 The diagram below illustrates a named entity recognition method provided in this application. Figure 3 As shown, the method in this application embodiment may include the following steps: S201, Load the pre-trained named entity recognition model. The named entity recognition model includes a shared adapter, an entity-specific adapter for each entity type, and a base model. The base model is obtained by freezing all parameters of the pre-trained large language model. The shared adapter and the entity-specific adapter are trained using the named entity recognition model training method described above. S202, Receive a named entity recognition request. The named entity recognition request carries the actual entity type of the current inference task and the text to be recognized. S203, Select a target entity-specific adapter that matches each actual entity type of the current inference task from the entity-specific adapters for each entity type; S204 combines the parameters of the base model, each target entity-specific adapter, and the shared adapter to generate the final target model for inference. S205, input the text to be recognized into the target model for forward propagation calculation, and output each recognized entity fragment in the text to be recognized and its corresponding entity type label; Specifically, the expression for the parameter combination is:

[0062] in, These are the model weights of the target model ultimately used for inference. These are the frozen parameters of the base model. It is the fusion weight of the shared adapter. These are the incremental weights of the shared low-rank adapter learned in the first phase of training. This refers to the number of entity adapters that have been loaded. It is the first Fusion weights for entity-specific adapters, In the second phase of training, the first... Incremental weights of entity-specific low-rank adapters learned for each entity type;

[0063] in, and These are the trainable first low-rank matrix and the second low-rank matrix;

[0064] in, and They are entity types The trainable third low-rank matrix and the fourth low-rank matrix.

[0065] In one possible implementation, the base model, shared adapter, and required entity adapters are loaded, with fusion weights of 0.6 for the shared model and 0.13 for each entity. Evaluation is performed on the validation set, and the results are shown in Table 1.

[0066] Table 1

[0067] As can be seen, compared with training on a single dataset, this application achieves an improvement of 0.85% and 0.60% on the i2b2 2012 and i2b2 2010 datasets, respectively; compared with simple joint training, this application achieves an improvement of 1.06% on i2b2 2012 and 0.24% on i2b2 2010, effectively alleviating the negative transfer problem; compared with standard continuous LoRA, this application achieves a significant improvement of 8.16% on i2b2 2010 and 0.95% on i2b2 2012. In particular, when the CLEF2013 data is introduced, the F1 score of continuous LoRA drops sharply to 39.91%, while the method of this application maintains stable performance through the entity perception mechanism, with only 1.2% of the number of parameters required for full fine-tuning, and training time is reduced by about 65%.

[0068] For example Figure 4 As shown, Figure 4This is a schematic diagram of an entity-aware adapter fine-tuning architecture provided in this application: In the application scenario of medical datasets, firstly, multi-source heterogeneous data from medical (including diseases and drugs) and general (including people and places) domains are input into the basic large language model with frozen parameters. Then, cross-domain general knowledge is learned through shared low-rank adapters. On this basis, entity-specific adapters such as disease, drug, and symptom are further differentiated to capture the unique patterns of each entity type. Finally, the entity-aware masking module dynamically combines the outputs of these adapters according to the reasoning requirements to generate a comprehensive combined output result.

[0069] In this embodiment, on the one hand, the entity richness of each dataset is dynamically quantified through entity coverage vectors, and an entity-aware mask loss function is designed based on this. This allows for the allocation of higher weights to samples with more comprehensive entity coverage during shared adapter training, guiding the model to learn common features across datasets. Simultaneously, an entity-specific low-rank adapter is trained independently for each entity type, enabling it to learn proprietary patterns only from data containing that entity type, avoiding interference between different data types. This design decouples general knowledge from typed knowledge, effectively mitigating negative transfer and maintaining robust performance across multiple heterogeneous datasets. On the other hand, a modular adapter structure allows the shared adapter and each entity-specific adapter to be trained, stored, and loaded independently. During the inference phase, a weighted fusion mechanism dynamically combines adapters for the required entity types, achieving plug-and-play flexible deployment. When a new entity type is added, only the independent adapter for that type needs to be trained, without modifying existing modules. Furthermore, adapter weights can be adjusted online, quickly responding to changes in business requirements.

[0070] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0071] Please see Figure 5 This illustration shows a schematic diagram of the structure of a named entity recognition model training device provided in an exemplary embodiment of this application. The named entity recognition model training device can be implemented as all or part of an electronic device through software, hardware, or a combination of both. The device includes a private dataset, a general model deployed for federated learning, and a local model. The general model is shared by all clients, and the local model is trained for a local service. The device 1 includes an association establishment module 10, a large language model processing module 20, an adapter construction module 30, a shared low-rank adapter fine-tuning module 40, an entity-specific low-rank adapter fine-tuning module 50, and a named entity recognition model generation module 60.

[0072] The association establishment module 10 is used to establish the association between the multi-source heterogeneous training dataset, entity coverage vector, and source dataset identifier; the entity coverage vector is used to represent the entity types contained in the multi-source heterogeneous training dataset. Large language model processing module 20 is used to load and freeze all parameters of the pre-trained large language model as the base model; The adapter building module 30 is used to build a shared low-rank adapter and an entity-specific low-rank adapter; wherein, the shared low-rank adapter is used to capture general knowledge across datasets and across entity types, and the entity-specific low-rank adapter is used to capture recognition patterns for specific entity types; The shared low-rank adapter fine-tuning module 40 is used to train the adapter based on the association relationship, the shared low-rank adapter, and the base model to obtain the shared adapter. The entity-specific low-rank adapter fine-tuning module 50 is used to train the adapter based on the association relationship, the entity-specific low-rank adapter, and the base model to obtain the entity-specific adapter for each entity type. The named entity recognition model generation module 60 is used to save the shared adapter, the entity-specific adapter for each entity type, and the base model to obtain a pre-trained named entity recognition model.

[0073] It should be noted that the named entity recognition model training device provided in the above embodiments is only illustrated by the division of the above functional modules when executing the named entity recognition model training method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the named entity recognition model training device and the named entity recognition model training method embodiments provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0074] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0075] In this embodiment, on the one hand, the entity richness of each dataset is dynamically quantified through entity coverage vectors, and an entity-aware mask loss function is designed based on this. This allows for the allocation of higher weights to samples with more comprehensive entity coverage during shared adapter training, guiding the model to learn common features across datasets. Simultaneously, an entity-specific low-rank adapter is trained independently for each entity type, enabling it to learn proprietary patterns only from data containing that entity type, avoiding interference between different data types. This design decouples general knowledge from typed knowledge, effectively mitigating negative transfer and maintaining robust performance across multiple heterogeneous datasets. On the other hand, a modular adapter structure allows the shared adapter and each entity-specific adapter to be trained, stored, and loaded independently. During the inference phase, a weighted fusion mechanism dynamically combines adapters for the required entity types, achieving plug-and-play flexible deployment. When a new entity type is added, only the independent adapter for that type needs to be trained, without modifying existing modules. Furthermore, adapter weights can be adjusted online, quickly responding to changes in business requirements.

[0076] Please see Figure 6 This illustration shows a schematic diagram of the structure of a named entity recognition device provided in an exemplary embodiment of this application. The named entity recognition model training device can be implemented as all or part of an electronic device through software, hardware, or a combination of both. The device 2 includes a loading module 70, a receiving module 80, an adapter selection module 90, a fusion module 100, and a recognition result output module 110.

[0077] Loading module 70 is used to load a pre-trained named entity recognition model. The named entity recognition model includes a shared adapter, an entity-specific adapter for each entity type, and a base model. The base model is obtained by freezing all parameters of the pre-trained large language model. The shared adapter and the entity-specific adapter are trained using the named entity recognition model training method described above. The receiving module 80 is used to receive named entity recognition requests, which carry the actual requirements of the current inference task and the text to be recognized. The adapter selection module 90 is used to select a target entity-specific adapter that matches the actual needs of the current inference task from the entity-specific adapters for each entity type. The fusion module 100 is used to fuse the base model, the target entity-specific adapter, and the shared adapter to generate the final target model for inference. The recognition result output module 110 is used to input the text to be recognized into the target model for forward propagation calculation and output each recognized entity fragment in the text to be recognized and its corresponding entity type label.

[0078] It should be noted that the named entity recognition model training device provided in the above embodiments is only illustrated by the division of the above functional modules when executing the named entity recognition model training method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the named entity recognition model training device and the named entity recognition model training method embodiments provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0079] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0080] In this embodiment, on the one hand, the entity richness of each dataset is dynamically quantified through entity coverage vectors, and an entity-aware mask loss function is designed based on this. This allows for the allocation of higher weights to samples with more comprehensive entity coverage during shared adapter training, guiding the model to learn common features across datasets. Simultaneously, an entity-specific low-rank adapter is trained independently for each entity type, enabling it to learn proprietary patterns only from data containing that entity type, avoiding interference between different data types. This design decouples general knowledge from typed knowledge, effectively mitigating negative transfer and maintaining robust performance across multiple heterogeneous datasets. On the other hand, a modular adapter structure allows the shared adapter and each entity-specific adapter to be trained, stored, and loaded independently. During the inference phase, a weighted fusion mechanism dynamically combines adapters for the required entity types, achieving plug-and-play flexible deployment. When a new entity type is added, only the independent adapter for that type needs to be trained, without modifying existing modules. Furthermore, adapter weights can be adjusted online, quickly responding to changes in business requirements.

[0081] This application also provides a computer-readable medium having program instructions stored thereon, which, when executed by a processor, implement the named entity recognition model training method provided in the above-described method embodiments.

[0082] This application also provides a computer program product containing instructions that, when run on a computer, causes the computer to execute the named entity recognition model training method of the various method embodiments described above.

[0083] Please see Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 7 As shown, the electronic device 1000 may include: at least one processor 1001, at least one network interface 1004, a user interface 1003, a memory 1005, and at least one communication bus 1002.

[0084] The communication bus 1002 is used to realize the connection and communication between these components.

[0085] The user interface 1003 may include a display screen and a camera. Optionally, the user interface 1003 may also include a standard wired interface and a wireless interface.

[0086] The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0087] The processor 1001 may include one or more processing cores. The processor 1001 connects to various parts within the electronic device 1000 using various interfaces and lines. It executes various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1005, and by calling data stored in the memory 1005. Optionally, the processor 1001 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 1001 may integrate one or more of the following: a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed on the screen; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip, without being integrated into the processor 1001.

[0088] The memory 1005 may include random access memory (RAM) or read-only memory. Optionally, the memory 1005 may include a non-transitory computer-readable storage medium. The memory 1005 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 1005 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 1005 may also be at least one storage system located remotely from the aforementioned processor 1001. Figure 7 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a named entity recognition model training application.

[0089] exist Figure 7 In the illustrated electronic device 1000, the user interface 1003 is mainly used to provide an input interface for the user and to obtain user input data; while the processor 1001 can be used to call the named entity recognition model training application stored in the memory 1005, and specifically perform the following operations: Establish the association between the multi-source heterogeneous training dataset, entity coverage vector, and source dataset identifier; the entity coverage vector is used to represent the entity types contained in the multi-source heterogeneous training dataset. Load and freeze all parameters of the pre-trained large language model as the base model; Construct a shared low-rank adapter and an entity-specific low-rank adapter; the shared low-rank adapter is used to capture general knowledge across datasets and entity types, while the entity-specific low-rank adapter is used to capture recognition patterns for specific entity types. Based on the association relationship and the shared low-rank adapter, the adapter is trained in combination with the base model to obtain the shared adapter; Based on the association relationship and the entity-specific low-rank adapter, the adapter is trained in combination with the base model to obtain the entity-specific adapter for each entity type. Save the shared adapter, the entity-specific adapter for each entity type, and the base model to obtain a pre-trained named entity recognition model.

[0090] In one embodiment, when the processor 1001 executes the operation of establishing the association between the multi-source heterogeneous training dataset, entity overlay vector, and source dataset identifier, it specifically performs the following operations: Obtain multiple heterogeneous training datasets from multiple sources; Scan and count the entity type indicators contained in each multi-source heterogeneous training dataset to obtain the entity coverage vector of each multi-source heterogeneous training dataset; Obtain the source dataset identifier for each multi-source heterogeneous training dataset; For each multi-source heterogeneous training dataset, associate its corresponding entity coverage vector with the source dataset identifier, as the association relationship between the multi-source heterogeneous training dataset, entity coverage vector, and source dataset identifier.

[0091] In one embodiment, when processor 1001 performs adapter training based on association relationships, a shared low-rank adapter, and a base model to obtain a shared adapter, it specifically performs the following operations: Inject a shared low-rank adapter into the preset first key projection layer of the base model; The parameters of the injected shared low-rank adapter are instantiated into trainable first low-rank matrices and second low-rank matrices to obtain the first initialization model; The current training sample is retrieved from multiple heterogeneous training datasets from multiple sources. Input the current training sample into the first initialized model to calculate the weighted loss of the current training sample; The parameters of the first low-rank matrix and the second low-rank matrix are updated using a weighted loss. Continue executing the step of traversing the current training samples from multiple multi-source heterogeneous training datasets until the training count reaches a preset number of rounds, resulting in a shared adapter that encodes general entity recognition knowledge across all datasets.

[0092] In one embodiment, when the processor 1001 performs the calculation of the weighted loss of the current training sample, it specifically performs the following operations: Get the current source dataset identifier of the current training sample; Retrieve the current entity coverage vector corresponding to the current source dataset identifier from the association relationships; Calculate the dynamic scalar weights of the current training sample based on the current entity coverage vector; Perform machine learning forward propagation on the current training samples to quantify the original cross-entropy loss; The weighted loss of the current training sample is generated based on dynamic scalar weights, cross-entropy loss, and a preset entity perception mask loss function.

[0093] In one embodiment, when processor 1001 performs adapter training based on association relationships and entity-specific low-rank adapters, combined with the base model, to obtain entity-specific adapters for each entity type, it specifically performs the following operations: To freeze the shared adapter, the gradient computation of the shared adapter is disabled, based on freezing all parameters of the pre-trained large language model. Determine the target entity type; the target entity type is each preset entity type; From the association relationships, filter out the multi-source heterogeneous training datasets that indicate the existence of target entity types in the entity coverage vectors, so as to retain the multi-source heterogeneous training datasets containing target entity types as a subset of training data dedicated to target entity types; In the pre-defined second key projection layer of the base model, inject an entity-specific low-rank adapter; The injected entity-specific low-rank adapter is instantiated into trainable third and fourth low-rank matrices to obtain the second initialization model. The second initialization model is trained based on a subset of training data specific to the target entity type, resulting in an entity-specific adapter for each entity type.

[0094] In one embodiment, when the processor 1001 trains the second initialization model using a subset of training data specific to the target entity type, it specifically performs the following operations: Input a subset of training data specific to the target entity type into the second model to perform forward propagation in machine learning on the subset of training data and calculate the original second cross-entropy loss. Stop iterating when the second cross-entropy loss reaches its minimum; or, If the second cross-entropy loss has not reached its minimum, update the parameters of the third and fourth low-rank matrices, and continue to execute the step of inputting a subset of training data specific to the target entity type into the second model until the second cross-entropy loss reaches its minimum, at which point the iteration stops.

[0095] In this embodiment, on the one hand, the entity richness of each dataset is dynamically quantified through entity coverage vectors, and an entity-aware mask loss function is designed based on this. This allows for the allocation of higher weights to samples with more comprehensive entity coverage during shared adapter training, guiding the model to learn common features across datasets. Simultaneously, an entity-specific low-rank adapter is trained independently for each entity type, enabling it to learn proprietary patterns only from data containing that entity type, avoiding interference between different data types. This design decouples general knowledge from typed knowledge, effectively mitigating negative transfer and maintaining robust performance across multiple heterogeneous datasets. On the other hand, a modular adapter structure allows the shared adapter and each entity-specific adapter to be trained, stored, and loaded independently. During the inference phase, a weighted fusion mechanism dynamically combines adapters for the required entity types, achieving plug-and-play flexible deployment. When a new entity type is added, only the independent adapter for that type needs to be trained, without modifying existing modules. Furthermore, adapter weights can be adjusted online, quickly responding to changes in business requirements.

[0096] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program for training the named entity recognition model can be stored in a computer-readable storage medium. When executed, the program can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory, or random access memory, etc.

[0097] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A method for training a named entity recognition model, the method comprising: The method includes: Establish the association between the multi-source heterogeneous training dataset, entity coverage vector, and source dataset identifier; the entity coverage vector is used to characterize the entity types contained in the multi-source heterogeneous training dataset. Load and freeze all parameters of the pre-trained large language model as the base model; Construct a shared low-rank adapter and an entity-specific low-rank adapter; wherein the shared low-rank adapter is used to capture general knowledge across datasets and entity types, and the entity-specific low-rank adapter is used to capture recognition patterns for specific entity types; Based on the aforementioned association and the shared low-rank adapter, and combined with the base model, adapter training is performed to obtain the shared adapter; Based on the association relationship and the entity-specific low-rank adapter, adapter training is performed in conjunction with the base model to obtain an entity-specific adapter for each entity type. Save the shared adapter, the entity-specific adapter for each entity type, and the base model to obtain a pre-trained named entity recognition model.

2. The method of claim 1, wherein, The process of establishing the association between multi-source heterogeneous training datasets, entity coverage vectors, and source dataset identifiers includes: Obtain multiple heterogeneous training datasets from multiple sources; Scan and count the entity type indicators contained in each multi-source heterogeneous training dataset to obtain the entity coverage vector of each multi-source heterogeneous training dataset; Obtain the source dataset identifier for each of the multi-source heterogeneous training datasets; For each multi-source heterogeneous training dataset, associate its corresponding entity coverage vector and source dataset identifier as the association relationship between the multi-source heterogeneous training dataset, entity coverage vector, and source dataset identifier.

3. The method of claim 1, wherein, The step of training an adapter based on the association relationship, the shared low-rank adapter, and the base model to obtain a shared adapter includes: The shared low-rank adapter is injected into the preset first key projection layer of the basic model; The parameters of the injected shared low-rank adapter are instantiated into trainable first low-rank matrices and second low-rank matrices to obtain the first initialization model; The current training sample is retrieved from multiple heterogeneous training datasets from multiple sources. The current training sample is input into the first initialization model to calculate the weighted loss of the current training sample; The parameters of the first low-rank matrix and the second low-rank matrix are updated using the weighted loss. Continue executing the step of traversing the current training samples from multiple multi-source heterogeneous training datasets until the training count reaches a preset number of rounds, and obtain a shared adapter that encodes general entity recognition knowledge across all datasets.

4. The method of claim 3, wherein, The calculation of the weighted loss of the current training sample includes: Obtain the current source dataset identifier of the current training sample; Query the current entity coverage vector corresponding to the current source dataset identifier from the association relationship; Calculate the dynamic scalar weights of the current training sample based on the current entity coverage vector; Perform machine learning forward propagation on the current training samples to quantify the original cross-entropy loss; The weighted loss of the current training sample is generated based on the dynamic scalar weights, the cross-entropy loss, and a preset entity-aware mask loss function.

5. The method according to claim 4, characterized in that, The calculation expression for the dynamic scalar weight is as follows: in, It is the dynamic scalar weight of the i-th training sample. This is the basic weight hyperparameter, with a value of 0.

3. n represents the total number of global entity types, used to identify whether the dataset contains entity types. =1 indicates that the dataset contains entity types. , =0 indicates that the dataset does not contain entity types. , This refers to the number of unique entity types that appear in all datasets; The function expression of the preset entity-aware mask loss function is: in, For weighted loss, It is the original cross-entropy loss, used to measure the difference between the model's predictions for dataset s and the true labels.

6. The method according to claim 1, characterized in that, The step of training adapters based on the association relationship, the entity-specific low-rank adapter, and the base model to obtain an entity-specific adapter for each entity type includes: Based on freezing all parameters of the pre-trained large language model, the gradient calculation of the shared adapter is disabled to freeze the shared adapter; Determine the target entity type; the target entity type is each preset entity type; From the association, filter out the multi-source heterogeneous training datasets that indicate the existence of the target entity type in the entity coverage vector, so as to retain the multi-source heterogeneous training datasets containing the target entity type as a dedicated training data subset of the target entity type; In the preset second key projection layer of the base model, the entity-specific low-rank adapter is injected; The injected entity-specific low-rank adapter is instantiated into trainable third and fourth low-rank matrices to obtain the second initialization model. The second initialization model is trained based on a subset of training data specific to the target entity type to obtain an entity-specific adapter for each entity type.

7. The method according to claim 6, characterized in that, The step of training the second initialization model based on a subset of training data specific to the target entity type includes: A subset of training data specific to the target entity type is input into the second model to perform forward propagation in machine learning on the subset of training data and calculate the original second cross-entropy loss. Stop iterating when the second cross-entropy loss reaches its minimum; or, If the second cross-entropy loss has not reached its minimum, update the parameters of the third low-rank matrix and the fourth low-rank matrix, and continue to execute the step of inputting the specific training data subset of the target entity type into the second model until the second cross-entropy loss reaches its minimum, and then stop iterating.

8. A named entity recognition method, characterized in that, The method includes: Load a pre-trained named entity recognition model, the named entity recognition model including a shared adapter, an entity-specific adapter for each entity type, and a base model; wherein, the base model is obtained by freezing all parameters of a pre-trained large language model; the shared adapter and the entity-specific adapter are trained by the named entity recognition model training method according to any one of claims 1-7; Receive a named entity recognition request, the named entity recognition request carrying each actual entity type of the current inference task and the text to be recognized; From the entity-specific adapters of each entity type, select a target entity-specific adapter that matches each actual entity type of the current inference task; The base model, each target entity-specific adapter, and the shared adapter are combined with parameters to generate the final target model for inference. The text to be identified is input into the target model for forward propagation calculation, and each identified entity fragment in the text to be identified and its corresponding entity type label are output.

9. A named entity recognition model training device, characterized in that, The device includes: The association establishment module is used to establish the association between the multi-source heterogeneous training dataset, entity coverage vector, and source dataset identifier; the entity coverage vector is used to characterize the entity types contained in the multi-source heterogeneous training dataset. The large language model processing module is used to load and freeze all parameters of a pre-trained large language model as the base model. An adapter building module is used to build a shared low-rank adapter and an entity-specific low-rank adapter; wherein, the shared low-rank adapter is used to capture general knowledge across datasets and across entity types, and the entity-specific low-rank adapter is used to capture recognition patterns for specific entity types; The shared low-rank adapter fine-tuning module is used to train the adapter based on the correlation relationship, the shared low-rank adapter, and the base model to obtain the shared adapter. The entity-specific low-rank adapter fine-tuning module is used to train the adapter based on the association relationship, the entity-specific low-rank adapter, and the base model to obtain the entity-specific adapter for each entity type. The named entity recognition model generation module is used to save the shared adapter, the entity-specific adapter for each entity type, and the base model to obtain a pre-trained named entity recognition model.

10. A named entity recognition device, characterized in that, The device includes: A loading module is used to load a pre-trained named entity recognition model, the named entity recognition model including a shared adapter, an entity-specific adapter for each entity type, and a base model; wherein, the base model is obtained by freezing all parameters of a pre-trained large language model; the shared adapter and the entity-specific adapter are trained by the named entity recognition model training method according to any one of claims 1-7; The receiving module is used to receive named entity recognition requests, wherein the named entity recognition requests carry the actual requirements of the current inference task and the text to be recognized; The adapter selection module is used to select a target entity-specific adapter that matches the actual needs of the current inference task from the entity-specific adapters of each entity type. The fusion module is used to fuse the base model, the target entity-specific adapter, and the shared adapter to generate the final target model for inference. The recognition result output module is used to input the text to be recognized into the target model for forward propagation calculation, and output each recognized entity fragment in the text to be recognized and its corresponding entity type label.

Images