VTE clinical named entity recognition method based on large model

Abstract

The invention relates to a VTE clinical named entity recognition method based on a large model, and the method comprises the following steps: obtaining and cleaning a VTE medical record text data set, and then converting a data format; deploying the large model on a local server by using a tool, retrieving a data set related to the VTE field in a data warehouse by means of an existing tool, and selecting an original data set by calculating a similarity score between a specified query word and the data set; adopting multiple strategies to generate enhanced data; then generating a context representation matrix through a pre-training encoder; and two feedforward networks are used to calculate vector representations of a starting position and an ending position of a text fragment respectively, then a relative position coding matrix is added to jointly calculate an entity type score of the text fragment, and the entity type with the maximum score is selected as a final entity type prediction result of the text fragment. According to the method, the problems of data scarcity and category imbalance in traditional clinical named entity recognition are solved.

Description

Technical Field

[0001] This invention relates to the field of natural language processing, and in particular to a method for clinical named entity recognition based on a large model for VTE. Background Technology

[0002] Venous thromboembolism (VTE) is a significant public health burden, particularly impacting vulnerable populations such as postoperative patients, cancer patients, and those with limited mobility. This disease can lead to complications such as deep vein thrombosis and pulmonary embolism, and in severe cases, even death. Therefore, extracting structured information from electronic medical records is crucial for disease diagnosis and treatment, and clinical named entity recognition (NER) technology provides an effective solution for the structured processing of clinical text. In clinical settings, doctors spend considerable time organizing medical records, filling out quality control reports, or retrieving patient histories; researchers manually screen literature and extract clinical research data, which is not only time-consuming (e.g., entity extraction from a single article can take several hours) but also prone to omissions or errors due to subjective judgment. Deep learning technology allows healthcare workers to escape tedious information processing. Well-trained medical models can accurately extract key information from vast amounts of data, such as extracting medication lists for patients with "diabetic nephropathy" from 1,000 discharge summaries, or screening relevant studies on "PD-1 inhibitor treatment for lung cancer" from 100,000 medical articles, increasing efficiency by tens of times while avoiding human oversight.

[0003] Deep learning methods, which have developed rapidly in recent years, utilize neural network models to learn entity recognition patterns directly from text. By pre-training on large-scale corpora, they can capture rich contextual information. Fine-tuning on labeled data in the medical field can significantly improve the performance of medical named entity recognition. Large language models such as BenTsao-7B, DoctorGPT-7B, and DoctorGLM-6B have achieved good results in the medical natural language processing field. However, due to limited labeled data, complex language structures, nested entity expressions, and imbalanced entity distribution, clinical named entity recognition in the VTE field still faces many challenges, mainly in the following three aspects: scarce labeled data resources, class imbalance in existing data, and structural complexity of clinical texts; this requires further technical exploration. Summary of the Invention

[0004] In view of the above-mentioned problems in the existing technology, the technical problem to be solved by the present invention is: how to train a model under the condition of scarce data to improve the accuracy of named entity recognition in the VTE domain.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0006] A clinical named entity recognition method for VTE based on a large model includes the following steps:

[0007] S1: Obtain the publicly available VTE medical record text dataset, which includes data of five entity types: body parts, disease symptoms, disease names, surgical procedures, and treatment drugs.

[0008] S2: Use NLTK and spaCy tools to clean the VTE medical record text data in sequence, and then convert the cleaned VTE medical record text data into a structured JSON format;

[0009] S3: Deploy large models on a local server using the Ollam tool;

[0010] S4: Use the DataFinder tool to perform semantic retrieval in the Hugging Face data warehouse to obtain textual data of relevant entity types in the VTE domain as supplementary data;

[0011] S5: Merge the data for each entity type and the corresponding supplementary data in the VTE medical record text dataset to obtain the original dataset;

[0012] Several core strategies are selected as the strategies to be used by the large model when processing the original dataset. Then, the original dataset in JSON format is input into the large model. The large model processes the original dataset in JSON format according to the core strategies and outputs the data-augmented dataset D.

[0013] S6: Select n words or characters from D to form the input text sequence X, where X = {x1, x2, ..., xn}. n}, where x n Let X represent the nth term. X is processed by a pre-trained encoder to obtain the context representation matrix H corresponding to X. X And H X ∈R n×v , where v represents the token representation dimension of each term; the pre-trained encoder is a BERT, RoBERTa, or GPT series;

[0014] S7: Text data x located at any position i in X i And the text data x at position j j And i>j, x i and x j Inputting each corresponding feedforward network (FNN) yields x as the output. i The corresponding vector representation q i,α , and x j The corresponding vector representation kj,α Where α represents the entity type, the calculation expression is as follows:

[0015] q i,α =W q,α h i +b q,α

[0016] k j,α =W k,α h j +b k,α

[0017] Among them, h i and h j Let h represent the context embedding vectors at positions i and j, respectively. i ,h j ∈H X W q,α and W k,α They represent q respectively. i,α The corresponding query weight matrix and k j,α The corresponding key weight matrix; b q,α and b k,α They represent q respectively. i,α The corresponding query bias vector and k j,α The corresponding key bias vector;

[0018] S8: Calculate the score s of the corresponding entity type of the text fragments between i and j. α(i,j) The calculation expression is as follows:

[0019]

[0020] in, q i,α The transpose of Rel j-i The encoding matrix representing the relative positions of i and j is expressed as follows:

[0021]

[0022] Where d represents the dimension, and R0 represents the rotation matrix corresponding to the first group of two-dimensional components of the word vector. This represents the rotation matrix corresponding to the last group of two-dimensional components of the word vector. Represents the number of groups, R0 and The calculation expression is as follows:

[0023]

[0024] Where θ0 represents the preset angle parameter of the first set of matrices. Indicates the first Preset angle parameters for the group matrix;

[0025] Iterate through all positions in X to obtain the score for each entity type for the text fragment between any two positions, and then set D to the scores of all entity types. score ;

[0026] S9: Construct the loss function Loss, and then... score The score corresponding to each entity type is used as a separate input. The large model is trained using the AdamW optimizer with a loss function (Loss). The weight matrix and bias vector corresponding to each entity type are updated via gradient descent and backpropagation. The weight matrix includes the query weight matrix W′. q,α Bond weight matrix W′ k,α The bias vector includes the query bias vector b′. q,α Bond bias vector b′ k,α Training stops when the loss stops changing or the maximum number of iterations is reached, yielding W′ for each entity type after training. k,α , W′ q,α and b′ q,α , b′ k,α The expression for calculating Loss is as follows:

[0027]

[0028] Where q and k represent the starting and ending vector representations of any text data span, respectively, and P α Let Q represent the span set of all entities of type α. α Let s represent the span set of all entities whose type is not α. α (q,k) represents the score when the text segment with a span of s[q:k] is α;

[0029] S10: Define a score threshold, use the weight matrix and bias vector corresponding to each trained entity type α, calculate the score of the new entity type corresponding to α using the method described in S8, discard all new entity type scores that are less than the score threshold, and select the maximum value from the remaining new entity type scores to save.

[0030] Iterate through all entity types and obtain the maximum score of the new entity type for each entity type. The maximum scores of the new entity type for the five types are as follows: body part score s BOD(i,j) Disease symptom score SYM(i,j) Surgical procedure score OPE(i,j) Disease name score DIS(i,j) and treatment drug score DRU(i,j) , will s BOD(i,j) s SYM(i,j) s OPE(i,j)s DIS(i,j) and s DRU(i,j) The entity type corresponding to the maximum value in the value is used as the final entity type prediction result for the text segment between i and j.

[0031] Preferably, the step of obtaining supplementary data in S4 is as follows:

[0032] S41: Create a prompt for a certain entity type, wherein the prompt format is [(Task Description),(Sample),(Answer)];

[0033] S42: Input the prompt as a specific query into the large model, output hypothetical text data about the prompt, and use the hypothetical text data about the prompt as an extended query;

[0034] Then, the DataFinder tool was used to perform semantic retrieval related to the prompt in the Hugging Face data warehouse, retrieving several named entity recognition task datasets related to the VTE domain. The BM25 algorithm was applied to calculate the similarity score between the extended query and the retrieved named entity recognition task datasets related to the VTE domain. The expression for calculating the score is as follows:

[0035] Score = BM25(D,Q)·log 10 (downloads+1)

[0036] in, q i Indicates a term, i.e., q i =x i |D| represents the length of the current document D, avgdl represents the average length of all documents in the document set, and TF(q) i D) represents term q i The number of occurrences in document D, IDF(q) i ) represents term q i The inverse document frequency, i.e. N represents the total number of documents, df(q i ) is a term containing q i The number of documents, k1 and b are both adjustable parameters, and downloads represents the number of times the dataset was downloaded;

[0037] S43: Sort all the obtained similarity scores in descending order, and select the VTE domain-related named entity recognition task datasets corresponding to the top-K similarity scores as supplementary data for a certain entity type.

[0038] Preferably, several core strategies in S5 include high-diversity few-sample hints, temperature annealing, and self-consistent decoding.

[0039] Compared with the prior art, the present invention has at least the following advantages:

[0040] This invention proposes a VTE named entity recognition method based on a large language model, which solves the problems of data scarcity and class imbalance in traditional clinical named entity recognition. Furthermore, it introduces relative positional encoding technology to improve the model's performance in nested entity recognition. The performance of this invention (ProDA-NER) outperforms current popular mainstream NER algorithms, laying a technical foundation for improving information extraction in medical scenarios and opening up possibilities for downstream applications in fields such as medical informatics and precision medicine.

[0041] 1. The dataset retrieval and dataset generation components in the data augmentation module improve the entity recognition performance of all models. By generating targeted supplementary samples using large models, it is highly effective in alleviating the data sparsity problem. The improvement is particularly significant in large medical models, with an increase of more than 3.5%. It overcomes the problem of scarce professional data and complex entity categories in named entity recognition tasks in the VTE field.

[0042] 2. The RoPE encoding technology plays a particularly significant role in improving the accuracy of entity boundary recognition. It effectively solves the problem of ambiguous entity positions in long texts. The precision, recall and F1 score of all models have increased significantly, and the F1 score has also improved. This shows that the module has a universal optimization effect on named entity recognition tasks and overcomes the shortcomings of existing technologies that lack the introduction of relative position information and cannot accurately capture the meaning of text.

[0043] 3. The system constructs a complete workflow covering training and inference. It supports adaptation to various mainstream large-scale models and allows for fine-tuning of the model by modifying prompts or task examples, thus facilitating optimization and improvement. Attached Figure Description

[0044] Figure 1 This forms the overall framework of the present invention.

[0045] Figure 2 This is a distribution chart of the number of entity types.

[0046] Figure 3 This is a performance comparison table between different methods.

[0047] Figure 4 The influence of entity length on the performance of the model of this invention.

[0048] Figure 5The table shows the comparison of ablation experiments, illustrating the impact of the three modules—dataset retrieval, dataset generation, and RoPE encoding—on the performance of the proposed model.

[0049] Figure 6 A schematic diagram of the input instruction format to support large models in Hugging Face. Detailed Implementation

[0050] The present invention will now be described in further detail.

[0051] This invention discloses a VTE named entity recognition method based on a large language model to provide clinical support services. The method includes the following steps: acquiring VTE medical record text data and cleaning it using NLTK and spaCy tools to convert the text and its corresponding entity annotations into structured JSON format; deploying large models such as DoctorGLM-6B, Falcon-7B, and ChatGLM3-6B on a local server using the Ollam tool; performing semantic retrieval in the Hugging Face data warehouse using the DataFinder tool to retrieve datasets that meet the specific needs of the VTE domain, calculating the similarity score (Score) between the query term and the retrieved dataset using the BM25 algorithm, and selecting a suitable dataset based on the score; integrating three core strategies—high-diversity few-shot hints, temperature annealing, and self-consistent decoding—to generate highly diverse sample data; and generating a context representation matrix H∈R using a pre-trained encoder. n×v Based on the context, the vector representations q of the entity's start and end positions are calculated using two feedforward networks, respectively. i,α and k j,α The relative position information s between coded position i and position j is calculated using the relative position coding matrix. α(i,j) Finally, a feedforward neural network is used to calculate the entity type score for each text span to determine whether it is an entity or an entity type.

[0052] See Figures 1-6 A clinical named entity recognition method for VTE based on a large model includes the following steps:

[0053] S1: Obtain the publicly available VTE medical record text dataset, which includes data of five entity types: body parts, disease symptoms, disease names, surgical procedures, and treatment drugs.

[0054] S2: Use NLTK and spaCy tools to clean the VTE medical record text data in sequence, and then convert the cleaned VTE medical record text data into structured JSON format; NLTK and spaCy tools are existing technologies;

[0055] S3: Deploy large models on a local server using the Ollam tool; Ollam tool is existing technology, and the large models are DoctorGLM-6B, Falcon-7B, ChatGLM3-6B, etc. One or more large models can be deployed. Depending on the purpose, a suitable large model is selected to process the input data. These large models are all known and publicly available models.

[0056] S4: Use the DataFinder tool to perform semantic retrieval in the Hugging Face data warehouse to obtain text data of relevant entity types in the VTE domain as supplementary data; both the DataFinder tool and the Hugging Face data warehouse are existing technologies; the text data of relevant entities in the VTE domain refers to the clinical named entity recognition task dataset related to VTE;

[0057] In step S4, the steps for obtaining supplementary data are as follows:

[0058] S41: Create a prompt for a certain entity type, wherein the prompt format is [(Task Description),(Sample),(Answer)];

[0059] S42: Input the prompt as a specific query into the large model, output hypothetical text data about the prompt, and use the hypothetical text data about the prompt as an extended query;

[0060] Then, the DataFinder tool was used to perform semantic retrieval related to the prompt in the Hugging Face data warehouse, retrieving several named entity recognition task datasets related to the VTE domain. The BM25 algorithm was applied to calculate the similarity score between the extended query and the retrieved named entity recognition task datasets related to the VTE domain. The expression for calculating the score is as follows:

[0061] Score = BM25(D,Q)·log 10 (downloads+1)

[0062] in, q i Indicates a term, i.e., q i =x i |D| represents the length of the current document D, avgdl represents the average length of all documents in the document set, and TF(q) i D) represents term q i The number of occurrences in document D, IDF(q)i ) represents term q i The inverse document frequency, i.e. N represents the total number of documents, df(q i ) is a term containing q i The number of documents, k1 and b are both adjustable parameters, and downloads represents the number of times the dataset has been downloaded; k1 is usually between 1.2 and 2.0, mainly limiting the maximum contribution of a word to the document score to prevent TF from growing indefinitely; b is usually set to 0.75, which introduces the document length factor into the scoring to solve the length insensitivity problem of TF-IDF; in addition, a higher value of downloads means a higher score;

[0063] S43: Sort all the obtained similarity scores in descending order, and select the VTE domain-related named entity recognition task datasets corresponding to the top-K similarity scores as supplementary data for a certain entity type.

[0064] S5: Merge the data for each entity type and the corresponding supplementary data in the VTE medical record text dataset to obtain the original dataset;

[0065] Several core strategies are selected as conditions for the large model to process the input data. Then, the original dataset in JSON format is input into the large model. The large model processes the original dataset in JSON format according to the core strategies and outputs the data-augmented dataset D.

[0066] Several core strategies in S5 include high-diversity few-sample hints, temperature annealing, and self-consistent decoding.

[0067] High diversity with few samples tip: In addition to the provided text examples, randomly introduce previously generated historical samples for data augmentation to improve diversity and avoid generating duplicate content.

[0068] Temperature annealing: When calling the model to generate, the temperature parameter is explicitly set, gradually increasing from low temperature (preferring deterministic output) to high temperature (encouraging diverse exploration); this adjustment mechanism promotes the diversity of generated content while maintaining output quality.

[0069] Self-consistent decoding: Given that large language models may generate non-unique or incorrect outputs for the same input, we select the answer with the highest frequency as the consensus output by statistically analyzing all output results corresponding to each independent input; if multiple candidate answers have the same frequency, we select the candidate answer with the shortest length as the final result.

[0070] S6: Select n words or characters from D to form the input text sequence X, where X = {x1, x2, ..., xn}.n}, where x n Let X represent the nth term. X is processed by a pre-trained encoder to obtain the context representation matrix H corresponding to X. X And H X ∈R n×v Where v represents the token representation dimension of each word; the pre-trained encoder is BERT, RoBERTa, or GPT series; the pre-trained encoder is existing technology; unlike traditional static word vectors (such as Word2Vec), this representation is dynamically generated by combining the context information of the text, and the same token will get different vectors in different contexts, which is more in line with the ambiguous characteristics of natural language.

[0071] S7: Text data x located at any position i in X i And the text data x at position j j And i>j, x i and x j The inputs are their respective feedforward networks (FNNs), which are existing technologies, and the output is x. i The corresponding vector representation q i,α , and x j The corresponding vector representation k j,α Where α represents the entity type, the calculation expression is as follows:

[0072] q i,α =W q,α h i +b q,α

[0073] k j,α =W k,α h j +b k,α

[0074] Among them, h i and h j Let h represent the context embedding vectors at positions i and j, respectively. i ,h j ∈H X W q,α and W k,α They represent q respectively. i,α The corresponding query weight matrix and k j,α The corresponding key weight matrix; b q,α and b k,α They represent q respectively. i,α The corresponding query bias vector and k j,α The corresponding key bias vector;

[0075] S8: Calculate the score s of the corresponding entity type of the text fragments between i and j. α(i,j)The calculation expression is as follows:

[0076]

[0077] in, q i,α The transpose of Rel j-i The encoding matrix representing the relative positions of i and j is expressed as follows:

[0078]

[0079] Where d represents the dimension, and R0 represents the rotation matrix corresponding to the first group of two-dimensional components of the word vector. This represents the rotation matrix corresponding to the last group of two-dimensional components of the word vector. Represents the number of groups, R0 and The calculation expression is as follows:

[0080]

[0081] Where θ0 represents the preset angle parameter of the first set of matrices. Indicates the first Preset angle parameters for the group matrix;

[0082] The scores for positions i and j are calculated using the relative position encoding matrix, i.e. R i and R j It is a rotation matrix that encodes the relative positional relationship between positions i and j. For example, when the word vector dimension d = 2, R i Rotation matrix definition: Similarly, R j Rotation matrix definition: Relative position encoding matrix j-i Defined as These rotation matrices provide rotation-invariant positional encodings, enabling the model to more effectively distinguish spatial relationships between distant markers in long text sequences. Furthermore, to support scalable integration with high-dimensional representations, the RoPE framework embeds positional information into a character vector at position m using the following formula: in, It is a rotation position encoding matrix, W q,k A projection matrix representing a query or key;

[0083] Iterate through all positions in X to obtain the score for each entity type for the text fragment between any two positions, and then set D to the scores of all entity types. score ;

[0084] S9: Construct the loss function Loss, and then...score The score corresponding to each entity type is used as a separate input. The large model is trained using the AdamW optimizer with a loss function (Loss). The weight matrix and bias vector corresponding to each entity type are updated via gradient descent and backpropagation. The weight matrix includes the query weight matrix W′. q,α Bond weight matrix W′ k,α The bias vector includes the query bias vector b′. q,α Bond bias vector b′ k,α Training stops when the loss stops changing or the maximum number of iterations is reached, yielding W′ for each entity type after training. k,α , W′ q,α and b′ q,α , b′ k,α The expression for calculating Loss is as follows:

[0085]

[0086] Where q and k represent the starting and ending vector representations of any text data span, respectively, and P α Let Q represent the span set of all entities of type α. α Let s represent the span set of all entities whose type is not α. α (q,k) represents the score when the text segment with a span of s[q:k] is α;

[0087] S10: Define a score threshold, use the weight matrix and bias vector corresponding to each trained entity type α, calculate the score of the new entity type corresponding to α using the method described in S8, discard all new entity type scores that are less than the score threshold, and select the maximum value from the remaining new entity type scores to save.

[0088] Iterate through all entity types and obtain the maximum score of the new entity type for each entity type. The maximum scores of the new entity type for the five types are as follows: body part score s BOD(i,j) Disease symptom score SYM(i,j) Surgical procedure score OPE(i,j) Disease name score DIS(i,j) and treatment drug score DRU(i,j) , will s BOD(i,j) s SYM(i,j) s OPE(i,j) s DIS(i,j) and s DRU(i,j)The entity type corresponding to the maximum value in the value is used as the final entity type prediction result for the text segment between i and j. In rare cases, the scores may be the same. There is a predefined list of entity type priorities (dis>bod>sym>ope>dru). When two entity types have the same score, the priority will be used to determine which type the text data belongs to.

[0089] Experimental content and results

[0090] The dataset chosen for this invention comes from a university-affiliated cancer hospital and originates from clinical medical records related to venous thromboembolism (VTE) in the electronic medical record (EMR) system. This dataset contains five medical entity categories: Body Parts (BOD), with 3270 entities; Symptoms of Disease (SYM), with 8547 entities; Disease Names (DIS), with 6428 entities; Surgical Procedures (OPE), with 1302 entities; and Drug Reagents (DRU), with 189 entities. The entity distribution in this dataset exhibits a significant imbalance.

[0091] 80% was selected as training data, and the remaining 20% ​​was selected as test data.

[0092] The evaluation metrics are precision, recall, and F1-score. Precision is based on a formula... Recall rate is based on a formula F1 score is based on the formula In this matrix, TP represents true positives, where the model correctly predicts a positive sample as positive; FP represents false positives, where the model incorrectly predicts a negative sample as positive; and FN represents false negatives, where the model incorrectly predicts a positive sample as negative. These three metrics together form the basis of the "confusion matrix," and their calculation provides a comprehensive measure of the model's performance.

[0093] To evaluate the performance of Named Entity Recognition (NER) models, this invention selects representative benchmark models covering both traditional methods and recent state-of-the-art methods:

[0094] Sequence-to-Set: This paper proposes a sequence labeling framework based on set learning, which reconstructs NER into a set prediction task to improve the performance of multi-label classification.

[0095] LEBERT: Combines lexical knowledge with a pre-trained BERT model to improve NER performance by incorporating entity information from an external dictionary.

[0096] W2NER: Proposes a word-level NER framework that integrates pre-trained word embeddings and contextual information in a unified architecture, and performs well on datasets with large vocabularies.

[0097] BS: Focuses on accurate detection of entity boundaries, achieved through explicit boundary labeling. This model employs a dual-task strategy to optimize boundary prediction and entity annotation, significantly improving recognition accuracy.

[0098] Triaffine: Captures complex entity interactions using ternary relations. Its triple-based structure excels in multi-level, multi-relational NER tasks, demonstrating strong performance.

[0099] BINDER enhances NER capabilities by optimizing relationship modeling and boundary-aware techniques. Compared to traditional methods, its architecture efficiently extracts complex entity relationships while minimizing redundancy.

[0100] In benchmark models, architectures such as LEBERT, Sequence-to-Set, and W2NER typically exhibit high precision but relatively low recall and F1 scores. This pattern suggests limitations in comprehensively recognizing complex entities and processing long clinical texts, often prioritizing precision over recall. Although models like BS, Triaffine, and BINDER show slight improvements in recall, their overall performance still lags behind large language models.

[0101] Experiments show that the method based on large models and employing the ProDA-NER framework exhibits significantly improved and more balanced performance. We selected five representative general large language models (LLMs) and three domain-specific medical language models as base models, based on their diversity in modality, language, and task orientation.

[0102] General Large Language Model:

[0103] GPT-4o: Supports cross-modal understanding and generation.

[0104] Hunyuan-7B: Developed by Tencent, it is specifically optimized for Chinese semantic processing.

[0105] ChatGLM3-6B: An open-source model focused on semantic understanding and reasoning.

[0106] Llama 2-7B: Released by Meta, it features multilingual processing and command following capabilities.

[0107] Falcon-7B: From the Technology Innovation Institute, it uses a lightweight Transformer architecture and is optimized for efficient text generation.

[0108] Medical-specific language models:

[0109] DoctorGPT-7B: Proficient in analyzing English medical literature, highly aligned with international clinical guidelines, and supports multilingual applications.

[0110] BenTsao-7B: Based on large-scale Chinese medical texts and clinical case corpora, it focuses on knowledge-based question-answering tasks that combine traditional Chinese and Western medicine.

[0111] DoctorGLM-6B is an optimized version of the general model ChatGLM3, pre-trained on a large-scale Chinese clinical corpus, and fine-tuned for diagnostic reasoning and medical dialogue tasks.

[0112] Performance differences among LLMs can be attributed to variations in model size, pre-training data (e.g., domain-specific versus general-purpose data), and word segmentation strategies, all of which affect semantic representation and entity boundary detection. In general, medical-specific models have comparable parameter sizes to general-purpose LLMs (e.g., Llama 2-7B, Falcon-7B), but through targeted training on medical corpora, they achieve breakthroughs in both precision and recall. Furthermore, the study observed a positive correlation between model capability and optimization direction: architectural upgrades of general-purpose LLMs (e.g., GPT-4o) and domain refinement of specialized models significantly improve nested entity extraction capabilities, highlighting the crucial role of targeted optimization in medical named entity recognition tasks.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A large model-based VTE clinical named entity recognition method, characterized by: Comprising the following steps: S1: Obtain a public VTE medical record text dataset, which includes data of five entity types, including body parts, disease symptoms, disease names, surgical operations, and treatment drugs; S2: Use the NLTK tool and the spaCy tool to sequentially clean the VTE medical record text data, and then convert the cleaned VTE medical record text data into a structured JSON format; S3: Deploy a large model on a local server using the Ollama tool; S4: Use the DataFinder tool to perform semantic retrieval in the Hugging Face data warehouse to obtain text data of related entity types in the VTE field as supplementary data; S5: Merge the data of each entity type in the VTE medical record text dataset and the corresponding supplementary data of each type to obtain an original dataset; Select several core strategies as conditions when the large model processes input data, and then input the original dataset in JSON format into the large model. The large model processes the original dataset in JSON format according to the core strategies, and outputs a dataset D after data augmentation; S6: Select n words or characters from D to form an input text sequence X, and X = {x1, x2,..., xn}, where x n represents the nth term, and X is processed through a pre-trained encoder to obtain a context representation matrix H n corresponding to X, and H X X ∈R n×v , where v represents the token representation dimension of each term; the pre-trained encoder is BERT, RoBERTa or GPT series;​ S7: text data x located at any position i in X i and text data x at position j j , and i > j, x i and x j are input into respective feedforward networks FNN, and x i corresponding vector representations q i,α , and x j corresponding vector representations k j,α , where α represents an entity type, and the calculation expression is as follows: q i,α = W q,α h i + b q,α k j,α = W k,α h j + b k,α Among them, h i and h j Let h represent the context embedding vectors at positions i and j, respectively. i ,h j ∈H X W q,α and W k,α They represent q respectively. i,α The corresponding query weight matrix and k j,α The corresponding key weight matrix; b q,α and b k,α They represent q respectively. i,α The corresponding query bias vector and k j,α The corresponding key bias vector; S8: Calculate the score s of the entity type corresponding to the text segment between i and j α(i,j) The expression is calculated as follows: wherein denotes the transpose matrix of q i,α denotes the transpose matrix of Rel j-i denotes the relative position encoding matrix for i and j, expressed as follows: wherein d represents the dimension, R0represents a rotation matrix corresponding to the first group of two-dimensional components of the word vector, represents a rotation matrix corresponding to the last group of two-dimensional components of the word vector, represents the number of groups, R0and The calculation expression of is as follows: Wherein, θ0 represents a preset angle parameter of the first group matrix, represents a preset angle parameter of the first group matrix. traversing all positions in X, obtaining the score of each entity type corresponding to the text segment between any two positions, the set D composed of the scores of all entity types score ; S9: Construct the loss function Loss, and then... score The score corresponding to each entity type is used as a separate input. The large model is trained using the AdamW optimizer with a loss function (Loss). The weight matrix and bias vector corresponding to each entity type are updated via gradient descent and backpropagation. The weight matrix includes the query weight matrix W′. q,α Bond weight matrix W′ k,α The bias vector includes the query bias vector b′. q,α Bond bias vector b′ k,α Training stops when the loss stops changing or the maximum number of iterations is reached, yielding W′ for each entity type after training. k,α , W′ q,α and b′ q,α , b′ k,α The expression for calculating Loss is as follows: where q, k represent the start vector representation and the end vector representation of any one text data span, respectively, P α denotes the span set of all entity types α, Q α denotes the span set of all entity types not α, s α (q, k) represents the score when the text segment of span s[q: k] is α. S10: Define a score threshold, use the trained weight matrix and bias vector corresponding to each entity type α, calculate the new entity type score corresponding to α using the method described in S8, discard all new entity type scores less than the score threshold, and select the maximum value from the remaining new entity type scores to save; The maximum value of the new entity type score of each entity type is obtained by traversing all entity types, and the maximum values of the new entity type scores of the five types are body part score s BOD(i,j) , disease symptom score s SYM(i,j) , surgical operation score s OPE(i,j) , disease name score S DIS(i,j) and treatment drug score S DRU(i,j) , and the maximum value of S BOD(i,j) , s SYM(i,j) , s OPE(i,j) , s DIS(i,j) and s DRU(i,j) corresponding to the entity type is taken as the final entity type prediction result corresponding to the text segment between i and j.

2. The large model-based VTE clinical named entity recognition method of claim 1, wherein: In S4, the steps to obtain supplementary data are as follows: S41: Establish a prompt for a certain entity type, the prompt format is [(Task Description), (Sample), (Answer)]; S42: Input the prompt into the large model as a special query, output the hypothetical text data about the prompt, and use the hypothetical text data about the prompt as an extended query; Then use the DataFinder tool to perform semantic retrieval related to the prompt in the Hugging Face data warehouse, retrieve several VTE field related named entity recognition task datasets, apply the BM25 algorithm to calculate the similarity score Score between the extended query and the retrieved several VTE field related named entity recognition task datasets, and the calculation expression of Score is as follows: Score = BM25(D, Q) - log 10 (downloads + 1) wherein, q i represents a term, i.e. q i = x i, |D| represents the length of the current document D, avgdl represents the average length of all documents in the document collection, TF(q i , D) represents the number of occurrences of the term q i in the document D, IDF(q i ) represents the inverse document frequency of the term q i , i.e. N represents the total number of documents, df(q i ) is the number of documents containing the term q i , k1 and b both represent adjustable parameters, and downloads represents the number of times the data set has been downloaded; S43: Arrange all the similarity scores in descending order, select the VTE field related named entity recognition task datasets corresponding to the top K similarity scores as the supplementary data of a certain entity type.

3. The large model-based VTE clinical named entity recognition method of claim 2, wherein: The several core strategies in S5 include high diversity few-shot prompt, temperature annealing, and self-consistency decoding.

Images