Traditional Chinese medicine term standardization algorithm based on multi-task learning and instruction fine tuning

Abstract

The invention relates to the technical field of natural language processing and traditional Chinese medicine, in particular to a traditional Chinese medicine terminology standardization algorithm based on multi-task learning and instruction fine tuning, and the algorithm comprises the steps: firstly constructing a training data set containing a target domain corpus and a standard terminology association corpus, and selecting a deep learning model pre-trained by a large-scale general text; obtaining at least two domain adaptive models through domain adaptive pre-training; screening the model with better performance to carry out single-task fine tuning and multi-task instruction fine tuning in sequence to obtain a term standardization model; after an entity is extracted from a traditional Chinese medicine clinical text, a standard term with a basis is output through three mapping modes. The traditional Chinese medicine term standardization accuracy and efficiency are improved, the problems that a traditional method is high in labor cost and poor in consistency are solved, and support is provided for traditional Chinese medicine data integration and application.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing and traditional Chinese medicine (TCM) technology, and in particular to a TCM terminology standardization algorithm based on multi-task learning and instruction fine-tuning. Background Technology

[0002] Traditional Chinese medicine (TCM), as a precious cultural heritage of my country, has a vast and complex knowledge system. In recent years, in order to promote the modernization and internationalization of TCM, my country has successively issued a series of terminology standards, such as GB / T 16751.1,2,3-2021 "Clinical Terminology of TCM Diagnosis and Treatment", and constructed a large-scale terminology system such as TCMMLS.

[0003] However, due to historical reasons and human factors, existing standards and terminology systems still suffer from numerous problems such as ambiguous terms, classification errors, and inconsistent expressions. These issues severely hinder the integration, mining, and information application of traditional Chinese medicine data.

[0004] Currently, solving the problem of terminology standardization mainly relies on experts manually constructing thesaurus. This method not only consumes a lot of manpower and time, but also makes it difficult to objectively quantify the semantic similarity between terms, and cannot meet the processing needs of massive text data. Although some studies have attempted to use automated methods, such as BERT-based classification models for symptom terminology standardization, these methods generally have limitations: the task is singular, failing to systematically cover multiple types of terms such as diseases, prescriptions, drugs, and syndrome types; and the models have not been sufficiently trained on large-scale, high-quality corpora in the field of traditional Chinese medicine, resulting in limited generalization ability and accuracy.

[0005] Therefore, there is an urgent need in this field for an automated terminology standardization solution that can deeply integrate knowledge from the field of traditional Chinese medicine, cover a comprehensive range of terminology types, and have strong generalization capabilities. Summary of the Invention

[0006] The purpose of this invention is to provide a standardization algorithm for traditional Chinese medicine terminology based on multi-task learning and instruction fine-tuning, so as to solve the problems existing in the prior art.

[0007] To achieve the above objectives, this invention provides a standardization algorithm for TCM terminology based on multi-task learning and instruction fine-tuning, comprising the following steps: S1: Constructing a training dataset, wherein the training dataset includes a target domain corpus dataset and a standard terminology-related corpus dataset; the target domain corpus dataset covers text resources from different periods within the domain, and the standard terminology-related corpus dataset is constructed based on recognized authoritative standards and professional materials in the domain, including multiple core terms in the domain and positive and negative samples generated based on the core terms, and the standard terminology-related corpus dataset is divided into a training set, a validation set, and a test set according to a preset ratio; S2: Selecting an initial pre-trained language model, wherein the initial pre-trained language model is a deep learning model adapted to the needs of target language text processing, and has been pre-trained using large-scale general texts; S3: Implementing domain-adaptive pre-training based on the target domain corpus... Texts from different periods in the dataset are used to pre-train an initial pre-trained language model, resulting in at least two domain-adaptive models. During the domain-adaptive pre-training process, tasks for predicting inter-sentence relationships are removed, and continuous text of a specified length that does not span documents is selected as input. A dynamic masking strategy is used to process the vocabulary in the input text. S4: Model selection and multi-stage fine-tuning. The domain-adaptive models obtained in step 3 are evaluated using a validation set, and the model with better performance is selected as the base model. Based on the terminology-related dataset of the target domain, the base model is sequentially fine-tuned using single-task and multi-task instructions to obtain a terminology standardization model for the target domain. S5: Term standardization is performed. Entities to be standardized are extracted from professional texts in the target domain, and the terminology standardization model obtained in step 4 is used to map these entities, outputting the corresponding standard terms.

[0008] Further, the professional text in the target domain mentioned in step 5 is TCM clinical text; when extracting entities to be standardized from TCM clinical text, an entity recognition model is used. The input of the entity recognition model is TCM clinical text, and the output is a text sequence of entities to be standardized in the TCM field, including diseases, prescriptions, drugs, syndromes, and symptoms; the mapping processing of the entities to be standardized includes three modes: standard mode, which directly maps entities to predefined TCM standard terms; synonym mode, which maps entities as synonyms to corresponding TCM standard terms based on a TCM thesaurus; and similarity_score mode, which maps entities to the closest TCM standard terms based on the similarity score calculated by the algorithm between the entity and the TCM standard terms, and outputs the mapping basis simultaneously when outputting standardized terms. The mapping basis includes the mapping mode and the source of the standard terms. If it is similarity_score mode, it also includes the corresponding similarity score.

[0009] Furthermore, the terminology-related datasets in the target domain mentioned in step 4 include a dataset of synonyms, hyponyms, and questions and answers in the field of traditional Chinese medicine (TCM). During the single-task fine-tuning and multi-task instruction fine-tuning processes, a Siamese network architecture based on Sentence-BERT is used to encode individual sentences as vectors, eliminating the need for sentence pairs as model input. During instruction fine-tuning, the input data includes text input information and corresponding task instruction information. The task instruction information is used to instruct the model to complete specific tasks related to TCM terminology standardization, including terminology matching, terminology hierarchical association recognition, and semantic understanding of TCM knowledge questions and answers.

[0010] Furthermore, during the multi-task instruction fine-tuning process described in step 4, synonym term matching task data, hyponym term association task data, and TCM question bank question answering task data are simultaneously input, enabling the model to learn general TCM text semantic representations through collaborative training of multiple tasks; model performance evaluation indicators include accuracy, F1 score, and Pearson correlation coefficient and Spearman correlation coefficient based on cosine distance, Euclidean distance, and Manhattan distance.

[0011] Furthermore, in step 4, when evaluating model performance using the validation set, for the synonym terminology dataset, accuracy is used as the core evaluation metric. The accuracy is determined by calculating the proportion of correctly predicted positive examples (True Positives) to all actual positive examples (the sum of True Positives and False Negatives). For the hyponym terminology dataset and the traditional Chinese medicine question-and-answer dataset, the cosine similarity between sample instances is first calculated, and then the Pearson correlation coefficient between the calculated cosine similarity and the label is used as the core evaluation metric. The correlation coefficients based on Euclidean distance and Manhattan distance are both determined by calculating the Pearson correlation coefficient between the corresponding distance between sample instances and the label.

[0012] Furthermore, the target language mentioned in step 2 is Chinese; the deep learning model adapted to the needs of Chinese text processing is the RoBERTa-wwm-ext-large model, and the RoBERTa-wwm-ext-large model has 325M parameters and has completed large-scale general text pre-training on 5.4B tokens containing Wikipedia text.

[0013] Further, the specified length in step 3 is 512; the dynamic masking strategy is to randomly mask words in the input text during each training iteration, and the masked words are different in different training iterations; the at least two domain adaptation models are specifically the Pre-TCM model based on traditional Chinese medicine text pre-training and the Pre-MCM model based on modern Chinese medicine text pre-training. In step 4, the Pre-MCM model is determined to be superior through validation set evaluation and is used as the base model.

[0014] Furthermore, the multiple core terms mentioned in step 1 include disease, symptom, syndrome, prescription, and drug terms in the field of traditional Chinese medicine; among the positive and negative samples, the positive samples are manually labeled term association samples, specifically 3037 pairs of synonym term association samples determined based on authoritative standards and professional data, and the negative samples are non-associated term samples that are not related to the core terms and are automatically constructed by the algorithm, and the number of negative samples is equal to the number of positive samples.

[0015] Furthermore, the multiple core terms mentioned in step 1 include disease, symptom, syndrome, prescription, and drug terms in the field of traditional Chinese medicine; among the positive and negative samples, the positive samples are manually labeled term association samples, specifically 3037 pairs of synonym term association samples determined based on authoritative standards and professional data, and the negative samples are non-associated term samples that are not related to the core terms and are automatically constructed by the algorithm, and the number of negative samples is equal to the number of positive samples.

[0016] Another aspect of this invention provides a terminology standardization system based on multi-task learning and instruction fine-tuning, including a dataset construction module for generating a corpus dataset of traditional Chinese medicine texts containing both traditional and modern TCM texts, and a standard terminology association corpus training set, validation set, and test set divided in a 7:2:1 ratio, containing terms such as diseases and symptoms, and an equal number of positive and negative samples; an initial model selection module for calling a RoBERTa-wwm-ext-large Chinese model with 325M parameters and pre-trained with 5.4B token general text; a domain-adaptive pre-training module for loading TCM corpus and pre-training the initial model to obtain Pre-TCM and Pre-MCM models, with built-in task removal, text length filtering, and dynamic masking units; a model selection and fine-tuning module for evaluating and selecting Pre-MCM as the base model, through single-task fine-tuning and multi-task instruction fine-tuning, with fine-tuning adopting the Siamese architecture of Sentence-BERT; and an entity extraction and standardization module, including an entity recognition unit and a standardization mapping unit.

[0017] This invention provides a standardization algorithm for traditional Chinese medicine terminology based on multi-task learning and instruction fine-tuning, which has the following beneficial effects: 1. By employing a training paradigm that combines domain-adaptive pre-training with multi-task instruction fine-tuning, the model has gained a deep understanding of the semantics and professional knowledge of traditional Chinese medicine (TCM) terminology. The final model achieves significantly higher accuracy on the test set than general-purpose models, realizing high-precision, automated standardization of TCM terminology and greatly reducing manual intervention.

[0018] 2. An authoritative dataset covering core terminology categories such as diseases, symptoms, syndromes, prescriptions, and drugs has been constructed. It innovatively introduces a variety of standardized mapping modes, such as standard mapping, synonym mapping, and similarity mapping, which can flexibly cope with the complex and diverse terminology expressions in clinical practice. The output results are accompanied by mapping evidence, and have high practicality and interpretability.

[0019] 3. The high-quality standardized data produced by this application can directly serve advanced applications such as the construction of TCM knowledge graphs, clinical decision support, smart medical systems, and enhanced retrieval generation, providing key technical support for the data-driven and intelligent development of TCM.

[0020] 4. By pre-training the model on large-scale traditional and modern TCM corpora, the model fundamentally understands the unique expressions and contexts of TCM texts, overcoming the problem of general models being unsuitable for specialized fields. Combined with multi-task instruction learning, the model can simultaneously handle multiple tasks such as terminology matching and relation recognition, exhibiting better robustness. Attached Figure Description

[0021] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 A flowchart of a standardization algorithm for traditional Chinese medicine terminology based on multi-task learning and instruction fine-tuning provided by this invention; Figure 2 A schematic diagram illustrating the construction of a standard term association corpus dataset for a TCM terminology standardization algorithm based on multi-task learning and instruction fine-tuning provided by this invention; Figure 3 A schematic diagram of the Siamese architecture of Sentence-BERT, a standardization algorithm for TCM terminology based on multi-task learning and instruction fine-tuning, provided by this invention. Figure 4 This is a schematic diagram of the core logic for model training and evaluation of a standardization algorithm for TCM terminology based on multi-task learning and instruction fine-tuning provided by the present invention. Figure 5 The graph shows the training accuracy variation of the Pre-MCM Fine-tuning model, which is based on a multi-task learning and instruction fine-tuning algorithm for standardizing TCM terminology provided by this invention, on a synonym dataset. Detailed Implementation

[0022] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0023] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0024] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0025] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0026] This embodiment provides a standardization algorithm for TCM terminology based on multi-task learning and instruction fine-tuning, including the following steps: S1 constructing a training dataset, S2 selecting an initial pre-trained language model, S3 implementing domain-adaptive pre-training, S4 model selection and multi-stage fine-tuning, and S5 performing terminology standardization.

[0027] See Figure 1As can be seen, each module is used to ensure that the training dataset contains both target domain corpus and standard term association corpus, providing the domain text foundation for the model and clarifying term association rules through positive and negative samples; the initial model must have Chinese text processing capabilities and undergo large-scale general pre-training to ensure basic semantic understanding performance; domain adaptive pre-training adapts the model to the characteristics of the traditional Chinese medicine field through targeted text input and task adjustment; multi-stage fine-tuning strengthens the model's adaptability to terminology standardization tasks through progressive training of single and multi-tasks; and finally, standardization is implemented through entity extraction and mapping.

[0028] It is easy to understand that this process can solve the problems of incomplete data coverage, poor model generalization, and lack of standardization basis in the standardization of traditional Chinese medicine terminology. It not only ensures the accuracy of terminology mapping, but also provides support for subsequent result traceability, thus meeting the rigorous requirements of the Chinese medicine field for data standardization.

[0029] Specifically, the professional text in the target domain mentioned in step 5 is TCM clinical text. When extracting entities to be standardized from TCM clinical text, an entity recognition model is used. The input of the entity recognition model is TCM clinical text, and the output is a text sequence of entities to be standardized in the TCM field, including diseases, prescriptions, drugs, syndromes, and symptoms. The mapping processing of the entities to be standardized includes three modes: standard mode, which directly maps entities to predefined TCM standard terms; synonym mode, which maps entities as synonyms to corresponding TCM standard terms based on a TCM thesaurus; and similarity_score mode, which maps entities to the closest TCM standard terms based on the similarity score calculated by the algorithm. When outputting standardized terms, the mapping basis is output simultaneously. The mapping basis includes the mapping mode and the source of the standard terms. If it is similarity_score mode, it also includes the corresponding similarity score.

[0030] Therefore, it is clear that the professional text is defined as TCM clinical text, such as electronic medical records and treatment records, which aligns with actual application scenarios. Entity extraction uses a specialized entity recognition model, rather than general text recognition tools, which can accurately locate five core entities: disease, prescription, drug, syndrome, and symptom. For example, if the input is "Female patient, 45 years old, with recurrent cough and phlegm for 2 months, accompanied by dry throat and sore throat, diagnosed as lung dryness and yin deficiency syndrome, treated with modified Sangxing Decoction," the model can automatically output the entity sequence "cough and phlegm - symptoms, lung dryness and yin deficiency syndrome - syndrome type, Sangxing Decoction - prescription." The mapping modes are divided into three categories: standard, synonym, and similarity_score, corresponding to three scenarios: direct matching, synonym association, and similarity calculation, respectively, covering different expressions of TCM terminology.

[0031] See Figure 2 It can be seen that the terminology association logic for various mapping patterns is as follows: Figure 2 Part (A) demonstrates the correspondence between standard terms and synonyms such as "Eight-Ingredient Kidney Qi Pill - Kidney Qi Pill" and "Awakeness - Poor Appetite," which can serve as examples to support the synonym pattern. Part (B) presents direct matching examples of standard terms, providing a reference for the standard pattern. It also requires outputting the mapping basis, such as "Mapping Pattern: Synonym Pattern; Standard Terminology Source: GB / T 16751.2-2021; Terminology Association: 'Lung Dryness and Yin Deficiency Syndrome' and 'Lung Yin Deficiency Syndrome' are synonyms," ensuring the traceability of each standardized operation step. The terminology association relationships in the basis can be referenced. Figure 2 Semantic association rules for positive and negative sample annotations.

[0032] At the same time, it is required to output the mapping basis, such as "mapping mode: synonym mode; standard terminology source: GB / T16751.2-2021; terminology association: 'lung dryness and yin deficiency syndrome' and 'lung yin deficiency syndrome' are synonymous terms", to ensure that each standardized operation is traceable.

[0033] It is easy to understand that this approach avoids the limitations of a single mapping model and improves the credibility of standardized results through a closed loop of entity extraction, precise mapping, and output. It is particularly suitable for the standardized processing of clinical diagnosis and treatment data, providing a unified terminology standard for subsequent medical record analysis and efficacy statistics.

[0034] Specifically, the terminology-related datasets in the target domain mentioned in step 4 include a dataset of synonyms, hyponyms, and questions and answers in the field of traditional Chinese medicine (TCM). During the single-task fine-tuning and multi-task instruction fine-tuning processes, the Siamese architecture of Sentence-BERT is adopted, encoding individual sentences independently into vectors, eliminating the need for sentence pairs as input. During the instruction fine-tuning process, the input data includes text input information and corresponding task instruction information. The task instruction information is used to instruct the model to complete specific tasks related to TCM terminology standardization, including terminology matching, terminology hierarchical association recognition, and semantic understanding of TCM knowledge questions and answers.

[0035] See Figure 3 and Figure 4It is evident that the model fine-tuning process emphasizes data relevance and training effectiveness: the terminology-related dataset includes three categories: synonym terms, hyponyms and hypernyms, and TCM question-and-answer databases, corresponding to the three core capabilities of terminology synonym recognition, terminology hierarchical relationship understanding, and contextual terminology matching. For example, the synonym dataset contains association pairs such as "stroke-apoplexy" and "common cold-cold," the hyponym dataset contains hierarchical pairs such as "common cold-wind-cold" and "prescription-exterior-releasing agent," and the question-and-answer database dataset contains question-and-answer pairs such as "commonly used drugs for treating wind-cold cold? Ephedra, Cinnamon Twig." These three types of data can improve the model's understanding of TCM terminology from different dimensions; the fine-tuning adopts the Siamese architecture of Sentence-BERT.

[0036] This architecture inputs Sentence A and Sentence B into two BERT encoders with identical structures and shared parameters. After pooling, vectors u and v are generated. The semantic association is quantified by calculating cosine similarity (cosine-sim(u,v)). This architecture supports independent encoding of individual sentences, eliminating the need for sentence pairs as input. Traditional architectures require simultaneous input of the term to be standardized and standard terms for matching, while this architecture encodes all standard terms into a vector library first. When the term to be standardized is input, similarity is calculated directly, significantly improving processing efficiency. Instruction fine-tuning involves adding task instructions such as "determine if 'lung dryness syndrome' and 'lung yin deficiency syndrome' are synonyms," clarifying the task objective for the model and avoiding mapping bias caused by task ambiguity. The combination of task instructions and text input is as follows: Figure 4 Data flow annotation for the “instruction fine-tuning unit” in the middle.

[0037] This shows that the model can specifically learn the core characteristics of TCM terminology during the fine-tuning stage, which not only improves training efficiency but also strengthens the model's understanding of the terminology standardization task, reduces irrelevant semantic interference, and makes the output results more consistent with the terminology association rules in the field of TCM.

[0038] Specifically, during the multi-task instruction fine-tuning process described in step 4, synonym term matching task data, hyponym term association task data, and TCM question-and-answer task data are simultaneously input, enabling the model to learn general TCM text semantic representations through collaborative training of multiple tasks. Model performance evaluation metrics include accuracy, F1 score / threshold, Pearson correlation coefficient based on cosine distance, Spearman correlation coefficient based on cosine distance, Pearson correlation coefficient based on Euclidean distance, Spearman correlation coefficient based on Euclidean distance, Pearson correlation coefficient based on Manhattan distance, and Spearman correlation coefficient based on Manhattan distance.

[0039] Therefore, multi-task fine-tuning is not simply data mixing, but rather improving the model's general semantic representation ability through collaborative training. Multi-task fine-tuning requires simultaneous input of three types of task data: synonym matching, hyponym / hypernym association, and question-and-answer from a question bank. The weights of these three types of data are equal to avoid biased learning caused by an excessively high proportion of data from one type. For example, if only synonym data is emphasized, the model may not be able to recognize the hyponym / hypernym relationship between a prescription and a diaphoretic agent. Collaborative training allows the model to simultaneously master the logical associations between multiple terms.

[0040] The process for inputting and collaborating multi-task data can be found by referring to... Figure 4 The annotation of the multi-task instruction fine-tuning unit; performance evaluation metrics include accuracy, F1 score / threshold, and Pearson and Spearman correlation coefficients based on three distances: cosine, Euclidean, and Manhattan, totaling eight metrics, rather than solely relying on accuracy: accuracy directly reflects the correct proportion of term mappings, F1 score / threshold balances precision and recall, and the three distance correlation coefficients quantify the semantic association between terms numerically: for example, a higher cosine correlation coefficient indicates stronger consistency between the model's judgment of term semantic similarity and human annotation. The calculation logic of each metric can be found in [reference needed]. Figure 4 The formulas and annotations in the "Evaluation" section on the right.

[0041] Specifically, the target language in step 2 is Chinese; the deep learning model adapted to the needs of Chinese text processing is the RoBERTa-wwm-ext-large model, which has 325M parameters and has been pre-trained on 5.4B tokens containing Wikipedia text. The specified length in step 3 is 512; the dynamic masking strategy involves randomly masking words in the input text during each training iteration, with different words masked in different iterations; the at least two domain-adaptive models are specifically a Pre-TCM model pre-trained on traditional Chinese medicine text and a Pre-MCM model pre-trained on modern Chinese medicine text. In step 4, the Pre-MCM model is evaluated using a validation set to determine its superior performance and is used as the base model.

[0042] Therefore, the initial model selection was not random, but based on a dual consideration of the characteristics of Chinese TCM terminology and model performance: the target language was clearly Chinese, as TCM terminology is expressed entirely in Chinese, and a Chinese model can avoid semantic loss caused by cross-language translation; the initial model selected was RoBERTa-wwm-ext-large, which has two core advantages: first, the number of parameters is 325M, which ensures both the depth of semantic understanding and avoids the problem of excessively high training costs caused by models with too many parameters; second, it has already completed large-scale general pre-training on 5.4B tokens containing Wikipedia text. Wikipedia text covers knowledge from multiple domains, allowing the model to have basic Chinese semantic understanding capabilities without having to train from scratch, significantly shortening the cycle of subsequent domain-adaptive pre-training.

[0043] Targeted optimizations were made in the domain-adaptive pre-training stage. The text length was set to 512, which not only meets the semantic integrity requirements of TCM texts but also adapts to the maximum input length limit of the RoBERTa model, avoiding semantic loss due to text truncation. The dynamic masking strategy is clearly defined as randomly masking words in each training iteration, and the masked words are different in different iterations: for example, the first iteration masks "liver" in "liver and kidney yin deficiency syndrome", and the second iteration masks "deficiency", which allows the model to learn the semantics of terms from different perspectives and avoids dependence on fixed words. The domain-adaptive models are specifically divided into Pre-TCM and Pre-MCM. The selection of the two models is based on the performance on the validation set: after testing, the accuracy of Pre-MCM on the validation set is higher than that of Pre-TCM. The reason is that modern TCM texts such as journal articles and clinical records are closer to the current application scenarios of terminology standardization, such as electronic medical records. The semantic rules learned by the model are more adapted to actual needs. Therefore, Pre-MCM was selected as the base model.

[0044] See Figure 5 The training accuracy curve of the Pre-MCM Fine-tuning model on the synonym dataset can be understood as follows: the accuracy of the Pre-MCM model gradually improves during the training process, eventually reaching a maximum value of 0.80, which confirms the performance advantage of the model.

[0045] Specifically, the multiple core terms mentioned in step 1 include disease, symptom, syndrome, prescription, and drug terms in the field of traditional Chinese medicine; among the positive and negative samples, the positive samples are manually labeled term association samples, specifically 3037 pairs of synonym term association samples determined based on authoritative standards and professional data, and the negative samples are non-associated term samples that are not related to the core terms and are automatically constructed by algorithms, and the number of negative samples is equal to the number of positive samples.

[0046] Therefore, it can be seen that the core terms are clearly categorized into five types: disease, symptom, syndrome, prescription, and drug. These five types are the core elements of TCM diagnosis and treatment and research. For example, the disease is hypertension, the symptom is dizziness, the syndrome is liver yang hyperactivity syndrome, the prescription is Tianma Gouteng Decoction, and the drug is Gastrodia elata, covering the main application scenarios of terminology standardization. The positive samples are manually labeled term association samples, numbering 3037 pairs. The labeling is based on authoritative standards in the field of TCM, such as the GB / T series standards and professional textbooks, such as "Traditional Chinese Medicine Diagnostics". They are jointly reviewed by three TCM experts with associate senior or higher professional titles to ensure the accuracy of the association. For example, "liver yang hyperactivity syndrome - liver yang disturbance syndrome" has been confirmed by experts as synonymous terms. The negative samples are automatically constructed by the algorithm, numbering the same as the positive samples, 3037 pairs. The generation logic is to randomly match from non-associated terms, such as "hypertension - Yinqiao Powder", to ensure the balance of positive and negative samples. If there are too many positive samples, the model may tend to judge all terms as associated, leading to an increased false positive rate.

[0047] This application also provides a terminology standardization system based on multi-task learning and instruction fine-tuning, including: The dataset construction module generates a corpus dataset of TCM texts containing both traditional and modern TCM texts, and divides it into a 7:2:1 ratio for training, validation, and test sets containing terms related to diseases and symptoms, along with an equal number of positive and negative samples of standard terminology association data. The initial model selection module calls the RoBERTa-wwm-ext-large Chinese model with 325M parameters, pre-trained on 5.4B token general text. The domain-adaptive pre-training module loads the TCM corpus and pre-trains the initial model to obtain Pre-TCM and Pre-MCM models, incorporating task removal, text length filtering, and dynamic masking units. The model selection and fine-tuning module evaluates and selects the Pre-MCM as the base model, fine-tuning it through single-task and multi-task instructions, using the Siamese architecture of Sentence-BERT. The entity extraction and standardization module includes entity recognition and standardization mapping units.

[0048] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A standardization algorithm for traditional Chinese medicine terminology based on multi-task learning and instruction fine-tuning, characterized in that, Includes the following steps: S1: Construct a training dataset, which includes a target domain corpus dataset and a standard terminology association corpus dataset. The target domain corpus dataset covers text resources from different periods in the domain. The standard terminology association corpus dataset is constructed based on the recognized authoritative standards and professional materials in the domain. It includes multiple core terms in the domain and positive and negative samples generated based on the core terms. The standard terminology association corpus dataset is divided into a training set, a validation set, and a test set according to a preset ratio. S2: Select an initial pre-trained language model, which is a deep learning model adapted to the needs of target language text processing and has been pre-trained using large-scale general text. S3: Implement domain-adaptive pre-training. Based on texts from different periods in the target domain corpus dataset, pre-train the initial pre-trained language model to obtain at least two domain-adaptive models. During the domain-adaptive pre-training process, remove the task used to predict the relationship between sentences, select continuous text of a specified length that does not cross documents as input, and use a dynamic masking strategy to process the words in the input text. S4: Model selection and multi-stage fine-tuning. The domain-adaptive model obtained in step 3 is evaluated using a validation set to determine the model with better performance as the base model. Based on the terminology-related dataset of the target domain, the base model is fine-tuned in sequence using single-task and multi-task instructions to obtain the terminology standardization model of the target domain. S5: Perform terminology standardization, extract entities to be standardized from professional texts in the target domain, use the terminology standardization model obtained in step 4 to map the entities to be standardized, and output the corresponding standard terms.

2. The algorithm for standardizing TCM terminology based on multi-task learning and instruction fine-tuning according to claim 1, characterized in that, In step 5, the target domain's professional text is traditional Chinese medicine (TCM) clinical text. When extracting entities to be standardized from TCM clinical text, an entity recognition model is used. The input of the entity recognition model is the TCM clinical text, and the output is a text sequence of entities to be standardized in the TCM field, including diseases, prescriptions, drugs, syndromes, and symptoms. The mapping processing of the entities to be standardized includes three modes: standard mode, which directly maps entities to predefined TCM standard terms; synonym mode, which maps entities as synonyms to corresponding TCM standard terms based on a TCM thesaurus; and similarity_score mode, which maps entities to the closest TCM standard terms based on the similarity score calculated by the algorithm. When outputting standardized terms, the mapping basis is output simultaneously. The mapping basis includes the mapping mode, the source of the standard term, and, in the similarity_score mode, the corresponding similarity score.

3. The algorithm for standardizing TCM terminology based on multi-task learning and instruction fine-tuning according to claim 1, characterized in that, The terminology-related datasets in step 4 include a thesaurus of synonyms, a hyponym / hypernym dataset, and a question-and-answer dataset for traditional Chinese medicine (TCM). During the single-task and multi-task instruction fine-tuning processes, a Siamese network architecture based on Sentence-BERT is used to encode individual sentences as vectors, eliminating the need for sentence pairs as model input. During instruction fine-tuning, the input data includes text input information and corresponding task instruction information. This task instruction information instructs the model to complete specific tasks related to TCM terminology standardization, including terminology matching, terminology hierarchical association recognition, and semantic understanding of TCM knowledge questions and answers.

4. The algorithm for standardizing TCM terminology based on multi-task learning and instruction fine-tuning according to claim 3, characterized in that, In the multi-task instruction fine-tuning process described in step 4, synonym term matching task data, hyponym term association task data, and TCM question bank question answering task data are input simultaneously, so that the model learns the general semantic representation of TCM text under the collaborative training of multiple tasks; the model performance evaluation indicators include accuracy, F1 score, and Pearson correlation coefficient and Spearman correlation coefficient based on cosine distance, Euclidean distance and Manhattan distance.

5. The algorithm for standardizing TCM terminology based on multi-task learning and instruction fine-tuning according to claim 4, characterized in that, In step 4, when evaluating model performance using the validation set, for the synonym terminology dataset, accuracy is used as the core evaluation metric. The accuracy is determined by calculating the proportion of correctly predicted positive examples (True Positives) to all actual positive examples (the sum of True Positives and False Negatives). For the hyponym terminology dataset and the traditional Chinese medicine question-and-answer dataset, the cosine similarity between sample instances is calculated first, and then the Pearson correlation coefficient between the calculated cosine similarity and the label is used as the core evaluation metric. The correlation coefficients based on Euclidean distance and Manhattan distance are both determined by calculating the Pearson correlation coefficient between the corresponding distance between sample instances and the label.

6. The algorithm for standardizing TCM terminology based on multi-task learning and instruction fine-tuning according to claim 1, characterized in that, The target language mentioned in step 2 is Chinese; the deep learning model adapted to the needs of Chinese text processing is the RoBERTa-wwm-ext-large model, and the RoBERTa-wwm-ext-large model has 325M parameters and has completed large-scale general text pre-training on 5.4B tokens containing Wikipedia text.

7. The algorithm for standardizing TCM terminology based on multi-task learning and instruction fine-tuning according to claim 1, characterized in that, The specified length in step 3 is 512; the dynamic masking strategy is to randomly mask words in the input text during each training iteration, and the masked words are different in different training iterations; the at least two domain adaptation models are specifically the Pre-TCM model based on traditional Chinese medicine text pre-training and the Pre-MCM model based on modern Chinese medicine text pre-training. In step 4, the Pre-MCM model is determined to be superior through validation set evaluation and is used as the base model.

8. The algorithm for standardizing TCM terminology based on multi-task learning and instruction fine-tuning according to claim 1, characterized in that, The multiple core terms mentioned in step 1 include disease, symptom, syndrome, prescription, and drug terms in the field of traditional Chinese medicine. Among the positive and negative samples, the positive samples are manually labeled term association samples, specifically 3037 pairs of synonym term association samples determined based on authoritative standards and professional data. The negative samples are non-associated term samples that are automatically constructed by algorithms and are not related to the core terms. The number of negative samples is equal to the number of positive samples.

9. The algorithm for standardizing traditional Chinese medicine terminology based on multi-task learning and instruction fine-tuning according to claim 1, characterized in that, The multiple core terms mentioned in step 1 include disease, symptom, syndrome, prescription, and drug terms in the field of traditional Chinese medicine. Among the positive and negative samples, the positive samples are manually labeled term association samples, specifically 3037 pairs of synonym term association samples determined based on authoritative standards and professional data. The negative samples are non-associated term samples that are automatically constructed by algorithms and are not related to the core terms. The number of negative samples is equal to the number of positive samples.

10. A system comprising the standardization algorithm for traditional Chinese medicine terminology based on multi-task learning and instruction fine-tuning as described in any one of claims 1-9, characterized in that, include: The dataset construction module is used to generate a corpus dataset in the field of traditional Chinese medicine containing traditional and modern TCM texts, and to divide it into a training set, a validation set, and a test set containing terms such as diseases and symptoms and an equal number of positive and negative samples of standard term association corpus in a 7:2:1 ratio; The initial model selection module is used to call the RoBERTa-wwm-ext-large Chinese model with 325M parameters and pre-trained with 5.4B token general text. The domain-adaptive pre-training module is used to load TCM corpus and pre-train the initial model to obtain Pre-TCM and Pre-MCM models respectively. It has built-in task removal, text length filtering and dynamic masking units. The model selection and fine-tuning module is used to evaluate and select the Pre-MCM as the base model. It is fine-tuned through single-task fine-tuning and multi-task instruction fine-tuning, and the fine-tuning adopts the Siamese architecture of Sentence-BERT. The entity extraction and standardization module includes an entity recognition unit and a standardization mapping unit.

Images