Small sample traditional chinese medicine ancient book labeling model adaptation method

Abstract

The application relates to the field of artificial intelligence, and discloses a small-sample traditional Chinese medical ancient book labeling model adaptation method, which comprises the following steps: obtaining an original traditional Chinese medical ancient book text, extracting structured domain knowledge from the text, and coding the structured domain knowledge into a knowledge vector; selecting a general pre-training language model, fine-tuning the general pre-training language model by using the original traditional Chinese medical ancient book text, and generating an ancient book basic language model; selecting a source domain model from a preset model library according to a target domain to be adapted; adding a knowledge transfer layer in the source domain model for embedding the knowledge vector into the source domain model, so as to construct a knowledge-fused to-be-trained model; and fine-tuning the knowledge-fused to-be-trained model on small-sample labeling data of the target domain by using a transfer learning loss function, so as to obtain a final labeling model. The application can quickly learn the characteristics of a new school under the condition that only a small amount of target domain labeling samples are available, and the labor and time cost for data labeling for the new school is reduced.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method for adapting a small sample ancient Chinese medical book annotation model. Background Art

[0002] Ancient Chinese medical texts are the core carriers of Chinese medical wisdom, containing a wealth of theoretical knowledge and clinical experience. To modernize the use of these valuable resources, the use of artificial intelligence technology to perform automated named entity recognition, relationship extraction, and other intelligent annotation tasks on ancient texts has become a hot topic in industry research and development.

[0003] Currently, mainstream technical solutions for achieving such tasks typically rely on deep learning models trained on large-scale annotated data. For example, a general pre-trained language model (such as BERT or RoBERTa) is fine-tuned on a target corpus (e.g., ancient TCM texts). To improve the model's performance in a specific domain, existing technologies may employ a secondary pre-training approach: first training on a general classical text corpus, then training on a specific TCM corpus, gradually adapting the model to the domain's characteristics.

[0004] However, existing technologies still face the following major challenges in practical applications:

[0005] 1. The conflict between data scarcity and domain diversity: Traditional Chinese Medicine (TCM) is a vast theoretical system encompassing numerous schools (e.g., the Bian Que School, the Zhongjing School, and the Four Great Masters of the Jin and Yuan Dynasties). These schools, formed during different historical periods, exhibit significant differences in theory, terminology, prescriptions, and presentation styles. Labeling large amounts of training data for each sub-school is impractical, extremely costly, and time-consuming. Consequently, when adapting from one school to another, model performance often deteriorates dramatically due to the scarcity of labeled samples in the target domain.

[0006] 2. Insufficient utilization of implicit knowledge: Traditional models primarily learn from surface statistical patterns in text. However, TCM contains a vast amount of structured, implicit, and non-textual expertise, such as the distribution of meridians, the inter-organ relationships, and the inherent logic of symptom systems. Existing models are unable to effectively utilize this structured domain knowledge, resulting in bottlenecks in their understanding of text and difficulty accurately identifying deeper semantic relationships.

[0007] 3. Inefficient transfer and adaptation: Existing model transfer methods mostly rely on simple fine-tuning. When there are significant differences between the source domain (e.g., a fully trained genre model) and the target domain (a new, small-sample genre), this simple fine-tuning approach has limited effectiveness and cannot achieve truly fast and efficient knowledge transfer.

[0008] Therefore, how to solve the problem of rapid and efficient adaptation of models between different schools of traditional Chinese medicine in small sample scenarios, and enable the model to deeply integrate structured domain knowledge during the learning process, is a technical challenge that needs to be solved urgently in this field. Summary of the Invention

[0009] In order to solve the problems existing in the above-mentioned prior art, the technical solutions provided by the present invention are as follows:

[0010] The adaptation method of the annotation model for small sample ancient Chinese medicine books includes the following steps:

[0011] Obtaining original ancient Chinese medical texts, extracting structured domain knowledge from the texts and encoding it into knowledge vectors, wherein the domain knowledge at least includes the meridian system and symptom system of Chinese medicine;

[0012] Selecting a general pre-trained language model, and fine-tuning the general pre-trained language model using the original ancient Chinese medical text to generate an ancient book basic language model;

[0013] According to the target domain to be adapted, a source domain model is selected from a preset model library, wherein the source domain model and the target domain belong to the same TCM theory system but have different branches and schools;

[0014] Adding a knowledge transfer layer in the source domain model to embed the knowledge vector into the source domain model to construct a model to be trained that integrates the knowledge;

[0015] By using a transfer learning loss function including source domain loss, target domain loss and domain knowledge constraint loss, the model to be trained of the fused knowledge is fine-tuned on a small sample labeled data of the target domain to obtain a final labeled model adapted to the target domain.

[0016] Preferably, the encoding method of the symptom system is one-hot encoding, wherein if a specific symptom appears in the text, the value of the corresponding position in the encoding vector is 1, otherwise it is 0.

[0017] Preferably, the encoding method of the meridian system is a binary code with a preset number of bits, wherein each bit of the binary code corresponds to a unique meridian.

[0018] Preferably, the domain knowledge also includes modern medical knowledge integrated by establishing a mapping relationship between ancient Chinese medical terms and modern medical terms.

[0019] Preferably, the step of fine-tuning the model to be trained that integrates the knowledge includes: freezing the shallow parameters of the model to be trained, and training its deep parameters and the knowledge transfer layer.

[0020] Preferably, before fine-tuning the model to be trained that integrates the knowledge, it also includes: using a third-party large language model to automatically pre-examine the small sample labeled data of the target domain, and combining it with domain experts to perform manual review to verify the consistency and accuracy of the small sample labeled data.

[0021] Preferably, the method for the knowledge transfer layer to embed the knowledge vector into the source domain model includes:

[0022] Mapping the knowledge vector to a preset dimension consistent with the word embedding vector of the source domain model using a fully connected layer to obtain a mapped knowledge vector;

[0023] Perform vector addition on the mapped knowledge vector and the word embedding vector to form a fused input vector.

[0024] Beneficial effects

[0025] 1. By explicitly injecting structured domain knowledge into the model, the present invention provides the model with powerful prior knowledge, enabling it to quickly learn the characteristics of new genres even with only a small number of labeled samples in the target domain, thereby greatly reducing the manpower and time costs of data labeling for new genres.

[0026] 2. The proposed transfer learning framework, which includes a source domain, a target domain, and knowledge constraints, is specifically designed to address the adaptation issues across different schools of Traditional Chinese Medicine. It efficiently transfers general TCM theoretical knowledge from the source domain model to the target domain, effectively overcoming the performance degradation caused by differences between schools and improving the model's generalization capabilities.

[0027] 3. The present invention directly integrates structured knowledge such as meridians and symptoms into the feature calculation of the model through the knowledge transfer layer. This enables the model to understand the theoretical logic of traditional Chinese medicine implicit in the text, so that when performing entity and relationship labeling tasks, it can make more accurate judgments that are more in line with medical theory, significantly improving the accuracy and professionalism of the labeling results.

[0028] 4. The present invention is highly flexible and scalable, and can easily and quickly adapt an existing expert model to multiple new sub-fields, providing an efficient solution for the intelligent information processing of ancient Chinese medical books and even other vertical fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 The figure is a flow chart of a method for adapting a small sample ancient Chinese medical book annotation model provided in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments herein are only used to explain the present invention, rather than to limit the present invention.

[0031] The method provided in an embodiment of the present invention can be run on a computing device that may include: at least one processor, a memory, and a communication bus connecting the processor and the memory. The memory stores computer program instructions. When the processor executes these instructions, it implements the method for adapting a small sample size ancient Chinese medical text annotation model described in any of the following embodiments. The computing device can be a server, a personal computer (PC), a workstation, or any other electronic device capable of performing computing tasks.

[0032] Example 1

[0033] This embodiment provides a method for adapting a small sample of ancient Chinese medical books annotation models. Figure 1 This method aims to address the challenges of difficult model adaptation and high annotation costs in small sample sizes of ancient Chinese medical texts, especially when applied across schools of thought. This example uses the example of adapting a model trained on text from the Bian Que School of Traditional Chinese Medicine to the annotation task of the Zhang Zhongjing School.

[0034] S1. Obtain original ancient Chinese medical texts, extract structured domain knowledge from the texts and encode them into knowledge vectors, wherein the domain knowledge at least includes the meridian system and symptom system of Chinese medicine.

[0035] Those skilled in the art should know that the obtained original ancient Chinese medical text, such as the original text of "Treatise on Febrile Diseases" in the target domain, needs to be preprocessed. The preprocessing process may include:

[0036] 1) Text Cleaning: Utilize pre-set rules (e.g., regular expressions) to remove non-essential content such as special symbols and modern annotations from the text. 2) Format Unification: Standardize the text encoding format, such as converting to UTF-8, to avoid garbled text. 3) Word Segmentation and Variant Character Conversion: Utilize a word segmentation method that combines a TCM vocabulary with a machine learning model to segment text in classical Chinese. Furthermore, utilize a pre-built variant-to-common character mapping table to convert variant characters in the text into modern, standard Chinese characters.

[0037] During or after text preprocessing, structured domain knowledge is extracted from the text and encoded into knowledge vectors. In a preferred embodiment, the domain knowledge includes at least the meridian system and symptom system of traditional Chinese medicine.

[0038] It should be understood that for symptom systems (such as the "five deficiencies and five excesses"), one-hot encoding can be used. Specifically, a list of N core symptoms is predefined, and the symptom system of each sample is encoded as an N-dimensional one-hot vector. If the i-th symptom appears in the text description, the i-th position of the vector is 1, otherwise it is 0. For example, a list containing symptoms such as "weak pulse", "lack of energy", "lack of appetite", "constipation", and "short and dark urine" is preset. A text describing "weak pulse" and "lack of energy" has the first two digits of its encoding vector as [1,1,0,0,0].

[0039] For meridian systems (such as the twelve main meridians and the eight extraordinary meridians), a binary encoding with a predetermined number of bits can be used. For example, a 46-bit binary encoding vector can be constructed, with each bit corresponding to a specific meridian. By matching keywords in the text, the meridians mentioned are identified and the corresponding bits in the encoding vector are set to 1, otherwise set to 0.

[0040] Optionally, to further enrich the model's knowledge context, the domain knowledge can also include modern medical knowledge integrated by establishing a mapping relationship between ancient TCM terminology and modern medical terminology. This step can be completed in or before S101. Specifically, the authoritative Traditional Chinese Medicine Clinical Terminology System (TCMTS) and modern medical databases (such as NCBI) can be used to extract semantic feature vectors of terms using language models pre-trained in the medical field, such as PubMedBERT. Then, through cluster analysis, cosine similarity calculation, and other methods, a mapping relationship between TCM terms (such as "xiaoke zheng") and modern medical terms (such as "diabetes") can be established. This associated information can serve as supplementary knowledge and be incorporated into the model in subsequent steps.

[0041] In some preferred embodiments, in order to achieve a deeper modeling of TCM theory, the "meridian system and symptom system" described above can be further concretized into structural elements covering the following five dimensions. These elements together constitute the domain knowledge framework that supports the model's in-depth understanding and logical reasoning:

[0042] (1) Entity: refers to the basic conceptual unit in TCM theory. For example, it can be a disease name (such as "sun disease"), a meridian (such as "bladder meridian"), a prescription (such as "Guizhi decoction"), a medicinal material, etc.

[0043] (2) Relationship: refers to the semantic and logical association between entities. For example, it can be a subordinate relationship between entities (such as "Tai Yang Bing - belongs to - Bladder Meridian"), or a functional relationship (such as "Gui Zhi Tang - Main Indications - Tai Yang Stroke").

[0044] (3) Structure: refers to the chapters, sections, and logical hierarchy that are unique to ancient Chinese medical texts. For example, it can be the "chapter-article-syndrome" hierarchical structure embodied in classics such as "Treatise on Febrile Diseases".

[0045] (4) Function: refers to the value and role of a specific entity in diagnosis and treatment. For example, it can be the diagnostic and therapeutic value of a prescription (such as "Guizhi Decoction - Main Indication - Sunstroke").

[0046] (5) Control mechanism: refers to the top-level logic and laws that guide the operation of the entire theoretical system. For example, it can be the core logic of a particular school of thought, such as the transmission and transformation laws in the "Six-Jing Differentiation" theory.

[0047] By uniformly vectorizing and embedding these multi-dimensional structured elements, this embodiment enables the model to go beyond simple text recognition and instead learn and understand the inherent theoretical framework, knowledge structure, and logical relationships of specific schools of Traditional Chinese Medicine, thereby enabling more accurate and domain-logical judgments to be made when adapting to small samples.

[0048] S2. Select a general pre-trained language model, and use the original ancient Chinese medical text to fine-tune the general pre-trained language model to generate an ancient book basic language model.

[0049] The general pre-trained language model is not limited to any specific model; any model that performs well in Chinese language processing can be used. For example, models such as the BERT series and the ERNIE series can be selected. Considering the classical Chinese characteristics of ancient Chinese medical texts and the requirements for Chinese language processing, a suitable general pre-trained language model is the RoBERTa-Chinese model, which uses full-word masking technology to better capture word-level semantics.

[0050] Next, we fine-tune the selected general pre-trained language model using a certain amount (e.g., 1GB) of text from various ancient Chinese medical texts. This step aims to adapt the model to the general language paradigm of ancient texts, initially adapting it to the linguistic style and semantic characteristics of ancient Chinese medical texts, thereby generating a basic language model for ancient texts.

[0051] In an optional embodiment, a more efficient training strategy can be adopted during the step of fine-tuning the model to be trained that integrates the knowledge, namely, freezing the shallow parameters of the model to be trained (e.g., the parameters of the first 12 layers of the RoBERTa model) and focusing on training its deep parameters (e.g., the last 6 layers) and the newly added knowledge transfer layer. This strategy aims to retain the common underlying features learned by the model from massive data and the source domain model, preventing them from being "catastrophically forgotten" on small sample data, while focusing computing resources on learning specific high-level semantic features of the target domain, thereby improving training efficiency and stability. It can retain the common features learned by the model from the source domain while focusing training resources on adapting to the specific features of the target domain, thereby improving training efficiency and reducing the risk of overfitting.

[0052] In an optional implementation, to ensure the quality of the small sample annotated data in the target domain, a data quality control step can be added before fine-tuning. This step can utilize a powerful third-party large language model (such as the DeepSeek model) to perform an automated pre-review of the small sample annotated data to check the semantic consistency of the annotations and identify obvious annotation errors. Subsequently, problematic data flagged by the model pre-review is submitted to domain experts for manual review and correction. This combined human-machine review method can effectively ensure the quality of the data used for fine-tuning, thereby improving the accuracy of the final model.

[0053] S3. Select a source domain model from a preset model library according to the target domain to be adapted. The source domain model and the target domain belong to the same TCM theory system but have different branches and schools.

[0054] In some preferred embodiments, the preset model library includes several source domain models from different schools of thought. The principle for selecting a source domain model based on the target domain includes: the source domain model and the target domain belong to the same broad TCM theoretical system, but differ in their specific branches and schools. For example, a model (M_source) trained on the annotated data of the "Nanjing" (Difficult Classic) from the "Bian Que School" can be selected as the source domain model.

[0055] S4. Adding a knowledge transfer layer to the source domain model to embed the knowledge vector into the source domain model, so as to construct a model to be trained that integrates the knowledge.

[0056] In an optional embodiment, the specific method of embedding the knowledge vector into the source domain model by the knowledge transfer layer may include:

[0057] First, vectors representing knowledge from different domains (such as one-hot symptom vectors and binary meridian vectors) are concatenated to form a combined knowledge vector.

[0058] A fully connected layer is then used to map the concatenated knowledge vector (whose dimensions may not match the internal vector dimensions of the model, for example, a 51-dimensional vector concatenated from a 5-dimensional one-hot symptom vector and a 46-dimensional meridian encoding vector) to a preset dimension. This preset dimension is exactly the same as the dimension of the word embedding vector of the source domain model (for example, 768 dimensions).

[0059] Finally, the dimensionally mapped knowledge vector is fused with the original word embedding vector. For example, a direct vector addition operation can be performed to form a final input vector that incorporates domain knowledge. This fused vector is then input into subsequent layers of the model (such as the Transformer layer) for processing. It should be understood that the vector fusion method here is not limited to vector addition; concatenation or other vector combination methods familiar to those skilled in the art can also be used.

[0060] S5. Using a transfer learning loss function including source domain loss, target domain loss, and domain knowledge constraint loss, fine-tune the model to be trained of the fused knowledge on the small sample labeled data of the target domain to obtain a final labeled model adapted to the target domain.

[0061] The transfer learning loss function can be designed as:

[0062] ;

[0063] Among them, D source and D target Datasets representing the source domain and target domain respectively; is the cross entropy loss, which is used to measure the difference between the model output and the true label; Input text for the model The predicted output of and are the true labels of the source domain and the target domain respectively; K is the input domain knowledge vector; Embedding vectors for domain knowledge output by the model; is the weight parameter to balance the source domain loss and the target domain loss and ; is the weight parameter of the domain knowledge constraint loss and .

[0064] The design concept of the transfer learning loss function is as follows:

[0065] It is the source domain loss item. Its function is to retain the general knowledge of TCM that has been learned by the source domain model (such as the basic terminology and diagnosis and treatment logic under the theoretical system) in the cross-school migration scenario of TCM, avoid "catastrophic forgetting", and reduce the cost of repeated learning.

[0066] The target domain loss term is used to guide the model to learn the school-specific characteristics of the target domain (such as the terminology system of the "Six-Jing Differentiation" of the Zhongjing School). Through fine-tuning with small sample data, the model can quickly adapt to new schools, solving the core problem of "data scarcity in small sample scenarios" in patents. The dynamic adjustment of can balance "knowledge retention" and "new feature learning".

[0067] This is a domain knowledge constraint loss term that enforces consistency between the model's output knowledge embedding and the input structured domain knowledge (meridians, symptom system), ensuring that the annotation results adhere to TCM theory and logic. This loss term uses TCM theory (such as 46-bit meridian encoding) as a hard constraint to prevent model outputs from violating medical common sense (e.g., incorrectly associating "liver depression and spleen deficiency" with a non-corresponding meridian).

[0068] This loss function comprehensively considers the model's performance in the source domain, the performance in the target domain, and the accurate expression of domain knowledge, thereby guiding the model to quickly adapt to the characteristics of the target domain while retaining the source domain knowledge.

[0069] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A small sample of ancient Chinese medical books annotation model adaptation method, characterized by: Including steps: Obtaining original ancient Chinese medical texts, extracting structured domain knowledge from the texts and encoding it into knowledge vectors, wherein the domain knowledge at least includes the meridian system and symptom system of Chinese medicine; Selecting a general pre-trained language model, and fine-tuning the general pre-trained language model using the original ancient Chinese medical text to generate an ancient book basic language model; According to the target domain to be adapted, a source domain model is selected from a preset model library, wherein the source domain model and the target domain belong to the same TCM theory system but have different branches and schools; Adding a knowledge transfer layer in the source domain model to embed the knowledge vector into the source domain model to construct a model to be trained that integrates the knowledge; Using a transfer learning loss function including a source domain loss, a target domain loss, and a domain knowledge constraint loss, the model to be trained of the fused knowledge is fine-tuned on a small sample of labeled data in the target domain to obtain a final labeled model adapted to the target domain; The domain knowledge constraint loss is used to force the knowledge embedding output by the trained model of the fusion knowledge to be consistent with the input domain knowledge, ensuring that the annotation results conform to the logic of traditional Chinese medicine theory.

2. The method for adapting a small sample size ancient Chinese medical book annotation model according to claim 1, characterized in that: The encoding method of the symptom system is one-hot encoding, wherein if a specific symptom appears in the text, the value of the corresponding position in the encoding vector is 1, otherwise it is 0.

3. The method for adapting a small sample size ancient Chinese medical book annotation model according to claim 1, characterized in that: The encoding method of the meridian system is a binary code with a preset number of bits, wherein each bit of the binary code corresponds to a unique meridian.

4. The method for adapting a small sample size ancient Chinese medical book annotation model according to claim 1, characterized in that: The domain knowledge also includes modern medical knowledge integrated by establishing a mapping relationship between ancient Chinese medical terms and modern medical terms.

5. The method for adapting a small sample ancient Chinese medical book annotation model according to claim 1, characterized in that: The step of fine-tuning the model to be trained that integrates the knowledge includes: freezing the shallow parameters of the model to be trained, and training its deep parameters and the knowledge transfer layer.

6. The method for adapting a small sample ancient Chinese medical book annotation model according to claim 1, characterized in that: Before fine-tuning the model to be trained that integrates the knowledge, it also includes: using a third-party large language model to automatically pre-examine the small sample labeled data of the target domain, and combining it with domain experts to perform manual review to verify the consistency and accuracy of the small sample labeled data.

7. The method for adapting a small sample ancient Chinese medical book annotation model according to claim 1, characterized in that: The method for the knowledge transfer layer to embed the knowledge vector into the source domain model includes: Mapping the knowledge vector to a preset dimension consistent with the word embedding vector of the source domain model using a fully connected layer to obtain a mapped knowledge vector; Perform vector addition on the mapped knowledge vector and the word embedding vector to form a fused input vector.

Images