Precise structured representation and semantic comparison method for traditional Chinese medicine situation information

Abstract

The invention discloses an accurate structured representation and semantic comparison method for traditional Chinese medicine situational information. The method comprises the following steps: constructing a situational dimension semantic rich network; key dimension information is extracted from a traditional Chinese medicine situation text through a thinking chain structuring module and is converted into a structured situation text, and then multi-level dimension features are extracted through a situation information coding module; then, in-layer context information and inter-layer multi-dimensional information are captured through an in-layer context information enriching module and an inter-layer information intersection module in sequence to obtain semantic-rich context semantic features; training the context dimension semantic rich network and carrying out context semantic comparison; according to the method, the traditional Chinese medicine semantic understanding ability of a general Chinese large language model is mined in combination with a thinking chain prompt tuning method, traditional Chinese medicine situation information is accurately extracted, meanwhile, multi-level traditional Chinese medicine situation dimension information is fused to obtain situation semantic features with rich semantics, the accuracy of traditional Chinese medicine situation semantic comparison is improved, and the traditional Chinese medicine situation semantic comparison efficiency is improved. And valuable reference situations are provided for traditional Chinese medicine clinicians.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing and artificial intelligence technology, and in particular relates to a method for accurate structured representation and semantic comparison of traditional Chinese medicine context information. Background Art

[0002] Traditional Chinese medicine has a long history and has accumulated a large amount of literature and materials, which record a lot of theoretical knowledge, clinical diagnosis methods, treatment experience and prescriptions. These materials are not only of historical value, but also of great reference significance for modern Chinese medicine clinics. In modern Chinese medicine clinics, doctors often need to find inspiration from similar medical records, symptoms and diagnoses in order to better guide diagnosis and treatment. It is of great significance to provide clinicians with similar situations for diagnostic reference through situational semantic comparison in clinics. This information comparison can improve the accuracy and efficiency of diagnosis, help to explore the potential value of Chinese medicine literature, and promote the application and development of Chinese medicine in the field of modern medicine. However, a large amount of Chinese medicine situational information, such as patient symptom descriptions, doctor's diagnosis records and treatment plans, usually exists in the form of natural language. These natural language expressions are not structured data, often lack strict formats and specifications, and there are huge challenges in processing. In the records of Chinese medicine, the patient's constitution, cause of disease, changes in condition, choice of treatment plan, etc. are all highly dependent on situational information. For example, a symptom may have different meanings in different situations such as time, climate, and patient constitution. Traditional information processing methods mainly rely on keyword matching and shallow semantic analysis. Since contextual information is unstructured, traditional information processing technologies are difficult to directly understand and process these complex semantics and associations, resulting in inaccurate matching results or weak retrieval relevance. Specifically, in the field of traditional Chinese medicine, terms are highly context-dependent, and the same term may represent different meanings in different contexts. For example, the cause and treatment of "fever" may be different in different patients' constitutions, courses of disease, or seasons. Traditional keyword matching methods find it difficult to identify these semantic differences, resulting in inaccurate matching results. And technologies that rely only on shallow semantic analysis are also difficult to capture multi-dimensional contextual information in traditional Chinese medicine diagnosis and treatment. Therefore, existing information processing methods have great limitations when facing a highly contextualized and semantically complex field such as traditional Chinese medicine. It is particularly important to accurately structure the representation and semantic matching technology of traditional Chinese medicine contextual texts for these documents and clinical records.

[0003] With the rapid development of natural language processing (NLP) technology, especially the emergence of large language models (such as GPT, BERT, etc.), semantic understanding and processing capabilities have been significantly improved. These models are pre-trained on massive data, can capture rich semantic relationships in language, and demonstrate powerful language processing capabilities. However, in the field of traditional Chinese medicine, the application of large language models is still in its infancy. The complexity and context dependence of traditional Chinese medicine terms increase the challenges of natural language processing in this field. At present, some studies have attempted to use large language models to fine-tune limited traditional Chinese medicine data, but due to the relatively limited amount of traditional Chinese medicine literature and clinical data, the effect of such fine-tuning is not ideal and it is difficult to achieve generalized application. On the other hand, large language models are trained on large-scale data, and there is a certain amount of traditional Chinese medicine corpus in these data, which means that large language models have a certain ability to understand traditional Chinese medicine semantics without fine-tuning. The key challenge is how to effectively use the capabilities of these models to accurately structure the context information in traditional Chinese medicine literature and clinical data, and then obtain accurate semantic representations for comparison and reference of diagnostic contexts. In the context of traditional Chinese medicine, intuitive symptoms and abstract syndromes are two key elements of semantic comparison. The pre-trained large language model has a multi-layer structure. The lower layer captures intuitive and shallow concepts, and the top layer captures deeper and abstract rules or information, which are strongly correlated with symptoms and syndromes, respectively. This suggests that we can combine the characteristics of traditional Chinese medicine contexts and the capabilities of large language models to develop new semantic structuring and comparison methods to solve specific problems in traditional Chinese medicine information processing. Summary of the invention

[0004] In view of this, it is necessary to address the above technical issues. The present invention provides a new method for accurately structuring and semantically comparing TCM context information, comprising the following steps:

[0005] Step 1: construct a context dimension semantic enrichment network that can extract structural information and context semantic features of TCM context text, including a thought chain structuring module, a context information encoding module, an intra-layer context information enrichment module, and an inter-layer information intersection module;

[0006] Step 2: Using TCM context text as network input, the thinking chain structured module extracts key dimensional information and converts it into structured context text, then extracts multi-level dimensional features through the context information encoding module, and then successively captures the intra-layer context information and inter-layer multi-dimensional information through the intra-layer context information enrichment module and the inter-layer information intersection module to obtain semantically rich context semantic features;

[0007] Step 3, using the situational semantic features to train the situational dimension semantic enrichment network;

[0008] Step 4: Use the trained context dimension semantic enrichment network to perform context semantic comparison.

[0009] Furthermore, the thought chain structuring module includes a thought stimulation mechanism and a structured extraction mechanism. The specific steps of extracting key dimension information and converting it into structured situational text are as follows:

[0010] Step 20101, combining the requirements of precise structured extraction format, inputting TCM context text into the large language model of general Chinese, and analyzing from multiple preset TCM context dimensions and providing analysis basis for each dimension, so as to obtain context element extraction ideas;

[0011] Step 20102, combining the precise structured extraction format requirements and the context element extraction ideas to construct context element extraction prompts, and obtaining the structured context text of the TCM context text through the general Chinese large language model.

[0012] Furthermore, the precise structured extraction format of TCM information includes multiple TCM key contextual dimensions formulated by TCM experts through grounded theory, including age, symptoms / signs, disease name and syndrome type, etiology and pathogenesis, treatment principles, etc. The specific format is:

[0013] {Dimension 1 :Dimension 1 Contextual information; Dimensions 2 :Dimension 2 context information; ...; dimension n :Dimension n contextual information};

[0014] n∈{1,2,...,27} represents 27 key contextual dimensions of TCM contexts.

[0015] Furthermore, the context information encoding module is composed of a multi-layer large language model encoder and a semantic enrichment layer selection mechanism. The specific steps of extracting multi-level dimensional features are as follows:

[0016] Step 2201: All structured context texts in the database are encoded by a multi-layer large language model encoder to obtain hidden layer features output by each layer of the encoder;

[0017] Step 2202, respectively calculating the recall rate of each hidden layer feature in all hidden layer features of the database in the “symptom” dimension retrieval and the “syndrome type” dimension retrieval;

[0018] Step 20203, select the hidden layer features with the highest recall rate in the "symptom" dimension as the shallow symptom hidden layer features, and select the hidden layer features with the highest recall rate in the "syndrome type" dimension as the deep syndrome type hidden layer features. The shallow symptom hidden layer features and the deep syndrome type hidden layer features constitute the multi-level dimensional features.

[0019] Furthermore, the intra-layer context information enrichment module includes a shallow semantic feedforward network, a deep semantic feedforward network and a self-attention mechanism network, and the specific steps of capturing the intra-layer context information are as follows:

[0020] Step 20301, first collect a number of TCM context text samples from the database, generate a batch of shallow symptom hidden layer features and a batch of deep syndrome type hidden layer features through the context information encoding module, and the batch of shallow symptom hidden layer features and the batch of deep syndrome type hidden layer features are respectively compressed by the shallow semantic feedforward network and the deep semantic feedforward network;

[0021] Step 20302, respectively adding sinusoidal position coding to the compressed batch of shallow symptom hidden layer features and batch of deep syndrome hidden layer features;

[0022] Step 20303, the encoded batch shallow symptom hidden layer features are passed through the self-attention mechanism network to capture context information, and then the batch full-text shallow symptom features are obtained through the Gelu nonlinear activation layer and the layer normalization layer;

[0023] Step 20304, the encoded batch of deep certificate type hidden layer features are passed through the self-attention mechanism network to capture context information to obtain batch full-text deep certificate type features.

[0024] The shallow symptom features of the batch full text and the deep syndrome type features of the batch full text constitute the context information within the layer.

[0025] Furthermore, the inter-layer information intersection module includes a cross-attention mechanism network and a feedforward network, and the steps of obtaining semantically rich contextual semantic features are specifically as follows:

[0026] Step 20401, the batch of shallow symptom features of the full text are used as Query, and the batch of deep syndrome features of the full text are used as Key and Value to aggregate the deep and shallow semantic information through a cross-attention mechanism network and layer normalization to obtain a batch of multi-level situational semantic representations, wherein Query, Key and Value are QKV weights in the cross-attention mechanism network;

[0027] In step 20402, the batch multi-level situational semantic representation is mapped into a two-dimensional vector in the situational semantic representation space through a linear layer in a feed-forward network, and then sequentially passes through a Gelu activation layer, a layer normalization layer, and a mean pooling layer of a text length dimension to obtain a one-dimensional situational semantic feature with rich semantics.

[0028] Further, in step 3, the context dimension semantic enrichment network is trained using the context semantic features, specifically using batch context semantic features to calculate model loss and optimize the intra-layer context information enrichment module and the inter-layer information intersection module, and the specific steps are as follows:

[0029] Step 301, construct a contrast loss function, the calculation formula is:

[0030]

[0031] Among them, L Contrastive is the comparison loss function value; γ is a flag variable, when the two context semantic features are of the same type, γ is 0, and when they are of different types, γ is 1; max(·) means taking the maximum value; m is the threshold; X 1 , X 2 are two situational semantic features; D W Represents the similarity measure between two situational semantic features;

[0032] D W When it is Euclidean distance:

[0033] D W (X 1 ,X 2 )=||X 1 -X 2 || 2 ;

[0034] D W When it is cosine similarity:

[0035]

[0036] Step 302, calculating the contrast loss function values ​​of all binary combinations in the batch context semantic features and dividing them by the number of combinations to obtain the total contrast loss of the batch;

[0037] Step 303, freeze all parameters of the large language model in the thought chain structuring module and the context information encoding module, and use the batch total contrast loss to optimize the intra-layer context information enrichment module and the inter-layer information intersection module.

[0038] Furthermore, the trained context dimension semantic enrichment network is used to perform context semantic comparison. The specific steps are as follows:

[0039] Step 401, all TCM context texts in the database are input into the context dimension semantic enrichment network to extract database context semantic features and save them;

[0040] Step 402, query the TCM context text and input it into the context dimension semantic enrichment network to obtain the query context semantic features;

[0041] Step 403 , calculating the cosine similarity between the query context semantic features and the database context semantic features, and selecting the top K results in descending order of similarity as output.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] The present invention combines the thinking chain tuning method to design a reasonable TCM context text extraction prompt to fully tap the TCM semantic understanding ability of the pre-trained large language model and improve the accuracy of structured extraction of TCM context information;

[0044] The present invention simultaneously integrates different levels of TCM context semantics to learn better context semantic features, designs a context information encoding module to extract multi-level dimensional features of the context from two dimensions of shallow symptom semantics and deep syndrome semantics; the proposed intra-layer context information enrichment module adopts multi-level dimensional features to capture the full-text information of the context; the proposed inter-layer information intersection module adopts the fusion of multi-level semantics to obtain semantically rich context semantic features, thereby improving the accuracy of TCM context semantic comparison. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A schematic flow chart of an implementation method of the present invention is shown;

[0046] Figure 2 A schematic diagram showing the structure of each module working in an embodiment of the present invention is shown; DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] For the purpose of reference and clarity, the technical terms, abbreviations or acronyms used below are summarized and explained as follows:

[0049] LLM: Large Language Model, large language model.

[0050] Prompt: Input the prompt word of the language model to guide the output of the language model.

[0051] GeLU: non-linear activation function.

[0052] Epoch: Using all the data in the training set to fully train the model is called "one generation of training".

[0053] self_Att.:self-Attention, self-attention mechanism network.

[0054] cross_Att.:cross-Attention, cross attention mechanism network.

[0055] Recall@K: Recall rate@K, the ratio of the number of samples correctly identified as positive by the model to the number of all positive samples in the database among the top K search results from high to low.

[0056] The present invention discloses a method for accurately structuring and comparing traditional Chinese medicine context information to solve many problems existing in the prior art.

[0057] Figure 1 A method for accurately structuring and semantically comparing TCM context information includes the following steps:

[0058] Step 1: construct a context dimension semantic enrichment network that can extract structural information and context semantic features of TCM context text, including a thought chain structuring module, a context information encoding module, an intra-layer context information enrichment module, and an inter-layer information intersection module;

[0059] Step 2: Using TCM context text as network input, the thinking chain structured module extracts key dimensional information and converts it into structured context text, then extracts multi-level dimensional features through the context information encoding module, and then successively captures the intra-layer context information and inter-layer multi-dimensional information through the intra-layer context information enrichment module and the inter-layer information intersection module to obtain semantically rich context semantic features;

[0060] Step 3, using the situational semantic features to train the situational dimension semantic enrichment network;

[0061] Step 4, using the trained context dimension semantic enrichment network to perform context semantic comparison;

[0062] Specifically, Figure 2 As shown, in step 2, firstly, all TCM context texts in the database are converted into structured context texts through the thought chain structuring module, and then the context information encoding module extracts and constructs batch multi-level dimensional features from the perspectives of shallow symptom semantics and deep syndrome semantics, and then the batch multi-level dimensional features are input into the intra-layer context information enrichment module and the inter-layer information intersection module to obtain batch semantically enriched context semantic features.

[0063] Furthermore, the specific operation steps of step 2 in this example are as follows:

[0064] Step 201, input each TCM situation text in the database into the thought chain structured module to extract key dimension information and convert it into a structured situation text; wherein the thought chain structured module includes a thought stimulation mechanism and a structured extraction mechanism, and the specific steps are as follows:

[0065] Step 20101, in combination with the precise structured extraction format requirements, the TCM context text is input into the general Chinese language model and analyzed from multiple preset TCM context dimensions and the basis for analysis of each dimension is given to obtain the context element extraction ideas. That is to say, in the thinking stimulation stage, a prompt is designed to enable the large language model to output the extraction ideas of context elements. The specific steps are as follows:

[0066] Step 2010101, build the LLM boot prompt, the specific content is as follows:

[0067] Guidance Prompt = "You are a TCM expert. Please extract the elements listed below based on the following TCM medical case. If some elements do not exist in the medical case, the corresponding content of the elements will be empty. The elements are as follows:"

[0068] Step 2010102, construct the LLM extraction format prompt, the specific content is as follows:

[0069]

[0070] Among them, D n It is the key contextual dimensions of TCM context evaluated by TCM experts, totaling 27, n∈{1,2,...,27} represents 27 key contextual dimensions of TCM context, specifically gender, age, occupation, marriage and childbearing, birth time, population characteristics, menstrual history, symptoms / signs, time and space, luck factors, space, climate and phenology, group knowledge base, doctor's subjective conjecture, disease name and syndrome type, etiology and pathogenesis, auxiliary diagnostic tools, requirements for doctors, treatment principles, oral medication, external treatment, misdiagnosis, conditioning methods, compliance, symptoms, etiology and pathogenesis, tests and examinations, and other comments (notes). In other words, the precise structured extraction format of TCM context includes multiple key contextual dimensions of TCM formulated by TCM experts through grounded theory, including age, symptoms / signs, disease name and syndrome type, etiology and pathogenesis, treatment principles, etc. The specific format is:

[0071] {Dimension 1 :Dimension 1 Contextual information; Dimensions 2 :Dimension 2 context information; ...; dimension n :Dimension n context information},

[0072] That is {D 1 :D 1 Situational information; D 2 :D 2 Situational information; ...; D n :D n contextual information};

[0073] Step 2010103, build the example prompt of LLM, the specific content is as follows:

[0074] Example Prompt = "The following is an example, original text: " + example original text + example extraction result;

[0075] Example original text = "Acute attack of chronic bronchitis, pathogenic heat accumulated in the lungs, causing phlegm-heat syndrome, treatment is to clear away heat and promote lung function, resolve phlegm and relieve asthma. Name: XXX, male, 53 years old. Medical record number: XXXX. First visit: February 23, 2010. Cough and asthma have recurred for 5 years, worsening with fever for 1 week. In the past 5 years, the patient has often had symptoms such as coughing, sputum, and asthma, especially in winter. He has sought treatment from many places, but the effect is not obvious. He was diagnosed with chronic bronchitis. One week ago, due to the sudden drop in temperature, he failed to add clothes in time and caught a cold, which caused a relapse of the disease. At first, he had aversion to cold, fever, headache, cough, sputum, and sore throat. Self-medication with cold medicine and antibiotics had no effect, and the condition gradually worsened. The aversion to cold disappeared and the fever gradually became stronger. At present: body heat, body Temperature 38.8℃, no aversion to cold, coughing, choking cough, sputum, yellow and thick sputum, wheezing, especially at night, unable to lie flat in severe cases, chest tightness, sweating, thirst, dry and hard stool, red tongue, yellow fur, slippery and rapid pulse. Chest X-ray showed: thickened and disordered texture of both lungs. Chinese medicine diagnosis: wind-heat; the syndrome belongs to pathogenic heat in the lungs, which produces phlegm and heat, and the lungs fail to descend. Western medicine diagnosis: acute exacerbation of chronic bronchitis. Treatment is to clear heat and promote lung function, resolve phlegm and relieve wheezing, and the prescription is Ma Xing Shi Gan Tang plus or minus. Prescription: roasted ephedra 6g, almond 10g, gypsum 30g, raw licorice 6g, houttuynia cordata 20g, golden buckwheat 20g, polygonum cuspidatum 20g, trichosanthes seed 12g, light scutellaria 12g, mulberry bark 15g, loquat leaf 12g, plantain 10g. 3 doses. ”

[0076] Example extraction result = "

[0077] 1. Personal Information

[0078] 1. Gender: Male

[0079] 2. Age: 53

[0080] 3. Occupation:

[0081] 4. Marriage and childbearing:

[0082] 5. Date of birth:

[0083] 6. Characteristics of the population: In the past five years, the patient has often had symptoms such as coughing, expectoration, and wheezing, especially in winter. He has sought treatment from many sources but to no avail. He was diagnosed with chronic bronchitis. One week ago, the temperature dropped suddenly and he did not add clothes in time, so he caught a cold and his condition relapsed. He initially had aversion to cold, fever, headache, cough, expectoration, and sore throat. Self-medication with cold medicine and antibiotics had no effect. Instead, his condition gradually worsened, his aversion to cold disappeared, and his fever gradually increased.

[0084] 2. Symptoms / Signs

[0085] 1. History of menstruation:

[0086] 2. Symptoms / signs: Cough and asthma recurred for 5 years, aggravated with fever for 1 week. Fever, body temperature 38.8℃, no aversion to cold, coughing, choking cough, sputum, yellow and thick sputum, asthma, especially at night, unable to lie flat in severe cases, chest tightness, sweating, thirst, dry and hard stool, red tongue, yellow fur, slippery and rapid pulse.

[0087] 3. Space and time

[0088] 1. Date: 2010 / 2 / 23

[0089] 2. Luck Factor:

[0090] 3. Space:

[0091] 4. Climate and phenology:

[0092] 4. Decision Basis

[0093] 1. Group knowledge base:

[0094] 2. The doctor's guess:

[0095] 5. Dialectical Process

[0096] 1. Disease name and syndrome: acute exacerbation of chronic bronchitis, pathogenic heat accumulating in the lungs, causing phlegm-heat syndrome

[0097] 2. Etiology and pathogenesis:

[0098] 3. Auxiliary diagnostic tools: Chest X-ray shows: thickened and disordered texture of both lungs. TCM diagnosis: Wind-heat; the syndrome is pathogenic heat accumulating in the lungs, causing phlegm heat, and the lungs failing to descend.

[0099] 6. Treatment

[0100] 1. Requirements for doctors:

[0101] 2. Treatment principles: clearing away heat, promoting lung function, resolving phlegm and relieving asthma.

[0102] 3. Oral medication: Select Ma Xing Shi Gan Tang with modifications. Prescription: roasted ephedra 6g, almond 10g, gypsum 30g, raw licorice 6g, houttuynia 20g, golden buckwheat 20g, polygonum cuspidatum 20g, trichosanthes seed 12g, light scutellaria 12g, mulberry bark 15g, loquat leaf 12g, plantain 10g. 3 doses.

[0103] 4. External treatment:

[0104] 5. Bootstrapping:

[0105] 6. Mistakes and misdiagnosis:

[0106] 7. Maintenance method:

[0107] 8. Compliance:

[0108] 7. Efficacy evaluation

[0109] 1. Symptoms: After taking the medicine, the fever gradually subsided, the body temperature returned to normal (36.8℃), the headache and sore throat disappeared. The cough was still obvious, with a lot of sputum, which turned white but sticky and difficult to cough up, shortness of breath, especially at night, chest tightness, dry mouth, red tongue, yellow and greasy tongue coating, and slippery and rapid pulse.

[0110] 2. Etiology and pathogenesis:

[0111] 3. Examination and examination: Lung heat has been relieved, phlegm heat is still severe, and the lungs have failed to descend.

[0112] 4. His comments (notes):

[0113] ”

[0114] Step 2010104, construct the semantic understanding ability mining prompt of LLM, the specific contents are as follows:

[0115] Semantic understanding ability mining prompt = "Please give your extraction ideas";

[0116] Step 2010105, the guidance prompt, the extraction format prompt, the example prompt and the semantic understanding ability mining prompt are sequentially spliced ​​to obtain the LLM thinking chain prompt.

[0117] Thinking chain prompt = guidance prompt + format extraction prompt + example prompt + semantic understanding ability mining prompt;

[0118] Step 2010106, construct the LLM input text in the format [{"role":"system","content":thinking chain prompt},{"role":"user","content":situation text to be structured}] and input it into LLM to obtain the idea of ​​extracting situational elements.

[0119] Context element extraction idea = LLM ({"role":"system","content":Thinking chain prompt},{"role":"user","content":Context text to be structured}]);

[0120] Step 20102, combining the precise structured extraction format requirements and the context element extraction ideas to construct a context element extraction prompt, and obtain the structured context text of the TCM context text through the general Chinese large language model, that is, construct the context element extraction prompt input LLM to obtain the TCM context structured text, the specific steps are as follows:

[0121] Step 2010201, the guide prompt, extraction format prompt, example prompt, "The following is the extraction idea", situational element extraction idea, and "Please give the extraction result:" are sequentially spliced ​​to obtain the situational element extraction prompt of LLM,

[0122] Situational element extraction prompt = guidance prompt + extraction format prompt + example prompt

[0123] + "The following are the extraction ideas" + Situational element extraction ideas + "Please give the extraction results:"

[0124] Step 2010202: construct the LLM input text in the format of [{"role":"system","content":Context element extraction prompt},{"role":"user","content":Context text to be structured}] and input it into LLM to obtain the 27-dimensional TCM context structured text.

[0125] Traditional Chinese Medicine Contextual Structured Text = LLM(

[0126] [{"role":"system","content":Context element extraction prompt},

[0127] {"role":"user","content":context to be structured}]);

[0128] Step 202, constructing a context information encoding module, including a multi-layer large language model encoder and a semantic enrichment layer selection mechanism, wherein the step of extracting multi-layer dimensional features is specifically as follows:

[0129] Step 2201: All structured context texts in the database are encoded by a multi-layer large language model encoder to obtain the hidden layer features output by each layer of the encoder; specifically:

[0130] All structured contextual texts in the database are constructed as input in the format of [{"role":"user","content":TCM contextual structured text}] and encoded by the LLM encoder to obtain the hidden layer features H of each layer output of the LLM encoder. i ,i∈{0,1,..,h}, where h is the number of LLM encoder layers, H i The dimension is l×d, where l is the text length and d is the LLM hidden layer output dimension;

[0131] In this example, we limit the maximum text length to 2048. We remove samples with text length l>2048. For samples with text length l≤2048, we add a new feature in each hidden layer H i Pad the first dimension of the text by adding a 0 tensor of size (2048-l)×d to the text length, changing its dimension from l×d to 2048×d. At the same time, generate a mask of corresponding dimension 2048×1. The first l elements of the mask have a value of 1, and the last 2048-l elements have a value of 0, so that the attention layer in the subsequent steps only considers the first l valid values.

[0132] Step 2202, among all the hidden layer features in the database, calculate the Recall@1 of each hidden layer feature in the “symptom” dimension retrieval and the “syndrome type” dimension retrieval,

[0133] Specifically, all TCM context texts in the database are classified into "symptoms" and "syndrome types", and the category labels of "symptoms" and "syndrome types" of each TCM context text are obtained. For a TCM context text, the cosine similarity between its corresponding hidden layer features and all hidden layer features in the database is calculated, and the one with the highest similarity is selected as the retrieval result. If the category labels of the retrieval results are the same, the retrieval is correct.

[0134] For each hidden layer, calculate the average retrieval recall rate Recall@1 of the two category labels of "symptoms" and "syndrome type" of the hidden layer features of the database, which are used as the "symptom" dimension Recall@1 and the "syndrome type" dimension Recall@1 respectively;

[0135] Step 20203: For each TCM context text, the hidden layer feature with the highest Recall@1 in the “symptom” dimension is selected as the shallow symptom hidden layer feature, denoted as H 症状 ; Select the hidden layer feature with the highest Recall@1 in the “Syndrome Type” dimension as the deep syndrome type hidden layer feature, denoted as H 证型 , the two together constitute the multi-level dimensional feature H 症状 , H 证型 .

[0136] Step 203, constructing an intra-layer context information enrichment module, including a shallow semantic feedforward network, a deep semantic feedforward network and a self-attention mechanism network; the batch multi-level dimensional features are first passed through the intra-layer context information enrichment module to capture the intra-layer context information, and the intra-layer context information includes a batch of full-text shallow symptom features and a batch of full-text deep syndrome features. The specific steps are as follows:

[0137] Step 20301, in this embodiment, first collect B TCM context text samples from the entire database each time, obtain the corresponding shallow symptom hidden layer features and deep syndrome hidden layer features to form batch multi-level dimensional features, denoted as H 症状 , H 症型 , whose dimensions are all B×2048×d. In this embodiment, the number of TCM context text sample collections B is 8, that is, the dimension is 8×2048×d;

[0138] Then, the batch of shallow symptom hidden layer features and the batch of deep syndrome hidden layer features are respectively passed through the shallow semantic feedforward network and the deep semantic feedforward network to expand the last dimension from d to 2d and then compress it to Wherein, the shallow semantic feedforward network and the deep semantic feedforward network are both composed of two fully connected layers;

[0139] Step 20302, respectively adding sinusoidal position coding to the compressed batch of shallow symptom hidden layer features and batch of deep syndrome hidden layer features;

[0140]

[0141] Among them, FFN 浅层 It is a shallow semantic feedforward network, FFN 深层 is a deep semantic feedforward network, p is a sinusoidal positional encoding, H 症状 , H 症型 is the batch multi-level dimensional feature before compression, is a batch of multi-level dimensional features after compression and encoding, with dimensions of Batch multi-level dimensional features include batch shallow symptom hidden layer features and batch deep evidence type hidden layer features

[0142] Step 20303, the encoded batch shallow symptom hidden layer features are passed through the self-attention mechanism network to capture context information, and then the batch full-text shallow symptom features are obtained through the Gelu nonlinear activation layer and the layer normalization layer; wherein the self-attention mechanism network is a bidirectional self-attention layer, and the specific mathematical expression is as follows:

[0143]

[0144] Among them, LN represents the layer normalization layer, Gelu represents the nonlinear activation layer, self_Att. represents the self-attention mechanism network, Query, Key, and Value represent the QKV weights in the self-attention mechanism network, is the batch shallow symptom hidden layer feature, is the shallow symptom feature of the batch full text, and its dimensions are mask represents batch mask;

[0145] Step 20304, the encoded batch of deep certificate type hidden layer features are passed through the self-attention mechanism network to capture context information to obtain batch full-text deep certificate type features; wherein the self-attention mechanism network is a bidirectional self-attention layer, and the specific mathematical expression is as follows:

[0146]

[0147] Among them, self_Att. is the self-attention mechanism network, Query, Key, and Value represent the QKV weights in the self-attention mechanism network, is the batch full-text deep evidence type feature, and its dimensions are is a batch of deep-layer evidence-based hidden layer features, and mask represents a batch mask;

[0148] Step 204, constructing the inter-layer information intersection module, including a cross attention mechanism network and a feedforward network; the batch multi-level dimensional features are then passed through the inter-layer information intersection module to capture the inter-layer multi-dimensional information and combine the intra-layer context information to obtain semantically rich situational semantic features. The specific steps are as follows:

[0149] Step 20401, the batch of shallow symptom features of the full text are used as Query, and the batch of deep syndrome features of the full text are used as Key and Value. The deep and shallow semantic information is aggregated through a cross-attention mechanism network and layer normalization to obtain a batch of multi-level situational semantic representations. The cross-attention mechanism network is a bidirectional cross-attention layer. The specific mathematical expression is as follows;

[0150]

[0151] Among them, LN represents layer normalization, cross_Att. is the cross attention mechanism network, Query, Key, Value represent the QKV weights in the cross attention mechanism network, and Z is a batch multi-level context semantic representation with a dimension of

[0152] Step 20402, the batch of multi-level context semantic representations Z are mapped to two-dimensional vectors in the context semantic representation space through the linear layer in the feedforward network, so that each context semantic representation becomes 2048×1024 dimensions, and then sequentially through the Gelu activation layer, the layer normalization layer LN and the mean pooling of the text length dimension to obtain the batch of one-dimensional semantically rich context semantic features X. The specific mathematical expression is as follows:

[0153] X=avepool(Gelu(LN(FFN(Z)+Z)))

[0154] Among them, FFN is the feedforward network, which consists of two fully connected layers, avepool is the mean pooling layer, Gelu is the activation layer, LN is the layer normalization layer, X is the semantically rich contextual semantic features of the batch, and the dimension is B×1024, that is, B 1024-dimensional contextual semantic feature tensors.

[0155] Specifically, in step 3, the situational semantic features are used to train the situational dimension semantic enrichment network, specifically, batch situational semantic features are used to calculate the model loss and optimize the intra-layer context information enrichment module and the inter-layer information intersection module, and the specific steps are as follows:

[0156] Step 301, construct a contrast loss function, the calculation formula is:

[0157]

[0158] Among them, L Contrastive is the comparison loss function value; γ is a flag variable, when the two contextual semantic features are of the same type, γ is 0, and when they are of different types, γ is 1; max(·) means taking the maximum value; m is the threshold, which is set to 0.5 in this example; X 1 , X 2 are two situational semantic features; D W Represents the similarity between two contextual semantic features,

[0159] In this example, Euclidean distance can be used to measure:

[0160] D W (X 1 ,X 2 )=||X 1 -X 2 || 2 ;

[0161] In this example, cosine similarity can also be used for measurement, specifically:

[0162]

[0163] Step 302, calculate the contrast loss function value of all binary combinations in B contextual semantic features and divide it by the number of combinations to obtain the total contrast loss of the batch, that is, traverse all binary combinations in the batch to calculate the contrast loss. The final mathematical expression of the total contrast loss of the batch is as follows:

[0164]

[0165] Where L is the total contrast loss of the batch, X is the batch context semantic feature, B is the number of batch samples, in this example B = 8; X i ,X j are two situational semantic features, L Contrastive is the contrast loss function value;

[0166] Step 303, freeze all parameters of the large language model in the thought chain structuring module and the context information encoding module, and use the batch total contrast loss to optimize the intra-layer context information enrichment module and the inter-layer information intersection module.

[0167] Step 4: Use the trained context dimension semantic enrichment network to perform context semantic comparison. The specific steps are as follows:

[0168] Step 401, all TCM situation texts in the database are converted into structured situation texts through the thought chain structured module, and input into the situation dimension semantic enrichment network to extract the database situation semantic features and save them;

[0169] Step 402, the query TCM context text is structured through the thought chain structuring module and input into the context dimension semantic enrichment network to obtain the query context semantic features;

[0170] Step 403, calculating the cosine similarity between the query context semantic features and the database context semantic features, and selecting the first K results in descending order of similarity as output to obtain a comparison result.

[0171] In the present invention, the maximum length of the text is 2048, the number of collected TCM context texts B, that is, the batch size is 8, LLM uses GLM4 and freezes all its parameters, the initial learning rate is 0.00005, 20 Epochs are trained on 1 RTX4090 GPU, and the network is optimized using the adam optimizer.

[0172] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0173] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0174] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0175] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for accurate structured representation and semantic comparison of TCM context information, characterized in that: The following steps are involved: Step 1: construct a context dimension semantic enrichment network that can extract structural information and context semantic features of TCM context text, including a thought chain structuring module, a context information encoding module, an intra-layer context information enrichment module, and an inter-layer information intersection module; Step 2: Using TCM context text as network input, the thinking chain structured module extracts key dimensional information and converts it into structured context text, then extracts multi-level dimensional features through the context information encoding module, and then successively captures the intra-layer context information and inter-layer multi-dimensional information through the intra-layer context information enrichment module and the inter-layer information intersection module to obtain semantically rich context semantic features; Step 3, using the situational semantic features to train the situational dimension semantic enrichment network; Step 4: Use the trained context dimension semantic enrichment network to perform context semantic comparison.

2. According to claim 1, a method for accurate structured representation and semantic comparison of TCM context information is characterized in that: The thought chain structured module includes a thought stimulation mechanism and a structured extraction mechanism. The specific steps of extracting key dimension information and converting it into structured situational text are as follows: Step 20101, combining the requirements of precise structured extraction format, inputting TCM context text into the large language model of general Chinese, and analyzing from multiple preset TCM context dimensions and providing analysis basis for each dimension, so as to obtain context element extraction ideas; Step 20102, combining the precise structured extraction format requirements and the context element extraction ideas to construct context element extraction prompts, and obtaining the structured context text of the TCM context text through the general Chinese large language model.

3. According to claim 2, a method for accurate structured representation and semantic comparison of TCM context information is characterized in that: The precise structured extraction format of TCM information includes multiple TCM key contextual dimensions formulated by TCM experts, including age, symptoms / signs, disease name and syndrome type, etiology and pathogenesis, treatment principles, etc. The specific format is: {dimension 1: dimension 1 context information; dimension 2: dimension 2 context information; ...; dimension n :Dimension n contextual information}; n∈{1,2,...,27} represents 27 key contextual dimensions of TCM contexts.

4. According to claim 1, a method for accurate structured representation and semantic comparison of TCM context information is characterized in that: The context information encoding module is composed of a multi-layer large language model encoder and a semantic enrichment layer selection mechanism. The specific steps of extracting multi-level dimensional features are as follows: Step 2201: All structured context texts in the database are encoded by a multi-layer large language model encoder to obtain hidden layer features output by each layer of the encoder; Step 2202, among all hidden layer features in the database, respectively calculate the recall rate of each hidden layer feature in the "symptom" dimension retrieval and the "syndrome type" dimension retrieval; Step 2203, select the hidden layer features with the highest recall rate in the "symptom" dimension as the shallow symptom hidden layer features, and select the hidden layer features with the highest recall rate in the "syndrome type" dimension as the deep syndrome type hidden layer features. The shallow symptom hidden layer features and the deep syndrome type hidden layer features constitute the multi-level dimensional features.

5. According to claim 4, a method for accurate structured representation and semantic comparison of TCM context information is characterized in that: The intra-layer context information enrichment module includes a shallow semantic feedforward network, a deep semantic feedforward network and a self-attention mechanism network. The specific steps of capturing the intra-layer context information are as follows: Step 20301, first collect a number of TCM context text samples from the database, generate a batch of shallow symptom hidden layer features and a batch of deep syndrome type hidden layer features through the context information encoding module, and the batch of shallow symptom hidden layer features and the batch of deep syndrome type hidden layer features are respectively compressed by the shallow semantic feedforward network and the deep semantic feedforward network; Step 20302, respectively adding sinusoidal position coding to the compressed batch of shallow symptom hidden layer features and batch of deep syndrome hidden layer features; Step 20303, the encoded batch shallow symptom hidden layer features are passed through the self-attention mechanism network to capture context information, and then the batch full-text shallow symptom features are obtained through the Gelu nonlinear activation layer and the layer normalization layer; Step 20304, the encoded batch of deep certificate type hidden layer features are passed through the self-attention mechanism network to capture context information to obtain batch full-text deep certificate type features; The shallow symptom features of the batch full text and the deep syndrome type features of the batch full text constitute the context information within the layer.

6. A method for accurate structured representation and semantic comparison of TCM context information according to claim 5, characterized in that: The inter-layer information intersection module includes a cross-attention mechanism network and a feedforward network. The steps of obtaining semantically rich contextual semantic features are as follows: Step 20401, the batch of shallow symptom features of the full text are used as Query, and the batch of deep syndrome features of the full text are used as Key and Value to aggregate the deep and shallow semantic information through a cross-attention mechanism network and layer normalization to obtain a batch of multi-level situational semantic representations, wherein Query, Key and Value are QKV weights in the cross-attention mechanism network; In step 20402, the batch multi-level situational semantic representation is mapped into a two-dimensional vector in the situational semantic representation space through a linear layer in a feed-forward network, and then sequentially passes through a Gelu activation layer, a layer normalization layer, and a mean pooling layer of a text length dimension to obtain a one-dimensional situational semantic feature with rich semantics.

7. A method for accurate structured representation and semantic comparison of TCM context information according to claim 6, characterized in that: In step 3, the context dimension semantic enrichment network is trained using the context semantic features, specifically using batch context semantic features to calculate model loss and optimize the intra-layer context information enrichment module and the inter-layer information intersection module, and the specific steps are as follows: Step 301, construct a contrast loss function, the calculation formula is: Among them, L Contrastive is the comparison loss function value; γ is the flag variable, when the two contextual semantic features are of the same type, γ is 0, and when they are of different types, γ is 1; max(·) means taking the maximum value; m is the threshold; X1, X2 are two contextual semantic features; D W Represents the similarity measure between two situational semantic features; D W When it is Euclidean distance: D W (X1,X2)=||X1-X2||2; D W When it is cosine similarity: Step 302, calculating the contrast loss function values ​​of all binary combinations in the batch context semantic features and dividing them by the number of combinations to obtain the total contrast loss of the batch; Step 303, freeze all parameters of the large language model in the thought chain structuring module and the context information encoding module, and use the batch total contrast loss to optimize the intra-layer context information enrichment module and the inter-layer information intersection module.

8. According to claim 1, a method for accurate structured representation and semantic comparison of TCM context information, characterized in that: Use the trained context dimension semantic enrichment network to perform context semantic comparison. The specific steps are as follows: Step 401, all TCM context texts in the database are input into the context dimension semantic enrichment network to extract database context semantic features and save them; Step 402, query the TCM context text and input it into the context dimension semantic enrichment network to obtain the query context semantic features; Step 403 , calculating the cosine similarity between the query context semantic features and the database context semantic features, and selecting the top K results in descending order of similarity as output.

Images