Multi-modal traditional Chinese medicine data structured processing method and related device

Abstract

The invention discloses a multi-mode traditional Chinese medicine data structured processing method and a related device, and the method comprises the steps: obtaining traditional Chinese medicine diagnosis and treatment data which comprise text medical case data, tongue image data and pulse diagnosis signal data; according to the data activity labels, dividing the traditional Chinese medicine diagnosis and treatment data into high, medium and low activity data levels; generating a multi-modal feature set containing text symptom entity features, tongue condition description coding parameters and pulse condition description coding parameters based on the traditional Chinese medicine diagnosis and treatment data of the high-activity data hierarchy; key features of the traditional Chinese medicine diagnosis and treatment data of the middle activity data level are extracted, and a symptom-prescription medicine associated feature set is output; performing syndrome type feature integration on the low-activity data level traditional Chinese medicine diagnosis and treatment data; and based on a preset dynamic rule mapping library, carrying out feature mapping on a result of integration of the multi-modal feature set, the symptom-prescription drug associated feature set and the symptom type feature, and realizing structured processing on the multi-modal traditional Chinese medicine data.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a multimodal structured processing method for traditional Chinese medicine data and related devices. Background Art

[0002] As a traditional Chinese medical system, Traditional Chinese Medicine (TCM) has diagnostic and treatment data accumulated over thousands of years of clinical practice, covering multi-dimensional information such as theory, diagnosis, treatment, and efficacy evaluation.

[0003] Traditional Chinese medicine diagnosis and treatment data are mainly text records, such as medical records and prescription notes. After modern Chinese medicine is integrated with hospital information systems, structured data such as electronic medical records and examination reports are gradually generated, but a large amount of unstructured or semi-structured content is still retained, such as handwritten medical records of Chinese medicine practitioners and tongue diagnosis images.

[0004] Therefore, there is an urgent need for a method that can perform structured processing on multimodal TCM data. Summary of the Invention

[0005] The main technical problem solved by this application is to provide a multimodal TCM data structured processing method and related devices to achieve structured processing of multimodal TCM data.

[0006] In order to solve the above technical problems, a technical solution adopted in this application is: to provide a multimodal structured processing method for traditional Chinese medicine data, the method comprising: obtaining traditional Chinese medicine diagnosis and treatment data, the traditional Chinese medicine diagnosis and treatment data comprising text medical record data, tongue image data and pulse diagnosis signal data; dividing the traditional Chinese medicine diagnosis and treatment data into a high-activity data level, a medium-activity data level and a low-activity data level according to the data activity label of the traditional Chinese medicine diagnosis and treatment data; based on the traditional Chinese medicine diagnosis and treatment data of the high-activity data level, generating a multimodal feature set comprising text symptom entity features, tongue description coding parameters and pulse description coding parameters; extracting the key features of the traditional Chinese medicine diagnosis and treatment data of the medium-activity data level, and outputting a symptom-prescription association feature set; integrating the syndrome type features of the traditional Chinese medicine diagnosis and treatment data of the low-activity data level; and performing feature mapping on the results of the multimodal feature set, the symptom-prescription association feature set and the syndrome type feature integration based on a preset dynamic rule mapping library.

[0007] Among them, the division operation of the data activity label specifically includes: in response to the generation timestamp of the traditional Chinese medicine diagnosis and treatment data not exceeding the first timeliness threshold, marking the traditional Chinese medicine diagnosis and treatment data as a high-activity data layer; in response to the generation timestamp of the traditional Chinese medicine diagnosis and treatment data being greater than the first timeliness threshold and not exceeding the second timeliness threshold, marking the traditional Chinese medicine diagnosis and treatment data as a medium-activity data layer; in response to the generation timestamp of the traditional Chinese medicine diagnosis and treatment data being greater than the second timeliness threshold, marking the traditional Chinese medicine diagnosis and treatment data as a low-activity data layer; wherein, the second timeliness threshold is greater than the first timeliness threshold.

[0008] Among them, based on the traditional Chinese medicine diagnosis and treatment data at the high-activity data level, a multimodal feature set including text symptom entity features, tongue description coding parameters and pulse description coding parameters is generated, including: parsing the symptom description text in the text medical record data through the dynamically expanded traditional Chinese medicine terminology library, extracting tongue diagnosis description text data and pulse diagnosis description text data; parsing the tongue diagnosis description text data, mapping it to the preset tongue color code, tongue coating color code and thickness code; parsing the pulse diagnosis description text data, mapping it to the preset pulse position intensity level code and pulse type code.

[0009] Among them, the construction of a dynamically expandable TCM terminology database includes: extracting a set of basic TCM term identifiers from historical medical case data sets; monitoring unregistered word combinations in newly added medical case data, and when the frequency of occurrence of unregistered word combinations exceeds a first frequency threshold, generating corresponding derivative identifiers and adding them to the dynamically expandable TCM terminology database; calculating the average monthly call frequency of each identifier, and when the average monthly call frequency is continuously lower than the decay frequency threshold, migrating the identifier to a low-priority term storage area.

[0010] Among them, the extraction of key features includes: comparing the text differences between the current follow-up data and the historical initial diagnosis data, extracting the new symptom entity features and deleting the symptom entity features; identifying the changes in the prescription composition, recording the names of new medicinal materials and deleting the names of medicinal materials; extracting the key feature word combinations in the tongue diagnosis description text.

[0011] Among them, the syndrome type features of traditional Chinese medicine diagnosis and treatment data at the low-activity data level are integrated, including: integrating tongue color coding, tongue coating color coding and thickness coding into tongue image feature vector; converting pulse position intensity level coding and pulse type coding into pulse image intensity distribution vector.

[0012] Among them, the method also includes: constructing a unique identifier for each patient to identify the patient's corresponding traditional Chinese medicine diagnosis and treatment data; in response to the existence of tongue diagnosis description text data and tongue image data under the same unique identifier, associating the tongue diagnosis description text data under the same unique identifier with the corresponding tongue image data; in response to the existence of pulse diagnosis description text data and pulse diagnosis signal data under the same unique identifier, associating the pulse diagnosis description text data under the same unique identifier with the corresponding pulse diagnosis signal data.

[0013] In order to solve the above technical problems, another technical solution adopted in this application is: to provide a multimodal traditional Chinese medicine data structuring processing device, which includes: an acquisition module, a stratification module, a generation module, an association extraction module, a syndrome type mapping module and a rule mapping module. Acquisition module: used to acquire TCM diagnosis and treatment data, which includes text medical record data, tongue image data and pulse diagnosis signal data; stratification module: used to divide TCM diagnosis and treatment data into high-activity data layer, medium-activity data layer and low-activity data layer according to the data activity label of TCM diagnosis and treatment data; generation module: used to generate a multimodal feature set including text symptom entity features, tongue description coding parameters and pulse description coding parameters based on TCM diagnosis and treatment data of high-activity data layer; association extraction module: used to extract key features of TCM diagnosis and treatment data of medium-activity data layer, and output symptom-prescription association feature set; syndrome mapping module: used to integrate syndrome type features of TCM diagnosis and treatment data of low-activity data layer; rule mapping module: used to perform feature mapping on the multimodal feature set, symptom-prescription association feature set and syndrome type feature integration results based on a preset dynamic rule mapping library.

[0014] To solve the above technical problems, another technical solution adopted in the present application is: to provide a multimodal TCM data structuring processing device, which includes: a memory and at least one processor, the memory stores instructions; at least one processor calls the instructions in the memory to enable the multimodal TCM data structuring processing device to perform the steps of the multimodal TCM data structuring processing method as any one of the above items.

[0015] In order to solve the above technical problems, another technical solution adopted in this application is: providing a computer-readable storage medium, on which instructions are stored, and when the instructions are processed and executed, the steps of the multimodal traditional Chinese medicine data structured processing method as described above are implemented.

[0016] Different from the existing technology, the beneficial effects of this application are: realizing the synchronous processing of three types of heterogeneous data, namely text medical records, tongue images and pulse diagnosis signals, and outputting a multimodal structured feature set including text symptom entity features, tongue image encoding parameters and pulse image encoding parameters. According to the hierarchical processing of data activity labels: high-activity data generates multimodal features; medium-activity data extracts symptom-prescription association features; low-activity data integrates syndrome type features, reducing computational redundancy and improving resource utilization. Multimodal features, symptom-prescription association features and syndrome type features are converted into configurable structured outputs through a dynamic rule mapping library. The conversion of unstructured text to machine-processable codes is completed, realizing structured processing of multimodal TCM data. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flowchart of an implementation method of the multimodal structured processing method of traditional Chinese medicine data of the present application.

[0018] Figure 2 This is a schematic diagram of the structural framework of an embodiment of the multimodal TCM data structured processing device of the present application.

[0019] Figure 3 This is a schematic diagram of the structural framework of an embodiment of the multimodal TCM data structured processing device of the present application. DETAILED DESCRIPTION

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0021] The terms "including," "having," and any variations thereof, as used in the embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to the process, method, product, or device.

[0022] The term "multiple" in this document means two or more than two. In addition, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0023] To facilitate understanding of this embodiment, a multimodal structured processing method for traditional Chinese medicine data disclosed in an embodiment of the present invention is first introduced in detail. Figure 1 As shown, Figure 1 This is a flow chart of an implementation method of the multimodal structured processing method of traditional Chinese medicine data of the present application, and the method includes the following steps.

[0024] Step S11: Acquire TCM diagnosis and treatment data, which includes text medical record data, tongue image data, and pulse diagnosis signal data.

[0025] Specifically, TCM diagnosis and treatment data is acquired through medical information system interfaces or dedicated data acquisition equipment. Textual medical record data is derived from structured forms or unstructured text records in the electronic medical record system. Tongue image data is generated using tongue image acquisition equipment under standardized lighting conditions. Pulse diagnosis data is collected using a universal pulse diagnosis instrument, which specifically generates radial artery pulse signals at the wrist through a pressure sensor array and converts them into a digital signal sequence.

[0026] In some specific embodiments, acquiring text medical record data includes at least one of the following methods: extracting text fields for the chief complaint, present medical history, and past medical history from the electronic medical record database of the hospital information system; and converting handwritten text in paper medical records into processable text data using optical character recognition technology. Tongue image data generation must meet the following conditions: acquisition under a D65 standard light source using a color image sensor with a resolution of at least 1920×1080 pixels; and the image must include the complete shape of the tongue's front and lateral contours. Pulse diagnosis signal data acquisition can be achieved by using an existing pulse diagnosis instrument to diagnose the patient's radial artery pulse to obtain pulse diagnosis signal data.

[0027] In some application scenarios, text medical record data may include descriptive records of the four diagnostic methods of Traditional Chinese Medicine, namely, inspection, auscultation, questioning, and palpation, such as "red tongue with thin white fur" or "thready pulse"; tongue image data can be associated with the shooting timestamp and patient position identification code; pulse diagnosis signal data can synchronously record the ambient temperature and humidity parameters at the time of collection.

[0028] Step S12: According to the data activity labels of the TCM diagnosis and treatment data, the TCM diagnosis and treatment data are divided into a high-activity data layer, a medium-activity data layer and a low-activity data layer.

[0029] Specifically, data activity labels are categorized based on the timeliness of TCM diagnosis and treatment data. By categorizing data by timeliness, we can prioritize the computation of different data types in subsequent processing: high-activity data requires real-time processing to support clinical decision-making, low-activity data is suitable for long-term syndrome pattern mining, and medium-activity data balances timeliness with in-depth analysis needs.

[0030] In some specific embodiments, data activity tag generation includes marking data to generate a timestamp through a system clock module; allocating high-activity data layers to a memory cache area to support low-latency access; and migrating low-activity data layers to a distributed storage cluster to enable a data compression algorithm.

[0031] In some embodiments, the data activity label division operation specifically includes: in response to the generation timestamp of the traditional Chinese medicine diagnosis and treatment data not exceeding the first timeliness threshold, marking the traditional Chinese medicine diagnosis and treatment data as a high-activity data layer; in response to the generation timestamp of the traditional Chinese medicine diagnosis and treatment data being greater than the first timeliness threshold and not exceeding the second timeliness threshold, marking the traditional Chinese medicine diagnosis and treatment data as a medium-activity data layer; in response to the generation timestamp of the traditional Chinese medicine diagnosis and treatment data being greater than the second timeliness threshold, marking the traditional Chinese medicine diagnosis and treatment data as a low-activity data layer; wherein the second timeliness threshold is greater than the first timeliness threshold.

[0032] In some specific embodiments, the timeliness threshold can be set based on the decaying value of TCM data. For example, the first timeliness threshold can correspond to the dynamic change cycle of symptoms, with a typical value of 7 days; the second timeliness threshold can correspond to the window period for analyzing the relationship between prescriptions and medicines, with a typical value of 30 days.

[0033] Step S13: Based on the TCM diagnosis and treatment data at the high-activity data level, a multimodal feature set including text symptom entity features, tongue image description coding parameters and pulse image description coding parameters is generated.

[0034] In some embodiments, based on the traditional Chinese medicine diagnosis and treatment data at the high-activity data level, a multimodal feature set including text symptom entity features, tongue description coding parameters and pulse description coding parameters is generated, including: parsing the symptom description text in the text medical record data through a dynamically expanded traditional Chinese medicine terminology library, extracting tongue diagnosis description text data and pulse diagnosis description text data; parsing the tongue diagnosis description text data, mapping it to a preset tongue color code, tongue coating color code and thickness code; parsing the pulse diagnosis description text data, mapping it to a preset pulse position strength level code and pulse type code.

[0035] Specifically, the code mapping association can be based on the national standard "Classification and Code of Traditional Chinese Medicine Diseases and Syndromes": the tongue color code is mapped to the national standard "Tongue Color Classification Code", for example, "Pale Red Tongue" is mapped to the code SZ01; the pulse type code is mapped to the national standard "Pulse Morphology Classification Code", for example, "String Pulse" is mapped to the code MX04, realizing the standardized expression of multimodal data and supporting cross-system data interaction.

[0036] In some embodiments, the construction of a dynamically expandable TCM terminology library includes: extracting a set of basic TCM terminology identifiers from historical medical case data sets; monitoring unregistered word combinations in newly added medical case data, and when the frequency of occurrence of unregistered word combinations exceeds a first frequency threshold, generating corresponding derivative identifiers and adding them to the dynamically expandable TCM terminology library; calculating the average monthly call frequency of each identifier, and when the average monthly call frequency is continuously lower than the decline frequency threshold, migrating the identifier to a low-priority term storage area.

[0037] Specifically, natural language processing techniques were used to extract frequently occurring TCM terms from historical medical case datasets, such as electronic medical records and classic medical case literature from hospitals within the past five years, to form a basic set of identifiers. For example, core terms such as "headache," "stringy pulse," and "red tongue with white fur" were extracted from 100,000 historical medical records. Each term was assigned a standard code based on the "Classification and Code of Traditional Chinese Medicine Diseases and Syndromes" (GB / T15657), such as A.07.01 for "headache." The technical significance of this step lies in establishing the authority of the terminology database through historical data, ensuring that the basic terminology is consistent with the national standard classification system, and avoiding semantic deviations in subsequent analysis.

[0038] In some specific embodiments, when the frequency of occurrence of a certain unregistered word combination exceeds a preset first frequency threshold, such as 50 times, within a certain period, the system will generate a corresponding derivative identifier, such as generating a code for "tooth-marked tongue" that complies with the national standard classification, and add it to the terminology library; then calculate the average monthly call frequency of each identifier. If the average monthly call frequency of a certain identifier is continuously lower than the decline frequency threshold, such as 10 times, the identifier will be migrated to the low-priority term storage area. This construction mechanism realizes the dynamic management of the terminology library through data-driven: basic term extraction ensures consistency with national standards, monitoring and adding unregistered words can adapt to new terms appearing in clinical practice, and activity management can eliminate low-frequency terms to optimize retrieval efficiency. For example, when "dampness and toxicity in the lungs" appears frequently in new medical records, the system will include it in the terminology library according to the rules, and at the same time migrate the continuously low-frequency "dai pulse" to the low-priority area.

[0039] Step S14: extract key features of TCM diagnosis and treatment data at the active data level, and output a symptom-prescription association feature set.

[0040] In some embodiments, the extraction of key features includes: comparing the text differences between the current follow-up data and the historical initial diagnosis data, extracting the new symptom entity features and deleting the symptom entity features; identifying the change features of the prescription composition, recording the names of new medicinal materials and deleting the names of medicinal materials; and extracting the key feature word combinations in the tongue diagnosis description text.

[0041] Specifically, the key feature extraction at the active data level can be achieved through the linkage of text semantic comparison and prescription knowledge graph. For example, when processing the diagnosis and treatment data of a patient, the system compares the initial diagnosis record "cough, fever, red tongue with thin white fur, given Yinqiao Powder" with the follow-up record "cough worsened, yellow and thick sputum, red tongue with yellow fur, adjusted prescription to Yinqiao Powder with Scutellaria baicalensis and removed Schizonepeta tenuifolia". Through natural language processing technology, the new symptom entity feature "yellow and thick sputum" and the deleted feature "fever" are extracted. At the same time, the changes in the prescription composition are analyzed, the newly added medicinal material "Scutellaria baicalensis" is recorded, the medicinal material "Schizonepeta tenuifolia" is deleted, and "yellow fur" is extracted from the tongue image description as the key feature word combination.

[0042] In some specific embodiments, the system can automatically associate historical initial diagnosis electronic medical records with chronic disease follow-up data within one to six months. For example, the initial diagnosis record of a diabetic patient is "drinking more and eating more, red tongue with little fluid, and a thin and rapid pulse, and a prescription for Xiaoke prescription is given." The follow-up record six months later is "drinking more and eating more, numbness in the hands and feet is newly added, the tongue is pale and dark with petechiae, and the pulse is thin and astringent, and the prescription is adjusted to Xiaoke prescription combined with Taohong Siwu Decoction." The system extracts symptom evolution features through text comparison, identifies newly added medicinal materials "peach kernel" and "safflower" with the help of a standard prescription library, and interprets the "pale and dark tongue with petechiae" in the tongue image description as a combination of tongue color code and petechiae features. Ultimately, a feature set containing the symptom-prescription relationship is generated, providing support for the structured analysis of TCM clinical data.

[0043] Step S15: Integrate the syndrome type characteristics of the TCM diagnosis and treatment data at the low-activity data level.

[0044] In some embodiments, syndrome type features are integrated for TCM diagnosis and treatment data at a low-activity data level, including: integrating tongue color coding, fur color coding, and thickness coding into a tongue image feature vector; and converting pulse position strength level coding and pulse type coding into a pulse image strength distribution vector.

[0045] Specifically, the integration of syndrome-type features at the low-activity data level can be achieved through multidimensional feature vectorization. For example, for a historical medical record older than six months, the tongue image diagnosis description corresponds to the tongue color code A.12.03 "red tongue," the coating color code B.03.10 "yellow coating," and the coating thickness code B.03.15 "thin coating." These three codes are combined in a preset order into a three-dimensional tongue image feature vector [A.12.03, B.03.10, B.03.15]. The pulse diagnosis description "thin and rapid pulse" corresponds to the pulse position intensity level code B.04.08 "thin and rapid pulse," and the pulse type code's rapid pulse feature is converted into a two-dimensional pulse intensity distribution vector [B.04.08, rapid pulse intensity parameter]. In this way, discrete codes are integrated into a structured feature vector.

[0046] Step S16: Based on the preset dynamic rule mapping library, feature mapping is performed on the results of the integration of the multimodal feature set, the symptom-prescription and drug association feature set, and the syndrome type feature.

[0047] Specifically, the preset dynamic rule mapping library adopts a three-layer mapping architecture to realize feature association. The multimodal feature set matches the text symptom entity features, tongue image codes and pulse image codes generated by high-activity data to the disease categories of "Classification and Code of Chinese Medicine Diseases and Syndromes" GB / T15657. For example, the multimodal feature set includes: [fever = A.01.01][red tongue = A.12.03][pulse = B.04.08], which is mapped to the wind-heat syndrome category of A.01.01 cold through the feature combination of fever and pulse, and is associated with the syndrome code of B.03.10 yellow tongue coating. Symptom-prescription is based on the symptom evolution of Chinese medicine activity data and the increase or decrease of medicinal materials to establish a mapping link of symptom change-prescription adjustment-syndrome evolution. For example, if a patient's newly added symptom of thick, yellow sputum corresponds to B.03.10, syndrome of heat evil invading the lungs, and Scutellaria baicalensis is added to the prescription, the rule base automatically associates Scutellaria baicalensis's heat-clearing and dampness-relieving properties with the syndrome code, generating a mapping relationship from "Scutellaria baicalensis → B.03.10." The results of syndrome feature integration are mapped by using a clustering algorithm to match the tongue image feature vectors and pulse intensity distribution vectors of low-activity data to pre-set syndrome templates. For example, the tongue image vectors [A.12.02 Pale tongue, B.03.11 White fur, B.03.22 Greasy fur] and the pulse vectors [B.04.02 Deep pulse, 0.3 Weak, B.04.07 Thin pulse] are matched by the rule base using a weighted algorithm to A.06.05, syndrome of spleen deficiency and dampness excess, and the standardized feature code [SP-007] for this syndrome is generated.

[0048] In some embodiments, the method also includes: constructing a unique identifier for each patient to identify the patient's corresponding traditional Chinese medicine diagnosis and treatment data; in response to the existence of tongue diagnosis description text data and tongue image data under the same unique identifier, associating the tongue diagnosis description text data under the same unique identifier with the corresponding tongue image data; in response to the existence of pulse diagnosis description text data and pulse diagnosis signal data under the same unique identifier, associating the pulse diagnosis description text data under the same unique identifier with the corresponding pulse diagnosis signal data.

[0049] Specifically, a patient's unique identifier can be constructed using an encrypted combination of a timestamp and identity information. For example, during outpatient registration, the system automatically generates a string consisting of the visit date (e.g., 20250616), the hospital code (e.g., ZYY001), and the last four digits of the patient's ID card (e.g., 1234). This string is encrypted using the SHA-256 algorithm and serves as a unique identifier. This identifier is used throughout the storage and access of all patient medical data. When the system detects the presence of both tongue image description text data (e.g., "red tongue with thin yellow fur") and tongue image data (e.g., a JPG-formatted tongue photo) under the same identifier, a bidirectional association is established using the distributed database's indexing mechanism. For example, an "identifier-tongue image" collection is created in MongoDB, storing a mapping between text encodings (e.g., A.12.03+B.03.10+B.03.15) and image storage paths (e.g., / hospital / images / 20250616ZYY0011234_tongue.jpg).

[0050] In some specific embodiments, the association of pulse data can be implemented in conjunction with an IoT device interface. For example, after a patient completes a pulse diagnosis on a TCM diagnostic device, the device automatically generates pulse diagnosis signal data, such as a waveform file in XML format. At the same time, the electronic medical record system records a pulse diagnosis description text, such as "thin and rapid pulse." The system retrieves the unique identifier "20250616ZYY0011234" and finds that both belong to the same patient. The system then stores the text-parsed pulse code, such as B.04.08+50, and the characteristic vector of the signal data, such as the frequency domain analysis parameter, under the same document node. This process can be implemented by connecting the hospital information system (HIS) to the API of the TCM diagnostic device, for example, using the HL7FHIR standard interface to complete data interaction, ensuring that the timestamp error between the text description and the signal data does not exceed 5 minutes.

[0051] In the above scheme, the synchronous processing of three types of heterogeneous data, namely text medical records, tongue images and pulse diagnosis signals, is achieved, and a multimodal structured feature set containing text symptom entity features, tongue image encoding parameters and pulse image encoding parameters is output. The data is processed hierarchically according to the activity labels: high-activity data generates multimodal features; medium-activity data extracts symptom-prescription association features; low-activity data integrates syndrome type features, reducing computational redundancy and improving resource utilization. The multimodal features, symptom-prescription association features and syndrome type features are converted into configurable structured outputs through a dynamic rule mapping library. The conversion of unstructured text into machine-processable codes is completed, realizing the structured processing of multimodal TCM data.

[0052] See also Figure 2 , Figure 2 This is a schematic diagram of the structural framework of an embodiment of the multimodal TCM data structured processing device of the present application. Figure 2As shown, the multimodal TCM data structured processing device 20 includes: an acquisition module 21, a stratification module 22, a generation module 23, a correlation extraction module 24, a syndrome type mapping module 25 and a rule mapping module 26. Acquisition module 21: used to acquire TCM diagnosis and treatment data, which includes text medical record data, tongue image data and pulse diagnosis signal data; stratification module 22: used to divide TCM diagnosis and treatment data into high-activity data layer, medium-activity data layer and low-activity data layer according to the data activity label of TCM diagnosis and treatment data; generation module 23: used to generate a multimodal feature set including text symptom entity features, tongue description coding parameters and pulse description coding parameters based on TCM diagnosis and treatment data of high-activity data layer; association extraction module 24: used to extract key features of TCM diagnosis and treatment data of medium-activity data layer, and output symptom-prescription association feature set; syndrome mapping module 25: used to integrate syndrome type features of TCM diagnosis and treatment data of low-activity data layer; rule mapping module 26: used to integrate multimodal feature set, symptom-prescription association feature set and syndrome type feature integration into results for feature mapping based on a preset dynamic rule mapping library.

[0053] In some specific embodiments, the data activity label division operation of the stratification module 22 specifically includes: in response to the generation timestamp of the traditional Chinese medicine diagnosis and treatment data not exceeding the first timeliness threshold, marking the traditional Chinese medicine diagnosis and treatment data as a high-activity data layer; in response to the generation timestamp of the traditional Chinese medicine diagnosis and treatment data being greater than the first timeliness threshold and not exceeding the second timeliness threshold, marking the traditional Chinese medicine diagnosis and treatment data as a medium-activity data layer; in response to the generation timestamp of the traditional Chinese medicine diagnosis and treatment data being greater than the second timeliness threshold, marking the traditional Chinese medicine diagnosis and treatment data as a low-activity data layer; wherein, the second timeliness threshold is greater than the first timeliness threshold.

[0054] In some embodiments, the generation module 23 generates a multimodal feature set including text symptom entity features, tongue description coding parameters and pulse description coding parameters based on the traditional Chinese medicine diagnosis and treatment data at the high-activity data level, including: parsing the symptom description text in the text medical record data through the dynamically expanded traditional Chinese medicine terminology library, extracting tongue diagnosis description text data and pulse diagnosis description text data; parsing the tongue diagnosis description text data, mapping it to a preset tongue color code, tongue coating color code and thickness code; parsing the pulse diagnosis description text data, mapping it to a preset pulse position intensity level code and pulse type code.

[0055] In some embodiments, the construction of a dynamically expandable TCM terminology library includes: extracting a set of basic TCM terminology identifiers from historical medical case data sets; monitoring unregistered word combinations in newly added medical case data, and when the frequency of occurrence of unregistered word combinations exceeds a first frequency threshold, generating corresponding derivative identifiers and adding them to the dynamically expandable TCM terminology library; calculating the average monthly call frequency of each identifier, and when the average monthly call frequency is continuously lower than the decline frequency threshold, migrating the identifier to a low-priority term storage area.

[0056] In some embodiments, the association extraction module 24: extraction of key features includes: comparing the text differences between the current follow-up data and the historical initial diagnosis data, extracting the new symptom entity features and deleting the symptom entity features; identifying the change features of the prescription composition, recording the new medicinal material names and deleting the medicinal material names; extracting the key feature word combinations in the tongue diagnosis description text.

[0057] In some embodiments, the syndrome mapping module 25: integrates syndrome characteristics of TCM diagnosis and treatment data at the low-activity data level, including: integrating tongue color coding, tongue coating color coding and thickness coding into tongue image feature vector; converting pulse position intensity level coding and pulse type coding into pulse intensity distribution vector.

[0058] The system also includes: constructing a unique identifier for each patient to identify the patient's corresponding traditional Chinese medicine diagnosis and treatment data; in response to the existence of tongue diagnosis description text data and tongue image data under the same unique identifier, associating the tongue diagnosis description text data under the same unique identifier with the corresponding tongue image data; in response to the existence of pulse diagnosis description text data and pulse diagnosis signal data under the same unique identifier, associating the pulse diagnosis description text data under the same unique identifier with the corresponding pulse diagnosis signal data.

[0059] In the above scheme, the synchronous processing of three types of heterogeneous data, namely text medical records, tongue images and pulse diagnosis signals, is achieved, and a multimodal structured feature set containing text symptom entity features, tongue image encoding parameters and pulse image encoding parameters is output. The data is processed hierarchically according to the activity labels: high-activity data generates multimodal features; medium-activity data extracts symptom-prescription association features; low-activity data integrates syndrome type features, reducing computational redundancy and improving resource utilization. The multimodal features, symptom-prescription association features and syndrome type features are converted into configurable structured outputs through a dynamic rule mapping library. The conversion of unstructured text into machine-processable codes is completed, realizing the structured processing of multimodal TCM data.

[0060] Figure 3This is a schematic diagram of the structural framework of one embodiment of the multimodal TCM data structured processing device of the present application. The multimodal TCM data structured processing device 30 may have relatively large differences due to different configurations or performances, and may include one or more processors 31 and memory 32. The processor 31 may be configured to communicate with the memory 32, and execute a series of instruction operations in the memory on the multimodal TCM data structured processing device to implement the steps of the above-mentioned multimodal TCM data structured processing method. Those skilled in the art will understand that Figure 3 The structure of the multimodal TCM data structuring processing device shown does not constitute a limitation on the multimodal TCM data structuring processing device provided by the present invention, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0061] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the steps of a multimodal traditional Chinese medicine data structured processing method.

[0062] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0063] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the various embodiments of the method of the present invention. The aforementioned storage medium includes various media that can store program code, such as USB flash drives, mobile hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0064] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A multimodal structured processing method for traditional Chinese medicine data, characterized in that: The method includes: acquiring traditional Chinese medicine diagnosis and treatment data, wherein the traditional Chinese medicine diagnosis and treatment data includes text medical record data, tongue image data and pulse diagnosis signal data; dividing the traditional Chinese medicine diagnosis and treatment data into a high-activity data layer, a medium-activity data layer and a low-activity data layer according to data activity labels of the traditional Chinese medicine diagnosis and treatment data; generating a multimodal feature set including text symptom entity features, tongue description coding parameters and pulse description coding parameters based on the traditional Chinese medicine diagnosis and treatment data of the high-activity data layer; extracting key features of the traditional Chinese medicine diagnosis and treatment data of the medium-activity data layer and outputting a symptom-prescription association feature set; performing syndrome type feature integration on the traditional Chinese medicine diagnosis and treatment data of the low-activity data layer; and performing feature mapping on the multimodal feature set, the symptom-prescription association feature set and the results of the syndrome type feature integration based on a preset dynamic rule mapping library.

2. The multimodal structured processing method of traditional Chinese medicine data according to claim 1, characterized in that: The data activity label division operation specifically includes: in response to the generation timestamp of the TCM diagnosis and treatment data not exceeding the first timeliness threshold, marking the TCM diagnosis and treatment data as the high-activity data layer; in response to the generation timestamp of the TCM diagnosis and treatment data being greater than the first timeliness threshold and not exceeding the second timeliness threshold, marking the TCM diagnosis and treatment data as the medium-activity data layer; in response to the generation timestamp of the TCM diagnosis and treatment data being greater than the second timeliness threshold, marking the TCM diagnosis and treatment data as the low-activity data layer; wherein, the second timeliness threshold is greater than the first timeliness threshold.

3. The multimodal structured processing method of traditional Chinese medicine data according to claim 1, characterized in that: The TCM diagnosis and treatment data based on the high-activity data level generates a multimodal feature set including text symptom entity features, tongue description coding parameters and pulse description coding parameters, including: parsing the symptom description text in the text medical record data through a dynamically expanded TCM terminology library to extract tongue diagnosis description text data and pulse diagnosis description text data; parsing the tongue diagnosis description text data and mapping it to preset tongue color code, tongue coating color code and thickness code; parsing the pulse diagnosis description text data and mapping it to preset pulse position intensity level code and pulse type code.

4. The multimodal structured processing method of traditional Chinese medicine data according to claim 3, characterized in that: The construction of the dynamically expandable TCM terminology library includes: extracting a set of basic TCM term identifiers from a historical medical case data set; monitoring unregistered word combinations in newly added medical case data, and when the frequency of occurrence of the unregistered word combination exceeds a first frequency threshold, generating corresponding derived identifiers and adding them to the dynamically expandable TCM terminology library; calculating the average monthly call frequency of each identifier, and when the average monthly call frequency is continuously lower than a decay frequency threshold, migrating the identifier to a low-priority term storage area.

5. The multimodal structured processing method of traditional Chinese medicine data according to claim 1, characterized in that: The extraction of the key features includes: comparing the text differences between the current follow-up data and the historical initial diagnosis data, extracting the new symptom entity features and the deleted symptom entity features; identifying the change features of the prescription composition, recording the names of the new medicinal materials and the deleted medicinal materials; and extracting the key feature word combinations in the tongue diagnosis description text.

6. The multimodal structured processing method for traditional Chinese medicine data according to claim 1 or 5, characterized in that: The syndrome type feature integration of the TCM diagnosis and treatment data at the low-activity data level includes: integrating the tongue color code, tongue coating color code and thickness code into a tongue image feature vector; and converting the pulse position strength level code and pulse type code into a pulse image strength distribution vector.

7. The multimodal structured processing method of traditional Chinese medicine data according to claim 3, characterized in that: The method also includes: constructing a unique identifier for each patient to identify the traditional Chinese medicine diagnosis and treatment data corresponding to the patient; in response to the existence of the tongue diagnosis description text data and the tongue image data under the same unique identifier, associating the tongue diagnosis description text data under the same unique identifier with the corresponding tongue image data; in response to the existence of the pulse diagnosis description text data and the pulse diagnosis signal data under the same unique identifier, associating the pulse diagnosis description text data under the same unique identifier with the corresponding pulse diagnosis signal data.

8. A multimodal TCM data structured processing device, characterized in that: The multimodal TCM data structuring processing device includes: an acquisition module for acquiring TCM diagnosis and treatment data, wherein the TCM diagnosis and treatment data includes text medical record data, tongue image data and pulse diagnosis signal data; a stratification module for dividing the TCM diagnosis and treatment data into a high-activity data layer, a medium-activity data layer and a low-activity data layer according to the data activity label of the TCM diagnosis and treatment data; a generation module for generating a multimodal feature set including text symptom entity features, tongue description coding parameters and pulse description coding parameters based on the TCM diagnosis and treatment data of the high-activity data layer; an association extraction module for extracting key features of the TCM diagnosis and treatment data of the medium-activity data layer and outputting a symptom-prescription association feature set; a syndrome mapping module for integrating syndrome type features of the TCM diagnosis and treatment data of the low-activity data layer; a rule mapping module for performing feature mapping on the multimodal feature set, the symptom-prescription association feature set and the results of the syndrome type feature integration based on a preset dynamic rule mapping library.

9. A multimodal TCM data structured processing device, characterized in that: The multimodal TCM data structuring processing device includes: a memory and at least one processor, the memory storing instructions; the at least one processor calls the instructions in the memory so that the multimodal TCM data structuring processing device performs the steps of the multimodal TCM data structuring processing method as described in any one of claims 1-7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are processed and executed, the steps of the multimodal traditional Chinese medicine data structured processing method as described in any one of claims 1-7 are implemented.

Images