Tumor diagnosis method and device based on meridian digital human and continuous behavior data

Abstract

The invention provides a tumor diagnosis method and device based on a meridian digital person and continuous behavior data. The method comprises the following steps: constructing the meridian digital person based on current physical state information of a patient to be diagnosed; collecting continuous behavior data of a patient to be diagnosed, and mapping each continuous behavior data of the patient to be diagnosed to the meridian digital person based on the meridian mapping table for visual display; and generating pathological image information based on a visual display result of the meridian digital human, then obtaining pathological text description of the pathological image information, and inputting the corresponding continuous behavior data, the pathological image information and the pathological text description into a pre-trained tumor diagnosis model to obtain a tumor diagnosis result of the to-be-diagnosed patient. According to the scheme, by capturing association and potential rules among different modal data, tumor features in different expression forms are accurately identified, and the applicability and reliability of diagnosis in various complex clinical scenes are improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a tumor diagnosis method and device based on meridian digital human and continuous behavior data. Background Art

[0002] With the deep integration of modern medicine and information technology, early diagnosis and precise treatment of tumors are increasingly valued. Traditional tumor diagnosis methods often rely on biochemical tests, medical imaging examinations, and pathological biopsies. Although these methods can help doctors discover tumors and determine their nature to a certain extent, there are some limitations. For example, some examinations are invasive and cause pain to patients, and the imaging features of some early tumors may not be obvious enough, which can easily lead to missed diagnosis.

[0003] At the same time, as a treasure of traditional Chinese medicine, Chinese medicine has a unique meridian theory system, which believes that the running state of qi and blood in the human meridians is closely related to health. In Chinese medicine's understanding of tumors, tumors show specific characteristics in terms of qi, blood, and meridians. From the perspective of qi and blood, qi and blood stasis is one of the common pathological bases for tumor formation. When the human body's qi is not smooth, the driving effect of qi is weakened, blood circulation is blocked, and blood stasis is easily gathered to form a mass over time. At the same time, qi and blood deficiency is also closely related to tumors. As a consumable pathological product, tumors will continuously consume the body's positive qi, resulting in a lack of source for qi and blood biochemistry and a gradual deficiency of qi and blood.

[0004] However, traditional Chinese medicine generally uses observation, auscultation, questioning and palpation to make a diagnosis. This diagnostic method is limited by the doctor's experience and has certain challenges in quantification and customization. It is difficult to accurately and in real time reflect the complex physiological and pathological changes in the human body.

[0005] Digital twin technology and artificial intelligence pre-trained models have shown great application potential in the medical field. Digital twin technology can achieve real-time mapping and interaction between physical entities and virtual digital models, making it possible to accurately present the dynamic physiological state of the human body. Artificial intelligence pre-trained models have achieved many results in the field of disease diagnosis with their powerful data learning and feature extraction capabilities. How to use digital twin technology and artificial intelligence pre-trained models for traditional Chinese medicine tumor diagnosis is a difficult problem that existing technologies urgently need to solve. Summary of the invention

[0006] The embodiment of the present application provides a tumor diagnosis method and device based on meridian digital human and continuous behavior data. A meridian digital human is constructed based on the physical state information of the patient to be diagnosed, and continuous behavior data is collected and a meridian mapping table is established to clarify its impact on the circulation of qi and blood and meridian information. Subsequently, a tumor diagnosis model composed of image, text, and continuous data feature processing branches is used to process pathological images, text descriptions, and continuous behavior data respectively with the help of convolutional layers, multi-head self-attention mechanisms, LSTM and other technologies, and perform feature fusion, and finally accurately obtain the probability distribution of patients suffering from tumors in different locations.

[0007] In a first aspect, the present application provides a tumor diagnosis method based on meridian digital human and continuous behavior data, the method comprising: Constructing a meridian digital human based on the current physical condition information of the patient to be diagnosed, wherein the meridian digital human includes the qi and blood circulation information and meridian information of the patient to be diagnosed; Collecting continuous behavior data of the patient to be diagnosed, the continuous behavior data includes four types of health index data, diet data, exercise data and sleep data, establishing a meridian mapping table, defining the influence of the continuous behavior data on the qi and blood circulation information and meridian information in the meridian mapping table, and mapping each continuous behavior data of the patient to be diagnosed to the meridian digital human for visual display based on the meridian mapping table; Pathological image information is generated based on the visualization display results of the meridian digital human, and then the pathological text description of the pathological image information is obtained. The corresponding continuous behavioral data, pathological image information and pathological text description are input into the pre-trained tumor diagnosis model to obtain the tumor diagnosis result of the patient to be diagnosed.

[0008] In a second aspect, the embodiment of the present application provides a tumor diagnosis device based on meridian digital human and continuous behavior data, including: A construction module, which constructs a meridian digital human based on the current physical condition information of the patient to be diagnosed, wherein the meridian digital human includes the qi and blood circulation information and meridian information of the patient to be diagnosed; A collection module is used to collect continuous behavior data of the patient to be diagnosed, wherein the continuous behavior data includes four types: health index data, diet data, exercise data, and sleep data, establish a meridian mapping table, define the influence of the continuous behavior data on the qi and blood circulation information and meridian information in the meridian mapping table, and map each continuous behavior data of the patient to be diagnosed to the meridian digital human for visual display based on the meridian mapping table; The diagnosis module generates pathological image information based on the visualization display results of the meridian digital human, then obtains the pathological text description of the pathological image information, and inputs the corresponding continuous behavior data, pathological image information and pathological text description into the pre-trained tumor diagnosis model to obtain the tumor diagnosis result of the patient to be diagnosed.

[0009] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute a tumor diagnosis method based on meridian digital human and continuous behavior data.

[0010] In a fourth aspect, an embodiment of the present application provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute a process, and the process includes a tumor diagnosis method based on meridian digital human and continuous behavior data.

[0011] The main contributions and innovations of the present invention are as follows: The embodiment of the present application comprehensively considers multiple types of information. It not only constructs a meridian digital human covering the circulation of qi and blood and meridian information based on the current physical state information of the patient to be diagnosed, but also collects various types of continuous behavioral data including health indicators, diet, exercise, sleep, etc., so that the basis for diagnosis is more comprehensive; the embodiment of the present application establishes a meridian mapping table to clarify the impact of continuous behavioral data on the circulation of qi and blood and meridian information, and can map the continuous behavioral data to the meridian digital human for visual display, and intuitively present the mutual relationship; the embodiment of the present application uses convolutional layers, multi-head self-attention mechanisms, LSTM and other technical means to extract features of pathological image information, pathological text descriptions, and continuous behavioral data respectively, and reasonably map them to preset dimensions, and then obtains tumor diagnosis results through the tumor prediction layer after fusing features through the feature fusion layer. By capturing the associations and potential laws between different modal data, tumor characteristics under different manifestations are accurately identified, thereby improving the applicability and reliability of diagnosis in various complex clinical scenarios.

[0012] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 is a flow chart of a tumor diagnosis method based on meridian digital human and continuous behavior data according to an embodiment of the present application; Figure 2is a structural block diagram of a tumor diagnosis device based on meridian digital human and continuous behavior data according to an embodiment of the present application; Figure 3 It is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0014] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0015] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0016] Embodiment 1 The embodiment of the present application provides a tumor diagnosis method based on meridian digital human and continuous behavior data. A meridian digital human is constructed based on the physical state information of the patient to be diagnosed, and continuous behavior data is collected and a meridian mapping table is established to clarify its influence on the circulation of qi and blood and meridian information. Subsequently, a tumor diagnosis model composed of image, text, and continuous data feature processing branches is used to process pathological images, text descriptions, and continuous behavior data respectively with the help of convolutional layers, multi-head self-attention mechanisms, LSTM and other technologies, and perform feature fusion, and finally accurately obtain the probability distribution of patients suffering from tumors in different locations. Specifically, refer to Figure 1 , the method comprising: Constructing a meridian digital human based on the current physical condition information of the patient to be diagnosed, wherein the meridian digital human includes the qi and blood circulation information and meridian information of the patient to be diagnosed; Collecting continuous behavior data of the patient to be diagnosed, the continuous behavior data includes four types of health index data, diet data, exercise data and sleep data, establishing a meridian mapping table, defining the influence of the continuous behavior data on the qi and blood circulation information and meridian information in the meridian mapping table, and mapping each continuous behavior data of the patient to be diagnosed to the meridian digital human for visual display based on the meridian mapping table; Pathological image information is generated based on the visualization display results of the meridian digital human, and then the pathological text description of the pathological image information is obtained. The corresponding continuous behavioral data, pathological image information and pathological text description are input into the pre-trained tumor diagnosis model to obtain the tumor diagnosis result of the patient to be diagnosed.

[0017] In some embodiments, physical examination data and medical imaging examination data of the patient to be diagnosed are collected as current body state information, and the current body state information is input into a constructed TCM diagnosis model to obtain Qi and blood circulation information and meridian information of the patient to be diagnosed, and then the Qi and blood circulation information and meridian information of the patient to be diagnosed are mapped into a standard human body model to obtain a meridian digital person corresponding to the patient to be diagnosed.

[0018] Specifically, routine physical examination data in the hospital system include blood sugar, blood lipids, liver function, kidney function, etc., while medical imaging data include X-ray, CT, MRI, etc. These routine physical examination data can comprehensively reflect a person's physical condition and possible pathological changes, and have potential correlation with qi, blood, and meridians in traditional Chinese medicine.

[0019] Specifically, the TCM diagnosis model is a neural network model. During the training process of the TCM diagnosis model, routine physical examination data and medical imaging data of a large number of patients are first obtained in the hospital system, and these data are pre-processed as training samples. Then, the Qi and blood circulation information and meridian information in the TCM diagnosis results of each patient are obtained to mark each training sample, and the TCM diagnosis model is trained with the marked training samples.

[0020] Specifically, the neural network model can be a deep neural network DNN or a convolutional neural network CNN, which can better handle the relationship between complex human meridians and multi-source data through the powerful nonlinear fitting ability of the neural network.

[0021] Specifically, the step of preprocessing the current physical state information includes: data cleaning and data standardization and normalization. In the process of data cleaning, repeated, erroneous or obviously illogical data records are removed, such as data points in physical examination data that exceed the normal physiological range too much and are obviously inconsistent with medical common sense. In the step of data standardization and normalization, various indicators of the patient are uniformly converted into a standard normal distribution form with a mean of 0 and a standard deviation of 1, so that various types of data are at the same comparable level in subsequent model analysis, avoiding deviations in model training due to large differences in data levels. Some data with limited ranges are normalized and mapped to the range of 0-1 to facilitate unified management and model operations.

[0022] Furthermore, the standard human body model in this solution is modeled by 3DMAX or other modeling software based on the real body structure of the human body, and the standard human body model includes detailed internal structures of the human body such as the skull, organs, blood vessels, and veins.

[0023] Exemplarily, different colors are used to map the Qi and blood circulation information of the patient to be diagnosed in the standard human body model. For example, the part with smooth Qi and blood circulation is displayed in red, the part with slightly poor Qi and blood circulation is displayed in yellow, and the part with poor Qi and blood circulation is displayed in blue. The thickness of the line is used to map the degree of meridian patency of the patient to be diagnosed in the standard human body model. The thicker the line, the smoother the meridian, and the thinner the line, the more congested the meridian.

[0024] In some embodiments, continuous behavioral data of the patient to be tested is obtained through a smart wearable device or a recording APP. For example, the patient's behavior is monitored through a smart watch, bracelet, etc., so as to obtain the health indicator data, exercise data and sleep data of the patient to be diagnosed. The dietary data of the patient to be diagnosed can be obtained by recording through the mobile phone / watch APP. At the same time, some health indicator data that cannot be measured by smart wearable devices but still need to be measured regularly, such as blood pressure, blood sugar, etc., can also be recorded through the mobile phone / watch APP.

[0025] Specifically, the continuous behavior data in this solution is monitored and recorded in real time, but the continuous behavior data is collected periodically, for example, the continuous behavior data is collected once a month or half a month, and the continuous behavior data is visualized and displayed in the meridian digital human based on the collection results.

[0026] In some embodiments, each type of continuous behavior data is first discretized, and then all permutations and combinations between the discretization results of different types of continuous behavior data are obtained to obtain a first data set, and each element in the first data set is input into a pre-trained meridian mapping model to obtain the influence of each permutation and combination on the qi and blood circulation information and meridian information to complete the establishment of the meridian mapping table.

[0027] For example, various possible continuous behavioral data can be covered by reasonably discretizing the value range of continuous behavioral data, thereby facilitating organization into a tabular form. For example, sleep time can be discretized into less than 6 hours, 6-8 hours, and more than 8 hours, thereby covering all sleep duration ranges.

[0028] Exemplarily, if the health indicator data include 10 results such as blood pressure range and blood sugar range after discretization, the dietary data include 13 results such as protein intake, vegetable intake, and carbohydrate intake, the exercise data include 5 results such as exercise intensity and exercise time, and the sleep data include 8 results such as sleep duration and sleep quality, then the first data set includes all possible combinations of all the above results, wherein, when discretizing continuous behavior data, different indicators in different types of continuous behavior data are discretized separately, for example, blood pressure data and blood sugar data are discretized separately in the health indicator data.

[0029] Specifically, in the process of using the meridian mapping model to establish the meridian mapping table, when the input behavioral data combination is "low vegetable intake in the diet, low exercise intensity, and less than 6 hours of sleep", the model outputs the corresponding meridian (such as the Stomach Meridian of Foot-Yangming) with a patency of 3 points (out of 10 points), and a Qi and blood sufficiency of 4 points, etc. The corresponding relationships between these inputs and outputs are filled in one by one into a pre-designed table. The rows of the table correspond to different behavioral data combinations, and the columns correspond to different meridians and Qi and blood circulation-related indicators, ultimately forming a complete meridian mapping table.

[0030] Furthermore, the meridian mapping model is a decision tree model, which is obtained by acquiring continuous behavioral data of a large number of patients in the hospital system and training the constructed decision tree with the diagnosis of traditional Chinese medicine experts as a benchmark to obtain the meridian mapping model.

[0031] In some embodiments, in the step of "mapping each continuous behavior data of the patient to be diagnosed into the meridian digital human for visual display based on the meridian mapping table", each type of continuous behavior data of the patient to be diagnosed is discretized to obtain a discretization result, and based on the discretization result, the corresponding mapping item is obtained in the meridian mapping table, and then the mapping item is mapped to the corresponding meridian digital human.

[0032] Exemplarily, this solution uses color to visualize the information on the circulation of Qi and blood, and uses the thickness of lines to visualize the meridian information. For example, when the patency of a meridian is 5 points, the thickness of the corresponding meridian will be adjusted in the meridian digital human. When the degree of Qi and blood circulation at a certain position is 10 points out of the full score, the color of the corresponding position in the meridian digital human will be changed to red to indicate that the Qi and blood circulation at that position is excellent.

[0033] Specifically, as the continuous behavior data is continuously updated over time, this solution can reflect the updated results on the meridian digital human in real time by linking the continuous behavior data with the meridian digital human, thereby realizing dynamic tracking of the individual's health status. For example, a person begins to adjust his diet structure, increasing the intake of vegetables and reducing the intake of high-oil and high-salt foods. After a period of time, the meridian digital human can intuitively see that the qi and blood status of the spleen and stomach meridians and other meridians related to it is gradually improving, and the actual effect of health intervention measures can be clearly known, which is convenient for timely adjustment of lifestyle or treatment plans. For doctors, traditional diagnosis often relies on the patient's subjective description and limited examination results. Combining the linkage of continuous behavior data and meridian digital human can provide more objective and comprehensive diagnostic basis from the perspective of meridian qi and blood operation. For example, when facing a patient who complains of physical fatigue and poor digestion, in addition to routine examinations, checking his diet data and the state of qi and blood deficiency and poor operation of the spleen and stomach meridians displayed by the meridian digital human can more accurately judge the condition, provide more sufficient evidence for disease diagnosis, and improve the accuracy of diagnosis.

[0034] In some specific embodiments, a convolutional layer is used to extract features of the pathological image information to obtain pathological image features, and then semantic understanding of the pathological image features is performed to obtain a pathological text description.

[0035] Specifically, since different pathological image information may differ in resolution, color mode, etc., when extracting features from the pathological image information, the pathological image information must first be preprocessed to unify the size, height, and contrast of the pathological image information, and map the range of each pixel value to the feature interval to ensure the consistency of the pathological image information input into the convolutional layer, so as to avoid interference with subsequent pathological image information generation and diagnosis due to differences in the image itself.

[0036] Furthermore, a semantic understanding module is constructed through the NIP technology to perform semantic understanding on the pathological image features to obtain a pathological text description corresponding to the pathological image information.

[0037] For example, if the image shows that the circulation of Qi and blood in the Stomach Meridian of Foot-Yangming is stagnant, the pathological text information obtained through semantic understanding through NIP technology is "Qi and blood are blocked in the Stomach Meridian of Foot-Yangming, and there may be pathological changes related to meridian obstruction."

[0038] In some embodiments, the tumor diagnosis model is composed of an image feature processing branch, a text feature processing branch, a continuous data feature processing branch, a feature fusion layer and a tumor prediction layer, wherein the image feature processing branch, the text feature processing branch and the continuous data feature processing branch are in a parallel structure, in the image feature processing branch, the pathological image information is firstly feature extracted to obtain a first feature, and then the first feature is mapped to a preset dimension by the image feature mapping layer, in the text feature processing branch, the pathological text description is first feature extracted based on a multi-head self-attention mechanism to obtain a second feature, and then the text feature mapping layer maps the second feature to a preset dimension, in the continuous data feature processing branch, the continuous behavior data is feature extracted based on LSTM to obtain a third feature, and then the continuous data feature mapping layer maps the third feature to a preset dimension, the feature fusion layer fuses the first feature, the second feature and the third feature under the preset dimension to obtain a fusion feature, the tumor prediction layer performs tumor prediction based on the fusion feature to obtain a tumor diagnosis result, and the tumor diagnosis result is the probability distribution of the patient to be diagnosed having tumors in different locations.

[0039] Specifically, this solution uses a feature mapping layer in the image feature processing branch, the text feature processing branch, and the continuous data feature processing branch to respectively map the first feature, the second feature, and the third feature to the same preset dimension, so that different types of feature information can be fused under the same preset dimension to improve the accuracy of tumor prediction.

[0040] Specifically, in the image feature processing branch, a combination of multiple convolutional layers and pooling layers is used to extract local features in the pathological image information, such as the morphology, color, texture and other features of the meridians in the pathological image, as well as abnormal area features that may be related to the tumor.

[0041] Specifically, in the text feature processing branch, a Transformer-based encoder is used to extract features from word embeddings, and then a multi-head self-attention mechanism is used to analyze the semantic associations and importance information between words, and to extract feature vectors that can reflect the core semantics of pathological text descriptions.

[0042] Specifically, since continuous behavior data has a strong timeliness, LSTM is used in the continuous data feature processing branch to process the continuous behavior data, so as to effectively capture the long-term dependencies in the sequence. Through cyclic calculations of multiple time steps, the law of how the behavior data changes over time and the correlation information between data at different time points are learned, and the hidden state vector reflecting the time series characteristics is extracted.

[0043] Furthermore, a fully connected layer is added between the feature fusion layer and the tumor prediction layer, and the fully connected layer performs fully connected processing on the fusion features and inputs the fully connected results into the tumor prediction layer.

[0044] Specifically, the fully connected layer can fully integrate and complement the features of different modalities based on the fusion features, and mine more valuable comprehensive feature information for subsequent tumor location probability inference.

[0045] Furthermore, the tumor prediction layer makes predictions based on fusion features by using multiple neurons, and the number of neurons is the same as the tumor location. If one wants to predict the probability of tumor incidence in the brain, lungs, and stomach, the number of neurons in the tumor prediction layer is 3, corresponding to the brain, lungs, and stomach, respectively.

[0046] Specifically, the occurrence and development of tumors are often accompanied by complex and diverse physical changes, and the manifestations in different individuals are also different, which may involve changes in multiple systems and multiple levels. With the characteristics of multimodal fusion, this model can better adapt to this complex manifestation of the disease, capture the associations and potential laws between different modal data, accurately identify tumor characteristics under different manifestations, and improve the applicability and reliability of diagnosis in various complex clinical scenarios. The tumor diagnosis model in this solution accepts data from different modalities at the same time, so as to fully explore and utilize the unique value contained in various types of data, avoid the limitations of relying solely on a single modality for diagnosis, and enable the diagnosis results to integrate multiple clues and be more comprehensive and accurate.

[0047] Embodiment 2 Based on the same idea, refer to Figure 2 , the present application also proposes a tumor diagnosis device based on meridian digital human and continuous behavior data, including: A construction module, which constructs a meridian digital human based on the current physical condition information of the patient to be diagnosed, wherein the meridian digital human includes the qi and blood circulation information and meridian information of the patient to be diagnosed; A collection module is used to collect continuous behavior data of the patient to be diagnosed, wherein the continuous behavior data includes four types: health index data, diet data, exercise data, and sleep data, establish a meridian mapping table, define the influence of the continuous behavior data on the qi and blood circulation information and meridian information in the meridian mapping table, and map each continuous behavior data of the patient to be diagnosed to the meridian digital human for visual display based on the meridian mapping table; The diagnosis module generates pathological image information based on the visual display results of the meridian digital human, then obtains the pathological text description of the pathological image information, and inputs the corresponding continuous behavior data, pathological image information and pathological text description into the pre-trained tumor diagnosis model to obtain the tumor diagnosis result of the patient to be diagnosed.

[0048] Embodiment 3 This embodiment also provides an electronic device, referring to Figure 3 , comprises a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to execute the steps in any of the above method embodiments.

[0049] Specifically, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0050] Among them, the memory 404 may include a large capacity memory 404 for data or instructions. For example, but not limitation, the memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In appropriate cases, the memory 404 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 404 may be inside or outside the data processing device. In a specific embodiment, the memory 404 is a non-volatile memory. In a specific embodiment, the memory 404 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (Erasable Programmable Read-Only Memory, EPROM for short), an electrically erasable PROM (Electrically Erasable Programmable Read-Only Memory, EEPROM for short), an electrically alterable ROM (Electrically Alterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. In appropriate circumstances, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM may be a fast page mode dynamic random access memory 404 (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0051] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0052] The processor 402 reads and executes the computer program instructions stored in the memory 404 to implement any one of the tumor diagnosis methods based on meridian digital human and continuous behavior data in the above embodiments.

[0053] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0054] The transmission device 406 can be used to receive or send data via a network. Specific examples of the above-mentioned network may include a wired or wireless network provided by a communication provider of the electronic device. In one example, the transmission device includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0055] The input / output device 408 is used to input or output information. In this embodiment, the input information may be current body state information, continuous behavior data, etc., and the output information may be tumor diagnosis results, etc.

[0056] Optionally, in this embodiment, the processor 402 may be configured to perform the following steps through a computer program: Constructing a meridian digital human based on the current physical condition information of the patient to be diagnosed, wherein the meridian digital human includes the qi and blood circulation information and meridian information of the patient to be diagnosed; Collecting continuous behavior data of the patient to be diagnosed, the continuous behavior data includes four types of health index data, diet data, exercise data and sleep data, establishing a meridian mapping table, defining the influence of the continuous behavior data on the qi and blood circulation information and meridian information in the meridian mapping table, and mapping each continuous behavior data of the patient to be diagnosed to the meridian digital human for visual display based on the meridian mapping table; Pathological image information is generated based on the visualization display results of the meridian digital human, and then the pathological text description of the pathological image information is obtained. The corresponding continuous behavioral data, pathological image information and pathological text description are input into the pre-trained tumor diagnosis model to obtain the tumor diagnosis result of the patient to be diagnosed.

[0057] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0058] In general, various embodiments may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the boxes, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0059] Embodiments of the present invention may be implemented by computer software that is executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros may be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. A computer program product may include one or more computer executable components configured to perform an embodiment when the program is run. One or more computer executable components may be at least one software code or a portion thereof. In addition, at this point, it should be noted that, for example, Figure 3 Any block of the logic flow in the program may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on physical media such as memory chips or storage blocks implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs, etc. Physical media are non-transitory media.

[0060] Those skilled in the art should understand that the technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0061] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A tumor diagnosis method based on meridian digital human and continuous behavior data, characterized in that: The following steps are involved: Constructing a meridian digital human based on the current physical condition information of the patient to be diagnosed, wherein the meridian digital human includes the qi and blood circulation information and meridian information of the patient to be diagnosed; Collecting continuous behavior data of the patient to be diagnosed, the continuous behavior data includes four types of health index data, diet data, exercise data and sleep data, establishing a meridian mapping table, defining the influence of the continuous behavior data on the qi and blood circulation information and meridian information in the meridian mapping table, and mapping each continuous behavior data of the patient to be diagnosed to the meridian digital human for visual display based on the meridian mapping table; Pathological image information is generated based on the visualization display results of the meridian digital human, and then the pathological text description of the pathological image information is obtained. The corresponding continuous behavioral data, pathological image information and pathological text description are input into the pre-trained tumor diagnosis model to obtain the tumor diagnosis result of the patient to be diagnosed.

2. A tumor diagnosis method based on meridian digital human and continuous behavior data according to claim 1, characterized in that: The physical examination data and medical imaging data of the patient to be diagnosed are collected as the current body state information, and the current body state information is input into the constructed TCM diagnosis model to obtain the Qi and blood circulation information and meridian information of the patient to be diagnosed, and then the Qi and blood circulation information and meridian information of the patient to be diagnosed are mapped into the standard human body model to obtain the meridian digital person corresponding to the patient to be diagnosed.

3. A tumor diagnosis method based on meridian digital human and continuous behavior data according to claim 1, characterized in that: First, each type of continuous behavior data is discretized, and then all permutations and combinations between the discretization results of different types of continuous behavior data are obtained to obtain a first data set. Each element in the first data set is input into a pre-trained meridian mapping model to obtain the influence of each permutation and combination on the qi and blood circulation information and meridian information to complete the establishment of the meridian mapping table.

4. A tumor diagnosis method based on meridian digital human and continuous behavior data according to claim 3, characterized in that: When discretizing continuous behavior data, different indicators in different types of continuous behavior data are discretized separately.

5. A tumor diagnosis method based on meridian digital human and continuous behavior data according to claim 1, characterized in that: Discretize each type of continuous behavior data of the patient to be diagnosed to obtain a discretization result, obtain a corresponding mapping item in the meridian mapping table based on the discretization result, and then map the mapping item to the corresponding meridian digital human.

6. A tumor diagnosis method based on meridian digital human and continuous behavior data according to claim 1, characterized in that: The pathological image information is feature extracted using a convolutional layer to obtain pathological image features, and then the pathological image features are semantically understood to obtain a pathological text description.

7. A tumor diagnosis method based on meridian digital human and continuous behavior data according to claim 1, characterized in that: The tumor diagnosis model consists of an image feature processing branch, a text feature processing branch, a continuous data feature processing branch, a feature fusion layer and a tumor prediction layer, wherein the image feature processing branch, the text feature processing branch and the continuous data feature processing branch are in a parallel structure, in the image feature processing branch, the pathological image information is firstly feature extracted to obtain a first feature, and then the first feature is mapped to a preset dimension by the image feature mapping layer, in the text feature processing branch, the pathological text description is firstly feature extracted based on a multi-head self-attention mechanism to obtain a second feature, and then the text feature mapping layer maps the second feature to a preset dimension, in the continuous data feature processing branch, the continuous behavior data is feature extracted based on LSTM to obtain a third feature, and then the continuous data feature mapping layer maps the third feature to a preset dimension, the feature fusion layer fuses the first feature, the second feature and the third feature under the preset dimension to obtain a fusion feature, the tumor prediction layer performs tumor prediction based on the fusion feature to obtain a tumor diagnosis result, and the tumor diagnosis result is the probability distribution of the patient to be diagnosed having tumors in different locations.

8. A tumor diagnosis device based on meridian digital human and continuous behavior data, characterized in that: include: A construction module, which constructs a meridian digital human based on the current physical condition information of the patient to be diagnosed, wherein the meridian digital human includes the qi and blood circulation information and meridian information of the patient to be diagnosed; A collection module is used to collect continuous behavior data of the patient to be diagnosed, wherein the continuous behavior data includes four types: health index data, diet data, exercise data, and sleep data, establish a meridian mapping table, define the influence of the continuous behavior data on the qi and blood circulation information and meridian information in the meridian mapping table, and map each continuous behavior data of the patient to be diagnosed to the meridian digital human for visual display based on the meridian mapping table; The diagnosis module generates pathological image information based on the visual display results of the meridian digital human, then obtains the pathological text description of the pathological image information, and inputs the corresponding continuous behavior data, pathological image information and pathological text description into the pre-trained tumor diagnosis model to obtain the tumor diagnosis result of the patient to be diagnosed.

9. An electronic device comprising a memory and a processor, characterized in that: The memory stores a computer program, and the processor is configured to run the computer program to execute a tumor diagnosis method based on meridian digital human and continuous behavior data as described in any one of claims 1-7.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which includes a program code for controlling a process to execute a process, wherein the process includes a tumor diagnosis method based on meridian digital human and continuous behavior data according to any one of claims 1 to 7.

Images