A heterogeneous data feature extraction method and system based on multi-modal information fusion

Abstract

The application relates to the technical field of data processing, and particularly discloses a heterogeneous data feature extraction method and system based on multi-modal information fusion, which comprises the following steps: S1, collecting multi-source pathological data, determining a skin tumor disease set and multiple disease pathological data, and acquiring historical patient data; S2, determining multiple historical staging data and disease development vectors of each skin tumor disease; S3, determining a disease course development vector of each patient of each disease staging label of each skin tumor disease; S4, calculating a label feature set of each disease staging label of each skin tumor disease and multiple staging-course feature sets; and S5, constructing a disease detection model and multiple disease course development prediction models of each skin tumor disease. The core features of diseases and stages can be accurately refined, full-chain information support from disease identification, stage judgment to progress prediction is provided for the clinic, and the accuracy and foresight of skin tumor diagnosis and treatment are improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and system for extracting features from heterogeneous data based on multimodal information fusion. Background Technology

[0002] For a long time, the diagnosis of skin tumors has relied on doctors' experience and single testing methods. In the early stages, it mainly relied on visual observation, the accuracy of which was greatly limited by experience. Later, non-invasive techniques such as dermoscopy were developed, but they can only detect superficial lesions and interpretation depends on professional training, making it easy to miss deep lesions. Although histopathological biopsy is the gold standard, it is an invasive procedure and it is difficult to detect lesions in their early stages.

[0003] With the increasing volume of biomedical data and the emergence of multimodal data, traditional methods lack effective integration tools, resulting in prominent data fragmentation issues. After 2012, breakthroughs in AI technology propelled medical applications. In 2015, the ISIC project began collecting dermoscopic images to train models. While this improved diagnostic accuracy, it was largely limited to a single modality and lacked systematic analysis of disease staging and progression, making it difficult to meet the needs of precision medicine.

[0004] Therefore, this invention proposes a method and system for extracting features from heterogeneous data based on multimodal information fusion. Summary of the Invention

[0005] This invention provides a method and system for heterogeneous data feature extraction based on multimodal information fusion. It identifies a set of skin tumor diseases and multiple disease pathological data by collecting multi-source pathological data, acquires historical patient data based on skin tumors, and determines the historical stage data, disease development vector, and disease progression vector of each patient in the stage data for each skin tumor disease within the skin tumor disease set. It calculates the tag feature set for each stage label of each skin tumor disease and multiple stage-progression feature sets, constructs a disease detection model based on skin tumors, and builds a disease progression prediction model for each stage label of each skin tumor disease. This overcomes the limitations of single data and independent analysis, accurately extracts the core features of each disease and stage, and constructs a complete system for disease detection and disease progression prediction. It provides clinical support across the entire chain of information from disease identification and staging to progression prediction, reduces diagnostic and treatment biases caused by data fragmentation, facilitates personalized treatment and early intervention, and improves the accuracy and foresight of skin tumor diagnosis and treatment.

[0006] This invention provides a method for feature extraction from heterogeneous data based on multimodal information fusion, comprising:

[0007] S1: Collect multi-source pathological data based on skin tumors, determine the skin tumor disease set and multiple disease pathological data, and obtain historical patient data based on skin tumors;

[0008] S2: Based on the skin tumor disease set, all disease pathology data and historical patient data, determine the historical stage data and disease development vector of each disease stage label in the stage label set of each skin tumor disease in the skin tumor disease set;

[0009] S3: Based on the historical staging data and disease development vector of each disease stage label in the staging label set of each skin tumor disease, determine the disease progression vector of each patient in the historical staging data of each disease stage label of each skin tumor disease;

[0010] S4: Based on the historical staging data of each disease stage label in the staging label set of each skin tumor disease and the disease progression vector of all patients in the historical staging data, calculate the label feature set of each disease stage label of each skin tumor disease and multiple staging-disease progression feature sets;

[0011] S5: Based on the tag feature set of all disease stage labels for all skin tumor diseases and the all stage-course feature set, construct a disease detection model for skin tumors, and construct a disease progression prediction model for each disease stage label for each skin tumor disease.

[0012] Preferably, a heterogeneous data feature extraction method based on multimodal information fusion collects multi-source pathological data on skin tumors to determine a skin tumor disease set and pathological data of multiple diseases, including:

[0013] Collect multi-source pathological data based on skin tumors, which includes multiple pathological sub-data.

[0014] Based on all pathological sub-data from the multi-source pathological data, a set of skin tumor diseases and the pathological data of each skin tumor disease in the set of skin tumor diseases are determined. The pathological data includes the disease name, the location of the disease, the set of stage labels, the disease characteristic data of each stage label in the stage label set, and multiple treatment options. The stage label set includes multiple disease stage labels.

[0015] Preferably, a heterogeneous data feature extraction method based on multimodal information fusion is used to obtain historical patient data based on skin tumors, including:

[0016] Obtain historical patient subdata for each skin tumor disease in the skin tumor disease set. The historical patient subdata includes clinical data of multiple patients, including name, gender, age, duration of illness, duration of multiple visits, diagnostic labels, and examination and imaging data.

[0017] Based on historical patient subdata of all skin tumor diseases in the skin tumor disease set, identify historical patient data based on skin tumors.

[0018] Preferably, a heterogeneous data feature extraction method based on multimodal information fusion, based on a skin tumor disease set, all disease pathology data, and historical patient data, determines the historical stage data and disease development vector of each disease stage label in the stage label set for each skin tumor disease in the skin tumor disease set, including:

[0019] Based on the set of stage labels in the pathological data of each skin tumor disease in the skin tumor disease set, and the diagnosis label with the latest consultation time in the clinical data of all patients in the historical patient sub-data of each skin tumor disease in the historical patient data, the historical patient sub-data of each skin tumor disease is divided to determine the historical stage data of each disease stage label in the set of stage labels of each skin tumor disease in the skin tumor disease set. The historical stage data includes the clinical data of multiple patients.

[0020] Based on the set of stage labels in the pathological data of each skin tumor disease in the skin tumor disease set, the disease development vector of each skin tumor disease in the skin tumor disease set is determined based on each disease stage label in the set of stage labels.

[0021] Preferably, a heterogeneous data feature extraction method based on multimodal information fusion determines the disease progression vector of each patient in the historical stage data of each disease stage label for each skin tumor disease, based on the historical stage data and disease progression vector of each disease stage label in the stage label set of each skin tumor disease, including:

[0022] From the historical staging data of each disease stage label for each skin tumor disease in the skin tumor disease set, the patient's clinical data includes the time of all visits and the diagnostic label. The diagnostic label is selected as the earliest time of visit for each disease stage label. The current stage diagnosis time is selected from the historical staging data of each patient for each disease stage label for each skin tumor disease.

[0023] Based on the historical staging data of each disease stage label for each skin tumor disease in the skin tumor disease set, the consultation time and diagnostic label of each patient's clinical data for all visits, as well as the set of stage labels in the pathological data of each skin tumor disease, the staging diagnosis time of each patient based on each disease stage label before the disease stage label in the set of stage labels is determined in the historical staging data of each disease stage label for each skin tumor disease.

[0024] Based on the historical staging data of each disease stage label for each skin tumor disease in the skin tumor disease set, the diagnosis time of the current stage of each patient, the diagnosis time of all disease stage labels before the disease stage label in the stage label set, and the disease development vector of each disease stage label for each skin tumor disease, the disease development vector of each patient in the historical staging data of each disease stage label for each skin tumor disease in the skin tumor disease set is determined.

[0025] Preferably, a heterogeneous data feature extraction method based on multimodal information fusion calculates a tag feature set for each disease stage label and multiple stage-disease progression feature sets for each skin tumor disease based on historical stage data for each disease stage label in the stage label set for each skin tumor disease and the disease progression vector of all patients in the historical stage data, including:

[0026] Based on the disease progression vector of all patients in the historical stage data of each disease stage label of each skin tumor disease in the skin tumor disease set, cluster analysis is performed on all patients in the historical stage data of each disease stage label of each skin tumor disease in the skin tumor disease set. Based on the cluster analysis results, multiple historical stage-disease progression data of each disease stage label of each skin tumor disease in the skin tumor disease set are determined. The historical stage-disease progression data includes the clinical data of multiple patients and the development label of each development feature in the disease progression vector.

[0027] For each disease stage label of each skin tumor disease in the skin tumor disease set, feature extraction is performed on the examination image data of each patient's clinical data from each visit in the historical stage-disease course data of each patient for each disease stage label of each skin tumor disease in the skin tumor disease set. This determines the image feature vector of each visit in the clinical data of each patient for each visit in the historical stage-disease course data of each patient for each disease stage label of each skin tumor disease in the skin tumor disease set.

[0028] Based on the disease development vector of each disease stage label for each skin tumor disease in the skin tumor disease set, the image feature vector of all visits in the clinical data of all patients in each historical stage-disease course data, and the disease development vector of all patients in each historical stage-disease course data, calculate the associated sub-feature set of the disease stage label of each disease stage label for each historical stage-disease course data of each skin tumor disease in the skin tumor disease set, and the associated sub-feature set of the disease stage labels before the disease stage label in the stage label set;

[0029] Based on the set of stage labels in the pathological data of each skin tumor disease in the skin tumor disease set, the set of associated sub-features of the disease stage labels in all historical stage-course data of all disease stage labels, and the set of associated sub-features of disease stage labels preceding the disease stage labels in the set of stage labels, the tag feature set of each disease stage label of each skin tumor disease in the skin tumor disease set, and the stage-course feature set of each disease stage label preceding the disease stage label in each historical stage-course data are calculated.

[0030] Preferably, a heterogeneous data feature extraction method based on multimodal information fusion constructs a disease detection model for skin tumors based on the tag feature set of all disease stage labels for all skin tumor diseases and the all stage-course feature set, and constructs a course development prediction model for each disease stage label of each skin tumor disease, including:

[0031] Based on the pathological data of each skin tumor disease in the skin tumor disease set and the tag feature set of all disease stage labels, a stage detection model for each skin tumor disease in the skin tumor disease set is constructed.

[0032] Based on the staging detection model of all skin tumor diseases in the skin tumor disease set, a disease detection model for the skin tumor disease set is constructed.

[0033] Based on the set of stage-course features of all historical stage-course data of each disease stage label for each skin tumor disease in the skin tumor disease set, a disease progression prediction model is constructed using all historical stage-course data of all disease stage labels preceding the disease stage label for each skin tumor disease in the skin tumor disease set.

[0034] This invention provides a heterogeneous data feature extraction system based on multimodal information fusion, used to execute any one of the heterogeneous data feature extraction methods based on multimodal information fusion in Examples 1 to 7, comprising:

[0035] Data Acquisition Module: Collects multi-source pathological data based on skin tumors, identifies the skin tumor disease set and pathological data of multiple diseases, and obtains historical patient data based on skin tumors;

[0036] Determining the module: Based on the skin tumor disease set, all disease pathology data, and historical patient data, determine the historical stage data and disease development vector of each disease stage label in the stage label set for each skin tumor disease in the skin tumor disease set;

[0037] Disease progression module: Based on the historical staging data and disease progression vector of each disease stage tag in the staging tag set of each skin tumor disease, determine the disease progression vector of each patient in the historical staging data of each disease stage tag of each skin tumor disease;

[0038] Calculation module: Based on the historical staging data of each disease stage label in the staging label set of each skin tumor disease and the disease progression vector of all patients in the historical staging data, calculate the label feature set of each disease stage label of each skin tumor disease and multiple stage-progression feature sets;

[0039] Building Module: Based on the tag feature set of all disease stage labels for all skin tumor diseases and the all stage-course feature set, a disease detection model for skin tumors is constructed, and a disease progression prediction model for each disease stage label for each skin tumor disease is constructed.

[0040] The beneficial effects of this invention compared to existing technologies are as follows: By collecting multi-source pathological data to determine a set of skin tumor diseases and multiple disease pathological data, historical patient data based on skin tumors is obtained. The historical stage data, disease development vector, and disease progression vector of each patient in the historical stage data are determined for each stage label of each skin tumor disease within the skin tumor disease set. The label feature set and multiple stage-progression feature sets for each disease stage label of each skin tumor disease are calculated. A disease detection model based on skin tumors is constructed, along with a disease progression prediction model for each stage label of each skin tumor disease. This overcomes the limitations of single data and independent analysis, accurately extracts the core features of each disease and stage, and constructs a complete system for disease detection and progression prediction. It provides clinical support across the entire chain of information from disease identification and staging to progression prediction, reducing diagnostic and treatment biases caused by data fragmentation, facilitating personalized treatment and early intervention, and improving the accuracy and foresight of skin tumor diagnosis and treatment. Meanwhile, the multimodal feature set and model framework constructed in this invention can serve as the basis for the subsequent introduction of reinforcement learning technology. That is, by using clinical diagnosis and treatment feedback, such as treatment effect and disease progression consistency, as reward signals for reinforcement learning, the feature weights and disease course prediction logic of the model can be dynamically optimized, further improving the model's adaptability to the dynamic changes in individual patients' conditions and extending the model's lifespan and accuracy stability in clinical applications.

[0041] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in this application.

[0042] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0043] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0044] Figure 1 This is a flowchart of a heterogeneous data feature extraction method based on multimodal information fusion in an embodiment of the present invention;

[0045] Figure 2 This is a schematic diagram of a heterogeneous data feature extraction system based on multimodal information fusion in an embodiment of the present invention. Detailed Implementation

[0046] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0047] Example 1:

[0048] This invention provides a method for feature extraction from heterogeneous data based on multimodal information fusion, referencing... Figure 1 ,include:

[0049] S1: Collect multi-source pathological data based on skin tumors, determine the skin tumor disease set and multiple disease pathological data, and obtain historical patient data based on skin tumors;

[0050] S2: Based on the skin tumor disease set, all disease pathology data and historical patient data, determine the historical stage data and disease development vector of each disease stage label in the stage label set of each skin tumor disease in the skin tumor disease set;

[0051] S3: Based on the historical staging data and disease development vector of each disease stage label in the staging label set of each skin tumor disease, determine the disease progression vector of each patient in the historical staging data of each disease stage label of each skin tumor disease;

[0052] S4: Based on the historical staging data of each disease stage label in the staging label set of each skin tumor disease and the disease progression vector of all patients in the historical staging data, calculate the label feature set of each disease stage label of each skin tumor disease and multiple staging-disease progression feature sets;

[0053] S5: Based on the tag feature set of all disease stage labels for all skin tumor diseases and the all stage-course feature set, construct a disease detection model for skin tumors, and construct a disease progression prediction model for each disease stage label for each skin tumor disease.

[0054] In this embodiment, multi-source pathological data based on skin tumors are collected. These data contain multiple pathological sub-data of different dimensions. By systematically sorting through these data, it is determined which skin tumor diseases exist, thereby forming a set of skin tumor diseases. At the same time, the pathological data corresponding to each disease is obtained, and historical patient data related to skin tumors are collected.

[0055] In this embodiment, after obtaining the basic data, the pathological data of all diseases in the skin tumor disease set and the historical patient data are combined to further clarify the historical stage data and disease development vector corresponding to each disease stage label in the stage label set of each skin tumor disease. The historical stage data is the relevant record of past patients under that stage.

[0056] In this embodiment, based on the historical staging data of each stage label of each skin tumor disease and the disease progression vector of all patients, a label feature set and multiple stage-progression feature sets are calculated for each disease and each stage label. The label feature set is the core identifier feature of the stage, and the stage-progression feature set is the core identifier feature of each stage before the stage.

[0057] In this embodiment, a disease detection model for skin tumors is constructed using the tag feature set of all stage tags for all skin tumor diseases and the all stage-disease course feature set. This model enables the identification of skin tumor types. At the same time, a disease progression prediction model is constructed for each stage tag of each skin tumor disease to predict the subsequent disease progression direction and status of the patient.

[0058] The beneficial effects of the above technologies are as follows: By collecting multi-source pathological data to determine a set of skin tumor diseases and multiple disease pathological data, historical patient data based on skin tumors is obtained. This allows for the determination of historical stage data, disease development vectors, and disease progression vectors for each patient within the stage label set of each skin tumor disease in the skin tumor disease set. Furthermore, it enables the calculation of label feature sets for each disease stage label of each skin tumor disease, as well as multiple stage-progression feature sets. This leads to the construction of a disease detection model based on skin tumors and a disease progression prediction model for each disease stage label of each skin tumor disease. This overcomes the limitations of single data and independent analysis, accurately extracting the core features of each disease and stage, and constructing a complete system for disease detection and progression prediction. It provides clinical support across the entire chain from disease identification and staging to progression prediction, reducing diagnostic and treatment biases caused by data fragmentation, facilitating personalized treatment and early intervention, and improving the accuracy and foresight of skin tumor diagnosis and treatment.

[0059] Example 2:

[0060] Based on Example 1, a heterogeneous data feature extraction method based on multimodal information fusion is proposed, which collects multi-source pathological data based on skin tumors, determines a skin tumor disease set and multiple disease pathological data, including:

[0061] Collect multi-source pathological data based on skin tumors, which includes multiple pathological sub-data.

[0062] Based on all pathological sub-data from the multi-source pathological data, a set of skin tumor diseases and the pathological data of each skin tumor disease in the set of skin tumor diseases are determined. The pathological data includes the disease name, the location of the disease, the set of stage labels, the disease characteristic data of each stage label in the stage label set, and multiple treatment options. The stage label set includes multiple disease stage labels.

[0063] In this embodiment, multi-source pathological data based on skin tumors are collected. This multi-source pathological data is not a single form or dimension of pathological information, but rather a combination of multiple pathological sub-data of different types and sources. These pathological sub-data cover a wide range, including professional medical books such as *Skin Tumors: A Global Perspective* and *Atlas of Skin Appendage Tumors*, medical research literature such as "36% of Dermatofibrosarcomas Protuberances Are Initially Misdiagnosed; Genetic Testing Aids Precision Diagnosis and Treatment" and "Myoepithelial Tumors Occurring in Acral Skin Have Unique and Reproducible Morphological and Immunophenotypic Features," as well as professional medical websites and databases. The content covers various aspects of skin tumors, including epidemiological data, pathogenesis research findings, clinical characteristics descriptions, diagnostic methods, treatment approaches, and prognoses. Each pathological subdata set may include macroscopic pathological data obtained through visual observation, such as the overall size, shape, color, boundary state, and surface condition of the tumor; it may also include microscopic pathological data obtained through microscopic observation, such as the morphological characteristics of tumor cells, the size and staining of cell nuclei, cell arrangement, and the boundary and infiltration of tumor tissue with surrounding normal tissue; and it may also include pathological data obtained through special detection methods, such as the expression results of various markers in immunohistochemical detection, the mutation, fusion, or amplification of specific genes in molecular pathology detection, and the distribution and properties of specific components within tumor tissue as revealed by special staining detection. By collecting these diverse pathological subdata sets, the pathological characteristics of skin tumors can be comprehensively captured from different levels and perspectives, providing rich and comprehensive raw data support for subsequent in-depth analysis and research on skin tumor diseases.

[0064] In this embodiment, all collected pathological sub-data are systematically organized, analyzed, and summarized. First, through comprehensive comparison and classification of various pathological sub-data, all existing skin tumor disease types are identified, forming a complete skin tumor disease set, clarifying which different types of skin tumors are included. Then, for each skin tumor disease in the disease set, various related pathological information is further extracted and integrated to construct the disease pathology data for that disease. These pathological data for this disease comprise several key components. The disease name is a unique identifier for each skin tumor, distinguishing it from other diseases. The lesion site clarifies the specific areas of the skin where the disease typically occurs; different skin tumors often have a preferred lesion site. The staging tag set consists of multiple different staging tags based on the severity and progression of the disease. For example, based on tumor size, depth of invasion, and the presence or absence of lymph node or distant metastasis, the disease is divided into stages I, II, III, and IV, each representing a specific stage of disease development. The corresponding disease characteristic data for each stage provides a detailed description of the pathological manifestations at that stage, such as the specific morphological changes of tumor cells, the extent of tumor invasion, and differences in the expression of relevant biomarkers. This characteristic data is crucial for differentiating between stages. Multiple treatment options are summarized based on the pathological and clinical characteristics of different stages of the disease, summarizing treatment methods applicable to different situations. These may include different types of treatments such as surgery, radiotherapy, drug therapy, targeted therapy, and immunotherapy, providing a reference for clinical practice in developing treatment strategies for different stages of the disease. In this way, a comprehensive and detailed pathological information file can be established for each type of skin tumor, clearly presenting the core attributes of the disease and relevant diagnostic and treatment information.

[0065] The beneficial effects of the above technologies are: collecting multi-source pathological data based on skin tumors, identifying the skin tumor disease set and multiple disease pathological data, reducing disease cognitive bias caused by data fragmentation, and providing comprehensive and structured information support for determining the historical staging data and disease development vector of each disease staging label in the skin tumor disease set and the staging label set of each disease in the skin tumor disease set.

[0066] Example 3:

[0067] Based on Example 2, a heterogeneous data feature extraction method based on multimodal information fusion is proposed to obtain historical patient data based on skin tumors, including:

[0068] Obtain historical patient subdata for each skin tumor disease in the skin tumor disease set. The historical patient subdata includes clinical data of multiple patients, including name, gender, age, duration of illness, duration of multiple visits, diagnostic labels, and examination and imaging data.

[0069] Based on historical patient subdata of all skin tumor diseases in the skin tumor disease set, identify historical patient data based on skin tumors.

[0070] In this embodiment, historical patient sub-data is obtained for each skin tumor disease in the skin tumor disease set. This process requires collecting information on past patients related to each specific skin tumor disease in the previously determined skin tumor disease set. This information collectively constitutes the historical patient sub-data for that disease. Each historical patient sub-data contains clinical data for multiple patients, which covers a wide range of information. Name serves as the patient's identifier, distinguishing different individuals; gender and age are basic demographic characteristics, as patients of different genders and age groups may differ in the type, probability, and progression of skin tumors. This information is crucial for subsequent analysis of the correlation between the disease and population characteristics; the onset time records the time when the patient first discovered skin tumor-related symptoms or was initially diagnosed with the skin tumor disease, which is key information for tracing the starting point of the patient's disease progression; the multiple consultation times detail the specific time points when the patient visited medical institutions multiple times for treatment of the skin tumor disease during the course of the disease. The nodes clearly present the patient's diagnosis and treatment history, allowing for an understanding of the pace of disease progression and response to treatment. Diagnostic labels represent the doctor's diagnostic conclusions regarding skin tumors at each visit, based on the patient's symptoms and examination results. These labels may include the specific disease name, stage, and other information, reflecting the progression or improvement of the patient's condition. Imaging data comprises the results of various imaging examinations performed during the diagnosis and treatment process, such as dermoscopy images, skin CT images, and ultrasound images. This imaging data visually presents the tumor's morphology, size, location, boundary characteristics, and relationship with surrounding tissues, serving as crucial evidence for diagnosis and assessment. By collecting this comprehensive patient clinical data, historical patient sub-databases containing multiple patients' diagnosis and treatment information are constructed for each type of skin tumor, comprehensively and meticulously preserving the past treatment trajectory and disease characteristics of patients with that disease.

[0071] In this embodiment, historical patient data for skin tumors is determined based on historical patient sub-data for all skin tumor diseases in the skin tumor disease set. This step requires summarizing and integrating the historical patient sub-data previously collected for each skin tumor disease. Since the skin tumor disease set includes various different skin tumor diseases, each disease has its corresponding historical patient sub-data, which records the clinical information of patients with different diseases. By systematically organizing and merging all these historical patient sub-data scattered across various disease dimensions, the integrity and consistency of the data are ensured, ultimately forming a unified historical patient data set covering the entire range of skin tumor diseases.

[0072] The beneficial effects of the above technologies are: obtaining historical patient data based on skin tumors can provide comprehensive and structured information support for determining the historical stage data and disease development vector of each disease stage label in the stage label set of each skin tumor disease set.

[0073] Example 4:

[0074] Building upon Example 3, a heterogeneous data feature extraction method based on multimodal information fusion, using a skin tumor disease set, all disease pathology data, and historical patient data, determines the historical stage data and disease development vector of each disease stage label in the stage label set for each skin tumor disease in the skin tumor disease set, including:

[0075] Based on the set of stage labels in the pathological data of each skin tumor disease in the skin tumor disease set, and the diagnosis label with the latest consultation time in the clinical data of all patients in the historical patient sub-data of each skin tumor disease in the historical patient data, the historical patient sub-data of each skin tumor disease is divided to determine the historical stage data of each disease stage label in the set of stage labels of each skin tumor disease in the skin tumor disease set. The historical stage data includes the clinical data of multiple patients.

[0076] Based on the set of stage labels in the pathological data of each skin tumor disease in the skin tumor disease set, the disease development vector of each skin tumor disease in the skin tumor disease set is determined based on each disease stage label in the set of stage labels.

[0077] In this embodiment, the operation revolves around each type of skin tumor. The pathological data for each disease includes a set of stage labels. This set contains stage labels corresponding to different stages of the disease's development; for example, some malignant skin tumors may have different stage labels such as Stage I, Stage II, Stage III, and Stage IV. In the historical patient data, within the historical patient sub-data for each disease, each patient's clinical data records information from multiple visits. The diagnostic label with the latest visit time represents the final stage of the disease in the current historical record, accurately defining the final stage of the patient's disease. During the classification process, the set of disease stage labels is used as the classification basis. All patients in the historical patient sub-data for that disease are assigned to the corresponding stage label according to their latest visit time diagnostic label. For example, if a patient's latest visit time diagnostic label is Stage II, then all of that patient's clinical data is classified into the Stage II category for that disease. By dividing the data in this way, a batch of patients' clinical data can be identified for each stage label of each disease. These data together constitute the historical stage data of that stage label, so that each stage has specific patient cases and clinical information as support.

[0078] In this embodiment, for each skin tumor disease, a disease progression vector is constructed based on the set of stage labels in its pathological data. The set of stage labels itself implies the progression order of the disease from early to late stages. For example, for basal cell carcinoma, the stage label set is: Stage I, Stage II, Stage III, and Stage IV. The disease progression vector for Stage I basal cell carcinoma is {duration from onset to Stage I, duration from Stage I to present}, and the disease progression vector for Stage II basal cell carcinoma is {duration from onset to Stage I, duration from Stage I to Stage II, duration from Stage II to present}, and so on, reflecting the gradual progression of the disease.

[0079] The beneficial effects of the above technologies are as follows: Based on the skin tumor disease set, all disease pathology data and historical patient data, the historical stage data and disease development vector of each disease stage label in the stage label set of each skin tumor disease in the skin tumor disease set can be determined, which can realize the staging and regularization of patient data and provide a structured description of the disease progression trajectory.

[0080] Example 5:

[0081] Building upon Example 4, a heterogeneous data feature extraction method based on multimodal information fusion determines the disease progression vector for each patient within the historical stage data of each disease stage label in the stage label set for each skin tumor disease, based on the historical stage data and disease progression vector of each disease stage label in the stage label set for each skin tumor disease. This includes:

[0082] From the historical staging data of each disease stage label for each skin tumor disease in the skin tumor disease set, the patient's clinical data includes the time of all visits and the diagnostic label. The diagnostic label is selected as the earliest time of visit for each disease stage label. The current stage diagnosis time is selected from the historical staging data of each patient for each disease stage label for each skin tumor disease.

[0083] Based on the historical staging data of each disease stage label for each skin tumor disease in the skin tumor disease set, the consultation time and diagnostic label of each patient's clinical data for all visits, as well as the set of stage labels in the pathological data of each skin tumor disease, the staging diagnosis time of each patient based on each disease stage label before the disease stage label in the set of stage labels is determined in the historical staging data of each disease stage label for each skin tumor disease.

[0084] Based on the historical staging data of each disease stage label for each skin tumor disease in the skin tumor disease set, the diagnosis time of the current stage of each patient, the diagnosis time of all disease stage labels before the disease stage label in the stage label set, and the disease development vector of each disease stage label for each skin tumor disease, the disease development vector of each patient in the historical staging data of each disease stage label for each skin tumor disease in the skin tumor disease set is determined.

[0085] In this embodiment, historical staging data corresponding to each disease stage label for each type of skin tumor is analyzed. This historical staging data includes clinical data from multiple patients, and each patient's clinical data records multiple consultation times and corresponding diagnostic labels. The core objective here is to determine a precise diagnosis time for the patient's current stage. Specifically, from all the patient's consultation records, all consultation records with a diagnostic label that exactly matches the current disease stage label are selected. Then, among the consultation times corresponding to these records, the earliest time point is selected and defined as the stage diagnosis time for the patient's current stage. For example, if a patient has basal cell carcinoma and their historical consultation records show three diagnoses of stage III, with consultation times of March 16, 2022, May 7, 2022, and July 22, 2022, then March 16, 2022, is selected as the stage diagnosis time for the patient's current stage III. In this way, a precise starting time node is determined for each current stage for each patient, clarifying the earliest time the patient entered that stage.

[0086] In this embodiment, the order of stage labels in the staging label set for each type of skin tumor disease is first determined. For example, if the staging label set is Stage I, Stage II, Stage III, and Stage IV, then for the Stage II disease stage label, the preceding stage label is Stage I; for Stage III, the preceding stages are Stage I and Stage II. After determining the order, for each patient and the current disease stage label, all the patient's medical records are reviewed. From the records, the medical records of each stage label preceding the current stage label are selected. Similarly, following the previous method, the earliest medical record time corresponding to each previous stage label is selected as the stage diagnosis time for the patient's corresponding previous stage label. For example, consider a skin tumor called basal cell carcinoma, with staging labels of Stage I, II, III, and IV. We are currently processing historical staging data for a patient in Stage III. We need to find the earliest consultation date in the patient's medical records for Stage I (October 19, 2020) and Stage II (December 7, 2021). These two dates represent the patient's stage diagnosis dates based on the Stage I and Stage II staging labels prior to Stage III. From the patient's clinical data, we extract the onset date as December 6, 2019, and the current date as September 22, 2022. Based on the onset date of December 6, 2019, and the earliest consultation date for Stage I... The patient's illness duration from onset to stage I was determined to be 318 days on October 19, 2020. Based on the earliest diagnosis date of stage I (October 19, 2020) to the earliest diagnosis date of stage II (December 7, 2021), the duration from stage I to stage II was determined to be 414 days. Based on the earliest diagnosis date of stage II (December 7, 2021) to the current stage diagnosis date (March 16, 2022), the duration from stage II to stage III was determined to be 99 days. Based on the current duration and the current stage diagnosis date (March 16, 2022) to the current time (September 22, 2022), the duration from stage III to the current time was determined to be 190 days. The patient's disease progression vector is represented as {318 days, 414 days, 99 days, 190 days}.

[0087] The beneficial effects of the above technology are as follows: Based on the historical staging data and disease development vector of each disease staging label in the staging label set of each skin tumor disease, the disease development vector of each patient in the historical staging data of each disease staging label of each skin tumor disease can be determined, which can realize the fine-grained tracking of the patient's disease trajectory.

[0088] Example 6:

[0089] Building upon Example 5, a heterogeneous data feature extraction method based on multimodal information fusion calculates a tag feature set for each disease stage label and multiple stage-progression feature sets for each skin tumor disease, based on historical stage data for each disease stage label in the stage label set for each skin tumor disease and the disease progression vector of all patients in the historical stage data. These features include:

[0090] Based on the disease progression vector of all patients in the historical stage data of each disease stage label of each skin tumor disease in the skin tumor disease set, cluster analysis is performed on all patients in the historical stage data of each disease stage label of each skin tumor disease in the skin tumor disease set. Based on the cluster analysis results, multiple historical stage-disease progression data of each disease stage label of each skin tumor disease in the skin tumor disease set are determined. The historical stage-disease progression data includes the clinical data of multiple patients and the development label of each development feature in the disease progression vector.

[0091] For each disease stage label of each skin tumor disease in the skin tumor disease set, feature extraction is performed on the examination image data of each patient's clinical data from each visit in the historical stage-disease course data of each patient for each disease stage label of each skin tumor disease in the skin tumor disease set. This determines the image feature vector of each visit in the clinical data of each patient for each visit in the historical stage-disease course data of each patient for each disease stage label of each skin tumor disease in the skin tumor disease set.

[0092] Based on the disease development vector of each disease stage label for each skin tumor disease in the skin tumor disease set, the image feature vector of all visits in the clinical data of all patients in each historical stage-disease course data, and the disease development vector of all patients in each historical stage-disease course data, calculate the associated sub-feature set of the disease stage label of each disease stage label for each historical stage-disease course data of each skin tumor disease in the skin tumor disease set, and the associated sub-feature set of the disease stage labels before the disease stage label in the stage label set;

[0093] Based on the set of stage labels in the pathological data of each skin tumor disease in the skin tumor disease set, the set of associated sub-features of the disease stage labels in all historical stage-course data of all disease stage labels, and the set of associated sub-features of disease stage labels preceding the disease stage labels in the set of stage labels, the tag feature set of each disease stage label of each skin tumor disease in the skin tumor disease set, and the stage-course feature set of each disease stage label preceding the disease stage label in each historical stage-course data are calculated.

[0094] In this embodiment, based on the disease progression vectors of all patients in the historical staging data of each disease stage label for each skin tumor disease in the skin tumor disease set, cluster analysis is performed on these patients, and then multiple historical stage-disease progression data are determined for each disease stage label based on the clustering results. The core here is to use the disease progression vector to divide patient groups. Cluster analysis groups patients with similar disease progression vectors into the same group, so that patients within the same group have similar disease progression patterns, and patients in different groups have significant differences in disease progression. The patient data corresponding to each cluster result, combined with the development label of each development feature in the disease progression vector, constitutes a historical stage-disease progression data. The development label is a further identification of each feature in the disease progression vector, such as rapid progression, slow progression, and normal progression. In this way, multiple historical stage-disease progression data will be formed under each disease stage label, and each data contains clinical data of patients with similar disease progression characteristics and corresponding development labels.

[0095] In this embodiment, feature extraction is performed on the examination image data of each patient at each visit from multiple historical staging-course data for each disease stage label to determine the image feature vector for each visit. Examination image data is crucial visual information in the patient's diagnosis and treatment process. For example, dermoscopy images can reveal details such as the color, boundary, and structure of tumors, while skin CT images can show the depth of tumor invasion and cell distribution. Feature extraction involves extracting clinically significant and discriminative information from this image data, such as the color distribution characteristics of the tumor area, the degree of irregularity of the boundary, and the texture features of the internal structure. Organizing these extracted features into vectors forms image feature vectors, with each image feature vector corresponding to the core information of the patient's image data at a particular visit.

[0096] In this embodiment, based on the disease development vector of each disease stage label for each skin tumor disease in the skin tumor disease set, the image feature vector of all visits in the clinical data of all patients in each historical stage-disease course data, and the disease course development vector of all patients in each historical stage-disease course data, the associated sub-feature set of the disease stage label for each disease stage label in each historical stage-disease course data of each skin tumor disease in the skin tumor disease set, and the associated sub-feature set of the disease stage labels preceding the disease stage label in the stage label set, the calculation formula can be expressed as:

[0097] ;

[0098] ;

[0099] ;

[0100] ;

[0101] ;

[0102] ;

[0103] ;

[0104] ;

[0105] in, The c-th feature parameter in the image feature vector representing the rate of change of the current stage visit time of patient a in the k-th historical stage-disease progress data corresponding to the stage diagnosis time of the first stage label preceding the current stage label of the i-th skin tumor disease is used. The c-th feature parameter in the image feature vector representing the number of visits corresponding to the current stage of the a-th patient in the k-th historical stage-disease progression data of the j-th disease stage label for the i-th skin tumor disease is given. The c-th feature parameter in the image feature vector of patient a, representing the number of visits to a patient based on the stage diagnosis time corresponding to the m-th stage label preceding the stage label in the stage label set for the i-th skin tumor, within the k-th historical stage-disease progress data of the j-th disease stage label. The c-th feature parameter in the image feature vector of patient a, representing the number of visits to a patient based on the stage diagnosis time corresponding to the (m-1)th stage label preceding the stage label in the stage label set for the i-th skin tumor, within the k-th historical stage-course data of the j-th disease stage label. This represents the eigenvalue of the j-th feature parameter in the disease progression vector of the a-th patient within the k-th historical stage-disease progression data of the j-th disease stage label for the i-th skin tumor. This represents the feature value of the jm-th feature parameter in the disease progression vector of the a-th patient within the k-th historical stage-disease progression data of the j-th disease stage label for the i-th skin tumor. Let c represent the feature change rate of the c-th feature parameter in the image feature vector of the k-th historical stage-disease progress data for the i-th skin tumor disease, corresponding to the current stage visit time of the 1st patient and the stage diagnosis time of the 1st disease stage label prior to the i-th disease stage label. This represents the first ascending sequence of the feature change rate of the c-th feature parameter in the image feature vector of all patients in the k-th historical stage-disease progress data corresponding to the current stage visit time and the stage diagnosis time of the 1st stage label preceding the current stage label, for the i-th skin tumor disease. Let c represent the rate of change of the c-th feature parameter in the image feature vector representing the number of visits corresponding to the stage diagnosis time of the m-th and (m-1)-th disease stage labels preceding the stage label of the 1st patient and the (m-1)-th patient in the k-th historical stage-course data of the j-th disease stage label of the i-th skin tumor disease. This represents the second ascending sequence of the feature change rate of the c-th feature parameter in the image feature vector of all patients in the k-th historical stage-course data corresponding to the stage diagnosis time of the m-th and (m-1)-th disease stage labels preceding the i-th disease stage label. Represents the rate of change of characteristics In ascending sequence The position in the middle, The first change consistency value of the c-th feature parameter in the image feature vector of all patients in the k-th historical stage-disease progress data corresponding to the stage visit time of the current stage and the stage diagnosis time of the 1st stage label before the aforementioned disease stage label in the j-th disease stage label of the i-th skin tumor disease. The first average rate of change of the c-th feature parameter in the image feature vector representing the number of visits corresponding to the current stage of all patients in the k-th historical stage-disease progress data of the j-th disease stage label for the i-th skin tumor disease, the current stage visit time, and the stage diagnosis time of the 1st disease stage label preceding the aforementioned disease stage label. Represents the rate of change of characteristics In ascending sequence The position in the middle, The second change consistency value of the c-th feature parameter in the image feature vector representing the number of visits corresponding to the stage diagnosis time of all patients in the k-th historical stage-disease progress data of the j-th disease stage label of the i-th skin tumor disease. The second average rate of change of the c-th feature parameter in the image feature vector representing the number of stage-course data corresponding to the stage diagnosis time of all patients at the m-th and (m-1)-th disease stage labels preceding the i-th disease stage label in the k-th historical stage-course data of the i-th skin tumor disease is given. The set of associated sub-features for the disease stage label of all patients with the j-th disease stage label of the i-th skin tumor, representing the k-th historical stage-course data. This represents the set of associated sub-features for all patients with the i-th skin tumor disease, specifically the m-th disease stage label preceding the j-th disease stage label in the k-th historical stage-course data. This represents the c-th feature parameter in the image feature vector of the patient's visit in the k-th historical stage-disease course data for the j-th disease stage label of the i-th skin tumor disease. TH represents the consistency threshold. This represents the number of patients in the k-th historical stage-disease progression data for the j-th disease stage label of the i-th skin tumor disease.

[0106] In this embodiment, based on the set of stage labels in the pathological data of each skin tumor disease in the skin tumor disease set, the set of associated sub-features of the disease stage labels in all historical stage-course data of all disease stage labels, and the set of associated sub-features of disease stage labels preceding the disease stage labels in the set of stage labels, the associated feature set of each disease stage label of each skin tumor disease in the skin tumor disease set, and the set of stage-course feature sets of disease stage labels preceding the disease stage labels in each historical stage-course data are calculated. The calculation formula can be expressed as:

[0107] ;

[0108] ;

[0109] ;

[0110] in, This represents the set of label features for the j-th stage of the i-th skin tumor. This represents the set of stage-course features for all patients with the i-th skin tumor, preceding the m-th disease stage label in the historical stage-course data of the f-th stage, which is the p-th disease stage label. This represents the position of the j-th stage label of the i-th skin tumor in the stage label set. This represents the position of the p-th stage label of the i-th skin tumor in the stage label set. This represents the j-th stage label for the i-th skin tumor. This refers to the m-th disease stage label preceding the p-th disease stage label in the historical stage-course data for all patients with the i-th skin tumor. The development label of the pm-th feature parameter in the disease progression vector represents the f-th historical stage-progression data of the p-th disease stage label for the i-th skin tumor. This represents the development label of the j-th feature parameter in the disease progression vector within the k-th historical stage-disease progression data for the j-th disease stage label of the i-th skin tumor. This represents the set of associated sub-features for all patients with the i-th skin tumor disease, preceding the m-th disease stage label in the f-th historical stage-course data. This represents the number of historical stage-course data for the p-th stage label of the i-th skin tumor disease. The k-th historical stage-course data of the j-th disease stage label for the i-th skin tumor disease is represented by a first exponential function of the historical stage-course data of the m-th disease stage label preceding the said disease stage label for all patients based on the f-th historical stage-course data of the p-th disease stage label.

[0111] The beneficial effects of the above technology are as follows: Based on the historical staging data of each disease stage label in the staging label set of each skin tumor disease and the disease progression vector of all patients in the historical staging data, the label feature set of each disease stage label of each skin tumor disease and multiple stage-progression feature sets can be calculated. This can provide refined data support for disease staging diagnosis and progression pattern mining, reduce feature bias caused by data contamination, and help improve the accuracy of subsequent models in identifying and analyzing disease staging.

[0112] Example 7:

[0113] Building upon Example 6, a heterogeneous data feature extraction method based on multimodal information fusion constructs a disease detection model for skin tumors based on the tag feature set of all disease stage labels for all skin tumor diseases and the all stage-course feature set. It also constructs a disease progression prediction model for each disease stage label of each skin tumor disease, including:

[0114] Based on the pathological data of each skin tumor disease in the skin tumor disease set and the tag feature set of all disease stage labels, a stage detection model for each skin tumor disease in the skin tumor disease set is constructed.

[0115] Based on the staging detection model of all skin tumor diseases in the skin tumor disease set, a disease detection model for the skin tumor disease set is constructed.

[0116] Based on the set of stage-course features of all historical stage-course data of each disease stage label for each skin tumor disease in the skin tumor disease set, a disease progression prediction model is constructed using all historical stage-course data of all disease stage labels preceding the disease stage label for each skin tumor disease in the skin tumor disease set.

[0117] In this embodiment, a staging detection model is constructed for each skin tumor disease based on its pathological data and the tag feature set of all disease stage labels within the skin tumor disease set. This focuses on each specific skin tumor disease. The pathological data for each disease contains its core pathological information, such as disease name, location of onset, set of stage labels, disease characteristic data for each stage, and treatment plan. This data forms the basis for defining the essential attributes of the disease and the staging criteria. The tag feature set of all disease stage labels is an extraction of key features for each stage label, covering multiple dimensions related to stage, including clinical, imaging, and disease course characteristics. These features are the core markers for distinguishing different stages. When constructing the staging detection model, the staging logic in the disease pathological data is combined with the feature information in the tag feature set, allowing the model to learn the pathological patterns of the disease and the correlations between the characteristics of each stage. For example, for malignant melanoma, the model is constructed by combining the staging criteria from stage I to stage IV in the pathological data, as well as information such as tumor thickness, lymph node metastasis, and imaging features in the feature set of each stage label. This model can output the corresponding staging result based on the input patient data, and achieve accurate detection of different stages of a single disease.

[0118] In this embodiment, the skin tumor disease set includes various types of skin tumors, each with its own dedicated staging detection model. These staging models not only determine the stage of the disease but also contain unique pathological, clinical, and imaging features, which are crucial for distinguishing different skin tumors. When constructing the disease detection model, the disease-specific features contained in the staging detection models of all individual diseases are integrated and filtered to form a disease feature library covering the entire skin tumor disease set. This disease detection model learns the characteristic differences between different skin tumor diseases, such as the differences in pathological features and imaging manifestations between basal cell carcinoma and squamous cell carcinoma, and the differences in disease progression and clinical symptoms between malignant melanoma and benign nevi. Through this integration, the disease detection model can first determine the specific type of skin tumor the patient has based on the input patient data, laying the foundation for further staging detection and achieving the identification and differentiation of various diseases and stages within the entire skin tumor disease set.

[0119] In this embodiment, the historical stage-course data corresponding to each disease stage label includes patient group data with similar disease course characteristics. The stage-course feature set of all previous stage labels records the disease course characteristics experienced by these patients in each preceding stage before entering the current stage. This information clearly reflects the patterns and trends in patient disease progression and is an important basis for predicting subsequent disease progression. When constructing a disease progression prediction model, this model is based on these preceding stage disease course feature sets to learn the characteristic changes in patients from previous stages to the current stage, as well as the correlation between different feature combinations and the speed and direction of subsequent disease progression. For example, for a patient group in a certain historical stage-course data of a certain stage II disease, the model combines their stage-course feature set from stage I to analyze the common characteristics and individual differences of these patients progressing from stage I to stage II, and constructs a model that can predict the possibility, estimated time, and possible characteristic changes of this patient group from the current stage II to stage III. In this way, a unique disease progression prediction model is built for different historical stage-disease data under each disease stage label, enabling accurate prediction of the subsequent disease progression of patient groups with different disease progression characteristics.

[0120] The beneficial effects of the above technologies are as follows: Based on the tag feature set of all disease stage tags for all skin tumor diseases and the all stage-course feature set, a disease detection model based on skin tumors can be constructed, and a disease progression prediction model for each disease stage tag of each skin tumor disease can be constructed. This can form a complete system of disease identification, stage judgment and progression prediction, realize accurate detection of disease and stage and prediction of disease progression, provide full-process data support for clinical practice from diagnosis to prognosis, reduce the functional limitations of single models, and improve the overall auxiliary ability for the diagnosis and treatment of skin tumors.

[0121] Example 8:

[0122] This invention provides a heterogeneous data feature extraction system based on multimodal information fusion, used to execute any one of the heterogeneous data feature extraction methods based on multimodal information fusion in Examples 1 to 7, with reference to... Figure 2 ,include:

[0123] Data Acquisition Module: Collects multi-source pathological data based on skin tumors, identifies the skin tumor disease set and pathological data of multiple diseases, and obtains historical patient data based on skin tumors;

[0124] Determining the module: Based on the skin tumor disease set, all disease pathology data, and historical patient data, determine the historical stage data and disease development vector of each disease stage label in the stage label set for each skin tumor disease in the skin tumor disease set;

[0125] Disease progression module: Based on the historical staging data and disease progression vector of each disease stage tag in the staging tag set of each skin tumor disease, determine the disease progression vector of each patient in the historical staging data of each disease stage tag of each skin tumor disease;

[0126] Calculation module: Based on the historical staging data of each disease stage label in the staging label set of each skin tumor disease and the disease progression vector of all patients in the historical staging data, calculate the label feature set of each disease stage label of each skin tumor disease and multiple stage-progression feature sets;

[0127] Building Module: Based on the tag feature set of all disease stage labels for all skin tumor diseases and the all stage-course feature set, a disease detection model for skin tumors is constructed, and a disease progression prediction model for each disease stage label for each skin tumor disease is constructed.

[0128] The beneficial effects of the above technologies are as follows: By collecting multi-source pathological data to determine a set of skin tumor diseases and multiple disease pathological data, historical patient data based on skin tumors is obtained. This allows for the determination of historical stage data, disease development vectors, and disease progression vectors for each patient within the stage label set of each skin tumor disease in the skin tumor disease set. Furthermore, it enables the calculation of label feature sets for each disease stage label of each skin tumor disease, as well as multiple stage-progression feature sets. This leads to the construction of a disease detection model based on skin tumors and a disease progression prediction model for each disease stage label of each skin tumor disease. This overcomes the limitations of single data and independent analysis, accurately extracting the core features of each disease and stage, and constructing a complete system for disease detection and progression prediction. It provides clinical support across the entire chain from disease identification and staging to progression prediction, reducing diagnostic and treatment biases caused by data fragmentation, facilitating personalized treatment and early intervention, and improving the accuracy and foresight of skin tumor diagnosis and treatment.

[0129] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A heterogeneous data feature extraction method based on multi-modal information fusion, characterized in that, Comprising: S1: Collect multi-source pathological data based on skin tumors, determine a skin tumor disease set and multiple disease pathological data, and obtain historical patient data based on skin tumors; S2: Based on the skin tumor disease set, all disease pathological data and historical patient data, determine the historical staging data and disease development vector of each disease staging label in the staging label set of each skin tumor disease in the skin tumor disease set; S3: Based on the historical staging data and disease development vector of each disease staging label in the staging label set of each skin tumor disease, determine the disease course development vector of each patient in the historical staging data of each disease staging label of each skin tumor disease; S4: Based on the historical staging data of each disease staging label in the staging label set of each skin tumor disease and the disease course development vector of all patients in the historical staging data, calculate the label feature set and multiple staging-disease course feature sets of each disease staging label of each skin tumor disease; S5: Based on the label feature set and all staging-disease course feature sets of all disease staging labels of all skin tumor diseases, build a disease detection model based on skin tumors, and build a disease course development prediction model for each disease staging label of each skin tumor disease. 2.The heterogeneous data feature extraction method based on multi-modal information fusion of claim 1, characterized in that, Collect multi-source pathological data based on skin tumors, determine a skin tumor disease set and multiple disease pathological data, comprising: Collect multi-source pathological data based on skin tumors, wherein the multi-source pathological data includes multiple pathological sub-data; Based on all pathological sub-data in the multi-source pathological data, determine a skin tumor disease set and disease pathological data of each skin tumor disease in the skin tumor disease set, wherein the disease pathological data includes disease name, disease site, staging label set, disease feature data of each disease staging label in the staging label set, and multiple treatment options, and the staging label set includes multiple disease staging labels. 3.The heterogeneous data feature extraction method based on multi-modal information fusion of claim 2, characterized in that, Obtain historical patient data based on skin tumors, comprising: Obtain historical patient sub-data of each skin tumor disease in the skin tumor disease set, wherein the historical patient sub-data includes patient clinical data of multiple patients, and the patient clinical data includes name, gender, age, disease time, multiple visit times, diagnosis label, and examination image data; Based on the historical patient sub-data of all skin tumor diseases in the skin tumor disease set, determine the historical patient data based on skin tumors.

4. The heterogeneous data feature extraction method based on multi-modal information fusion according to claim 3, characterized in that, Based on the skin tumor disease set, all disease pathological data and historical patient data, determine the historical staging data and disease development vector of each disease staging label in the staging label set of each skin tumor disease in the skin tumor disease set, comprising: The historical staging data of each disease staging label of each skin tumor disease is determined based on the set of staging labels in the disease pathology data of each skin tumor disease in the set of skin tumor diseases and the latest diagnosis label in the patient clinical data of all patients in the historical patient sub-data of each skin tumor disease in the historical patient data, wherein the historical staging data comprises patient clinical data of a plurality of patients; The disease progression vector of each skin tumor disease based on the set of staging labels is determined based on the set of staging labels in the disease pathology data of each skin tumor disease in the set of skin tumor diseases.

5. The heterogeneous data feature extraction method based on multi-modal information fusion according to claim 4, characterized in that, The disease progression vector of each patient in the historical staging data of each disease staging label of each skin tumor disease is determined based on the historical staging data of each disease staging label in the set of staging labels of each skin tumor disease and the disease progression vector, comprising: The diagnosis label of the earliest visit time is selected from the visit time and diagnosis label of all visits in the patient clinical data of each patient in the historical staging data of each disease staging label of each skin tumor disease in the set of skin tumor diseases, and the staging diagnosis time of the current staging of each patient in the historical staging data of each disease staging label of each skin tumor disease is determined; The staging diagnosis time of each disease staging label before the disease staging label in the set of staging labels is determined based on the visit time and diagnosis label of all visits in the patient clinical data of each patient in the historical staging data of each disease staging label of each skin tumor disease and the set of staging labels in the disease pathology data of each skin tumor disease; The disease progression vector of each patient in the historical staging data of each disease staging label of each skin tumor disease is determined based on the staging diagnosis time of the current staging of each patient, the staging diagnosis time of all disease staging labels before the disease staging label in the set of staging labels, and the disease progression vector of each disease staging label of each skin tumor disease. 6.The method of claim 5, wherein, The label feature set and a plurality of staging-disease course feature sets of each disease staging label of each skin tumor disease are calculated based on the historical staging data of each disease staging label in the set of staging labels of each skin tumor disease and the disease progression vector of all patients in the historical staging data, comprising: Based on the disease progression vector of all patients in the historical stage data of each disease stage label of each skin tumor disease in the skin tumor disease set, cluster analysis is performed on all patients in the historical stage data of each disease stage label of each skin tumor disease in the skin tumor disease set. Based on the cluster analysis results, multiple historical stage-disease progression data of each disease stage label of each skin tumor disease in the skin tumor disease set are determined. The historical stage-disease progression data includes the clinical data of multiple patients and the development label of each development feature in the disease progression vector. For each disease stage label of each skin tumor disease in the skin tumor disease set, feature extraction is performed on the examination image data of each patient's clinical data from each visit in the historical stage-disease course data of each patient for each disease stage label of each skin tumor disease in the skin tumor disease set. This determines the image feature vector of each visit in the clinical data of each patient for each visit in the historical stage-disease course data of each patient for each disease stage label of each skin tumor disease in the skin tumor disease set. Based on the disease development vector of each disease stage label for each skin tumor disease in the skin tumor disease set, the image feature vector of all visits in the clinical data of all patients in each historical stage-disease course data, and the disease development vector of all patients in each historical stage-disease course data, calculate the associated sub-feature set of the disease stage label of each disease stage label for each historical stage-disease course data of each skin tumor disease in the skin tumor disease set, and the associated sub-feature set of the disease stage labels before the disease stage label in the stage label set; Based on the set of stage labels in the pathological data of each skin tumor disease in the skin tumor disease set, the set of associated sub-features of the disease stage labels in all historical stage-course data of all disease stage labels, and the set of associated sub-features of disease stage labels preceding the disease stage labels in the set of stage labels, the tag feature set of each disease stage label of each skin tumor disease in the skin tumor disease set, and the stage-course feature set of each disease stage label preceding the disease stage label in each historical stage-course data are calculated.

7. The heterogeneous data feature extraction method based on multi-modal information fusion according to claim 6, characterized in that, Based on the tag feature set of all disease stage labels for all skin tumor diseases and the all stage-course feature set, a disease detection model for skin tumors is constructed, and a disease progression prediction model for each disease stage label of each skin tumor disease is also constructed, including: Based on the pathological data of each skin tumor disease in the skin tumor disease set and the tag feature set of all disease stage labels, a stage detection model for each skin tumor disease in the skin tumor disease set is constructed. Based on the staging detection model of all skin tumor diseases in the skin tumor disease set, a disease detection model for the skin tumor disease set is constructed. Based on the staging-course feature set of all previous staging labels of all historical staging-course data of each disease staging label of each skin tumor disease in the skin tumor disease set, a course development prediction model of the historical staging-course data of each disease staging label of each skin tumor disease in the skin tumor disease set is constructed.

8. A heterogeneous data feature extraction system based on multi-modal information fusion, characterized in that, The method comprises the following steps: An acquisition module acquires multi-source pathological data based on skin tumors, determines a skin tumor disease set and multiple disease pathological data, and obtains historical patient data based on skin tumors. A determination module determines historical staging data and disease development vectors of each disease staging label in a staging label set of each skin tumor disease in the skin tumor disease set based on the skin tumor disease set, all disease pathological data, and the historical patient data. A course module determines a course development vector of each patient in the historical staging data of each disease staging label of each skin tumor disease based on the historical staging data and the disease development vectors of each disease staging label in the staging label set of each skin tumor disease. A calculation module calculates a label feature set and multiple staging-course feature sets of each disease staging label of each skin tumor disease based on the historical staging data of each disease staging label in the staging label set of each skin tumor disease and the course development vectors of all patients in the historical staging data. A construction module constructs a disease detection model based on skin tumors based on the label feature sets and the staging-course feature sets of all disease staging labels of all skin tumor diseases, and constructs a course development prediction model of each disease staging label of each skin tumor disease.

Images