Image auxiliary diagnosis system based on AI

Abstract

The invention relates to intelligent medical treatment, in particular to an AI-based image auxiliary diagnosis system, medical staff can conveniently and quickly upload original medical images and medical data to an AI image auxiliary diagnosis platform through a medical auxiliary diagnosis sub-terminal, and the AI image auxiliary diagnosis platform determines the medical image and medical data according to the uploaded medical images and medical data. Preliminary diagnosis of patient diseases is carried out, and an auxiliary diagnosis effect is conveniently provided for a final diagnosis result. The AI image auxiliary diagnosis platform can further plan and provide a treatment planning scheme for the reference of medical staff according to the diagnosis result.

Description

Technical Field

[0001] The present invention relates to the field of smart medical technology, and more specifically, to an AI-based image-assisted diagnosis system. Background Art

[0002] Existing traditional medical assistance systems have a single data input method and are often unable to convert and judge based on actual medical images and medical data. The medical system's operating steps are complex, and medical personnel must input data based on a static guideline library. They are unable to dynamically optimize solutions based on specific individual pathological characteristics, resulting in insufficient solution applicability. During the judgment process, they only analyze a single modality from CT, MRI, or X-rays, and fail to integrate complementary information from multiple sources of imaging (such as the synergy of metabolic and anatomical information from PET-CT). This leads to insufficient sensitivity. After model training, the parameters are fixed and cannot be dynamically optimized based on the local data distribution of the hospital. This leads to poor generalization ability and static model defects. Therefore, based on the technical problems existing in the above-mentioned existing technologies, an AI-based image-assisted diagnosis system is designed that integrates image features, clinical data and biomechanics, which is suitable for clinical decision support in hospitals, emergency centers and telemedicine scenarios. Summary of the Invention

[0003] In view of the shortcomings of the existing technology, the purpose of the present invention is to provide an AI-based image-assisted diagnosis system to solve the problems existing in the above-mentioned background technology.

[0004] The above technical objectives of the present invention are achieved through the following technical solutions: an AI-based image-assisted diagnosis system, comprising: The medical auxiliary diagnosis sub-terminal is installed on the equipment of each medical staff and is used for medical staff to upload and input original medical images and patient medical data; The regional Internet of Things transmission platform is used to receive the data transmitted by the medical auxiliary diagnosis sub-terminal, classify it and assign serial numbers to it, and then upload and transmit the data bound to the serial numbers through the Internet of Things; An AI image-assisted diagnosis platform is provided at the system terminal, and is used to receive data transmitted by the regional IoT transmission platform, pre-process the initial data, extract image features from the processed data, and then confirm the auxiliary diagnosis results and provide treatment planning solutions based on the auxiliary diagnosis results; The AI ​​image-assisted diagnosis platform includes: a data preprocessing module, configured to receive the original medical images transmitted by the regional IoT transmission platform, remove noise and preprocess them, fit them with the patient's medical data to obtain multimodal image data, and then transmit the multimodal image data; an anatomical feature extraction module, configured to receive the multimodal image data transmitted by the data preprocessing module, extract basic features, convert them into model input feature vectors, and transmit them; A disease adaptive judgment engine is used to construct a solution screening adaptation model, which is based on the model input feature vector transmitted by the anatomical feature extraction module to obtain an auxiliary diagnosis result; Big data management module, used to store patient historical case data; The intelligent diagnosis and treatment decision module calculates the treatment plan based on the diagnosis and treatment results obtained by the disease adaptive judgment engine and the data stored in the big data management module; The visual interface module is used to integrate and display various data and synchronously transmit the data to the medical auxiliary diagnosis sub-terminal for medical staff to view and make auxiliary judgments.

[0005] Specifically, the visual interface module can be projected to the control terminal, or it can be transmitted to the corresponding medical auxiliary diagnosis sub-terminal according to the serial number for medical staff to view and improve the effect of auxiliary diagnosis.

[0006] Optionally, the anatomical feature extraction module includes: Image preprocessing unit, used to preprocess multimodal image data to solve noise and standardization problems before transmission; an organ localization unit, configured to sequentially perform body region localization, body system differentiation, and organ segmentation on the medical image of the multimodal image data processed by the image preprocessing unit, so as to extract organ features; a blood vessel tree segmentation and extraction unit, configured to extract blood vessel features based on the medical image of the multimodal image data processed by the image preprocessing unit; an organ-specific feature extraction unit, configured to extract organ-specific features representing organ specificity based on the organ features extracted by the organ positioning unit and the multimodal image data; A spatial relationship modeling unit, which uses an anatomical graph neural network to perform spatial modeling based on the organ features extracted by the organ positioning unit, thereby obtaining the organ spatial features; The feature integration unit integrates the organ features extracted by the organ positioning unit, the vascular features extracted by the vascular tree segmentation and extraction unit, the organ-specific features extracted by the organ-specific feature extraction unit, and the organ spatial features extracted by the spatial relationship modeling unit to generate a model input feature vector.

[0007] Optionally, the disease adaptive judgment engine includes: Feature preprocessing unit, based on the model input feature vector, filters abnormal features and then transmits them; The disease screening unit uses a clinically guided attention mechanism based on the model input feature vector to screen the body systems involved in the model input feature vector and then calculate the body systems involved in the auxiliary diagnosis results; The disease identification and diagnosis unit, based on the body systems involved in the disease calculated by the disease screening unit, uses the model input feature vector as the model input quantity in the preset multi-system disease identification sub-model to obtain the auxiliary diagnosis result.

[0008] Optionally, the intelligent diagnosis and treatment decision module includes: A plan-feature driven screening unit is used to build a plan screening adaptation model. Based on the calculated auxiliary diagnosis results, the plan screening adaptation model is used to screen suitable medical plans; the medical plans include: drug treatment plans and surgical treatment plans; A drug regimen calculation unit calculates the amount of each drug required by the patient based on the drug regimen selected by the regimen-feature driven screening unit and a pharmacokinetic adaptive model, and further optimizes the amount of drug required by the patient based on a multi-dimensional risk warning model based on the mutual resistance of each drug, thereby obtaining an auxiliary judgment result of the drug regimen; The surgical plan simulation calculation unit constructs a pathological model based on the model input feature vector. Then, based on the constructed pathological model and the surgical plan selected by the plan-feature driven screening unit, it performs virtual surgery based on the physical engine, simulates the surgical effect, and then provides auxiliary judgment results for the surgical plan. The intelligent diagnosis and treatment judgment unit is used to calculate the treatment plan based on the auxiliary judgment results of the drug plan calculation unit and the auxiliary judgment results of the surgical plan simulation calculation unit, and synchronously combine the patient historical case data stored in the big data management module.

[0009] Optionally, the intelligent diagnosis and treatment judgment unit also includes: a patient risk management block, which is used to construct a dynamic risk adaptive model, import the dynamic risk adaptive model as a model input based on the treatment planning plan, and then calculate the subsequent follow-up time and review plan.

[0010] An AI image-assisted diagnosis method based on the above-mentioned AI-based image-assisted diagnosis system includes: step S1, a regional Internet of Things transmission platform receives original medical images and patient medical data transmitted by a medical-assisted diagnosis sub-terminal, classifies and assigns corresponding serial numbers, and then binds the transmitted data to the serial numbers and uploads them; Step S2: The AI ​​image-assisted diagnosis platform receives data transmitted by the regional IoT transmission platform, removes noise and preprocesses the transmitted data, and then obtains multimodal image data; Step S3: The AI ​​image-assisted diagnosis platform extracts basic features based on the multimodal image data and converts them into model input feature vectors; Step S4: The AI ​​image-assisted diagnosis platform makes a diagnosis and treatment judgment based on the model input feature vector, and then obtains an auxiliary diagnosis result; Step S5: The AI ​​image-assisted diagnosis platform calculates a treatment plan based on the auxiliary diagnosis results and the patient's historical case data; Step S6: The AI ​​image-assisted diagnosis platform transmits the treatment plan to the medical auxiliary diagnosis sub-terminal, so that medical personnel can perform medical diagnosis based on the auxiliary diagnosis results and the treatment plan.

[0011] Optionally, the specific implementation process of step S3 is as follows, including: S3.1. After receiving the pre-processed medical image, a coarse segmentation method is used to sequentially output probability items for the trunk, limbs, and head and neck, and the involved body regions are identified based on the probability items. S3.2. Based on the body region determined in step S3.1, further locate the body system of the medical image, perform precise organ segmentation based on the corresponding body system feature items, output an organ mask of the corresponding organ on the medical image, and thereby derive the organ features of the medical image; S3.3. Based on the medical image, anisotropic diffusion tracking is used to obtain a number of vascular seed points in the medical image. Based on the number of vascular seed points, a diffusion tracking algorithm is used to obtain the distribution of vascular branches in the medical image and obtain vascular characteristics. S3.4. Based on the medical images and the organ features obtained in step S3.2, extract a morphological feature group, a texture feature group, and a functional feature group using a feature extraction method. The extracted morphological feature group, texture feature group, and functional feature group are integrated and a deep learning method is used to output organ-specific features. S3.5. Based on the acquired organ features, an anatomical graph neural network method is used to construct graph structures, and graph convolution operations are performed to output spatial enhancement features to represent the spatial relationship between organs, thereby deriving organ spatial features; S3.6. Integrate organ features, vascular features, organ-specific features, and organ spatial features to derive a model input feature vector.

[0012] Optionally, the specific implementation process of step S4 is as follows, including: S4.1. Filter the received model input feature vector for abnormal features and then perform preprocessing to filter out irrelevant factors; S4.2. Screen and identify the preprocessed model input feature vector, and determine the body system to which the model input feature vector belongs based on the organ features in the model input feature vector; S4.3. Based on the body system in which the model input feature vector is located, the corresponding system disease analysis sub-model is used, and the multi-system sign joint reasoning method is adopted to identify the disease and obtain the auxiliary diagnosis result.

[0013] A computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned AI image-assisted diagnosis method when the computer program is executed.

[0014] A computer program product, comprising a computer program stored on a non-transitory computer-readable storage medium, wherein the computer program comprises program instructions, which, when executed by a computer, cause the computer to perform an AI image-assisted diagnosis method.

[0015] In summary, the present invention has the following beneficial effects: 1. Medical staff can quickly and easily upload original medical images and medical data to the AI ​​image-assisted diagnosis platform through the medical auxiliary diagnosis sub-terminal. The AI ​​image-assisted diagnosis platform can make a preliminary diagnosis of the patient's disease based on the uploaded medical images and medical data, and provide auxiliary diagnosis effects for the final diagnosis result. Based on the diagnosis results, the AI ​​image-assisted diagnosis platform can further plan and provide treatment plans for medical staff to refer to.

[0016] 2. When diagnosing diseases, the AI ​​image-assisted diagnosis platform solves the problem of traditional image analysis in traditional systems ignoring anatomical-functional correlations, and also solves the problem of insufficient recognition of cross-system disease correlations. It does not directly judge based on existing technology, but uses AI big data models to further trace back to other related symptoms and adopts multi-system decision fusion to further improve the accuracy of diagnostic results. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic diagram of the system execution logic flow of the present invention; Figure 2 It is a schematic diagram of the original image segmentation processing method of the present invention; Figure 3 It is a schematic diagram of the auxiliary diagnosis process of the present invention; Figure 4 It is a flow chart of the AI ​​image-assisted diagnosis method of the present invention. DETAILED DESCRIPTION

[0018] To make the objectives, features, and advantages of the present invention more readily apparent, the following detailed description of the present invention is provided with reference to the accompanying drawings. The accompanying drawings illustrate several embodiments of the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein.

[0019] In the present invention, unless otherwise expressly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be a communication between the two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances. The terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of such features.

[0020] In the present invention, unless otherwise expressly specified and limited, a first feature being "above" or "below" a second feature may include the first and second features being in direct contact, or may include the first and second features being in contact not directly but through another feature between them. Moreover, a first feature being "above," "above," and "above" a second feature includes the first feature being directly above and obliquely above the second feature, or simply indicates that the first feature is higher in level than the second feature. A first feature being "below," "below," and "below" a second feature includes the first feature being directly below and obliquely below the second feature, or simply indicates that the first feature is lower in level than the second feature. The terms "vertical," "horizontal," "left," "right," "above," "below," and similar expressions are for illustrative purposes only and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore should not be understood as limiting the present invention.

[0021] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0022] The present invention provides an AI-based image-assisted diagnosis system, such as Figure 1 As shown, including: The medical auxiliary diagnosis sub-terminal is installed on the equipment of each medical staff and is used for medical staff to upload and input original medical images and patient medical data; The regional Internet of Things transmission platform is used to receive the data transmitted by the medical auxiliary diagnosis sub-terminal, classify it and assign serial numbers to it, and then upload and transmit the data bound to the serial numbers through the Internet of Things; An AI image-assisted diagnosis platform is provided at the system terminal, and is used to receive data transmitted by the regional IoT transmission platform, pre-process the initial data, extract image features from the processed data, and then confirm the auxiliary diagnosis results and provide treatment planning solutions based on the auxiliary diagnosis results; The AI ​​image-assisted diagnosis platform includes: a data preprocessing module, configured to receive the original medical images transmitted by the regional IoT transmission platform, remove noise and preprocess them, fit them with the patient's medical data to obtain multimodal image data, and then transmit the multimodal image data; an anatomical feature extraction module, configured to receive the multimodal image data transmitted by the data preprocessing module, extract basic features, convert them into model input feature vectors, and transmit them; A disease adaptive judgment engine is used to construct a solution screening adaptation model, which is based on the model input feature vector transmitted by the anatomical feature extraction module to obtain an auxiliary diagnosis result; Big data management module, used to store patient historical case data; The intelligent diagnosis and treatment decision module calculates the treatment plan based on the diagnosis and treatment results obtained by the disease adaptive judgment engine and the data stored in the big data management module; The visual interface module is used to integrate and display various data and synchronously transmit the data to the medical auxiliary diagnosis sub-terminal for medical staff to view and make auxiliary judgments.

[0023] Furthermore, the anatomical feature extraction module includes: Image preprocessing unit, used to preprocess multimodal image data to solve noise and standardization problems before transmission; an organ localization unit, configured to sequentially perform body region localization, body system differentiation, and organ segmentation on the medical image of the multimodal image data processed by the image preprocessing unit, so as to extract organ features; a blood vessel tree segmentation and extraction unit, configured to extract blood vessel features based on the medical image of the multimodal image data processed by the image preprocessing unit; an organ-specific feature extraction unit, configured to extract organ-specific features representing organ specificity based on the organ features extracted by the organ positioning unit and the multimodal image data; A spatial relationship modeling unit, which uses an anatomical graph neural network to perform spatial modeling based on the organ features extracted by the organ positioning unit, thereby obtaining the organ spatial features; The feature integration unit integrates the organ features extracted by the organ positioning unit, the vascular features extracted by the vascular tree segmentation and extraction unit, the organ-specific features extracted by the organ-specific feature extraction unit, and the organ spatial features extracted by the spatial relationship modeling unit to generate a model input feature vector.

[0024] Furthermore, the disease adaptive judgment engine includes: Feature preprocessing unit, based on the model input feature vector, filters abnormal features and then transmits them; The disease screening unit uses a clinically guided attention mechanism based on the model input feature vector to screen the body systems involved in the model input feature vector and then calculate the body systems involved in the auxiliary diagnosis results; The disease identification and diagnosis unit, based on the body systems involved in the disease calculated by the disease screening unit, uses the model input feature vector as the model input quantity in the preset multi-system disease identification sub-model to obtain the auxiliary diagnosis result.

[0025] Furthermore, the intelligent diagnosis and treatment decision module includes: A plan-feature driven screening unit is used to build a plan screening adaptation model. Based on the calculated auxiliary diagnosis results, the plan screening adaptation model is used to screen suitable medical plans; the medical plans include: drug treatment plans and surgical treatment plans; A drug regimen calculation unit calculates the amount of each drug required by the patient based on the drug regimen selected by the regimen-feature driven screening unit and a pharmacokinetic adaptive model, and further optimizes the amount of drug required by the patient based on a multi-dimensional risk warning model based on the mutual resistance of each drug, thereby obtaining an auxiliary judgment result of the drug regimen; The surgical plan simulation calculation unit constructs a pathological model based on the model input feature vector. Then, based on the constructed pathological model and the surgical plan selected by the plan-feature driven screening unit, it performs virtual surgery based on the physical engine, simulates the surgical effect, and then provides auxiliary judgment results for the surgical plan. The intelligent diagnosis and treatment judgment unit is used to calculate the treatment plan based on the auxiliary judgment results of the drug plan calculation unit and the auxiliary judgment results of the surgical plan simulation calculation unit, and synchronously combine the patient historical case data stored in the big data management module.

[0026] In the specific implementation process, based on the diagnosis results, a scheme screening and adaptation model is constructed. The scheme screening and adaptation model adopts a three-level matching mechanism. The first level is based on the diagnosis results to connect the clinical guideline knowledge map or the AI ​​big data model based on the scheme library formed by the past case treatment measures, and then obtain the scheme on how to treat the disease; the second level is to perform scheme-driven screening based on the characteristics, which is expressed as a scheme ,in It represents the existence of the key features of the treatment plan, with a value of 0 / 1. For feature weights, for example, in the treatment of acute myocardial infarction, there are three treatment options: 1. For intravenous thrombolysis, the key features must be onset <12 hours and ST-segment elevation. The applicability of the regimen is 0.92 according to the above formula. 2. For primary PCI, the key features are onset <24 hours and cardiogenic shock. The applicability of the protocol calculated using the above formula is 0.97. 3. CABG, where the key features are multivessel disease and left main coronary artery disease >50%, the appropriateness of the scheme is 0.85 according to the above formula; Based on the above treatment plans, risk threshold features are also set as contraindication features to avoid medication risks. For example, the risk threshold features in Treatment Plan 1 are recent cerebral hemorrhage and blood pressure >180, the risk threshold features in Treatment Plan 2 are contrast agent allergy and glomerular filtration rate <30, and the risk threshold features in Treatment Plan 3 are severe pulmonary insufficiency. When the features involve risk threshold features, the treatment plan is directly rejected to ensure the patient's life safety. When drug administration is involved in the treatment plan, a pharmacodynamic interaction algorithm is used based on the pharmacokinetic adaptive model, which is expressed as ,in Expressed as the effect change, It is expressed as the concentration change and the pharmacokinetic interaction algorithm is used, which is expressed as ,in Expressed as therapeutic index, dynamically fitted based on the patient's medical information and The index outputs the risk index score and outputs corresponding treatment measures according to the risk level score, including: monitoring relevant indicators in low-risk cases; dose adjustment + search for alternative drugs in medium-risk cases; absolute contraindications in high-risk cases, triggering a red alert and prohibiting the plan; When the treatment plan involves surgery, a standard model of the corresponding body system is used. Organ characteristics, organ-specific characteristics, and organ spatial characteristics are substituted into the standard model to form a pathology model. Based on the surgical content, the surgery is simulated based on the physics engine. The success rate and complications are output based on the surgical probability and complication probability for reference by medical staff. Integrate the above-mentioned drug and surgical plans, intelligently select and screen treatment plans based on the patient's medical information, and then obtain a treatment plan.

[0027] Furthermore, the intelligent diagnosis and treatment judgment unit also includes: a patient risk management block, which is used to construct a dynamic risk adaptive model, import the dynamic risk adaptive model as a model input based on the treatment planning plan, and then calculate the subsequent follow-up time and review plan.

[0028] In other embodiments, the follow-up time can be calculated based on the patient's medical information, which can be expressed as ,in , and then plan the re-examination and re-examination time period, and calculate the follow-up content according to the treatment plan. The data reference can be a reference database formed by historical case data and a reference database trained by the AI ​​big data model.

[0029] An AI image-assisted diagnosis method based on the above-mentioned AI-based image-assisted diagnosis system includes: step S1, a regional Internet of Things transmission platform receives original medical images and patient medical data transmitted by a medical-assisted diagnosis sub-terminal, classifies and assigns corresponding serial numbers, and then binds the transmitted data to the serial numbers and uploads them; Step S2: The AI ​​image-assisted diagnosis platform receives data transmitted by the regional IoT transmission platform, removes noise and preprocesses the transmitted data, and then obtains multimodal image data; Step S3: The AI ​​image-assisted diagnosis platform extracts basic features based on the multimodal image data and converts them into model input feature vectors; Step S4: The AI ​​image-assisted diagnosis platform makes a diagnosis and treatment judgment based on the model input feature vector, and then obtains an auxiliary diagnosis result; Step S5: The AI ​​image-assisted diagnosis platform calculates a treatment plan based on the auxiliary diagnosis results and the patient's historical case data; Step S6: The AI ​​image-assisted diagnosis platform transmits the treatment plan to the medical auxiliary diagnosis sub-terminal, so that medical personnel can perform medical diagnosis based on the auxiliary diagnosis results and the treatment plan.

[0030] Optionally, the specific implementation process of step S3 is as follows, including: S3.1. After receiving the pre-processed medical image, a coarse segmentation method is used to sequentially output probability items for the trunk, limbs, and head and neck, and the involved body regions are identified based on the probability items. S3.2. Based on the body region determined in step S3.1, further locate the body system of the medical image, perform precise organ segmentation based on the corresponding body system feature items, output an organ mask of the corresponding organ on the medical image, and thereby derive the organ features of the medical image; S3.3. Based on the medical image, anisotropic diffusion tracking is used to obtain a number of vascular seed points in the medical image. Based on the number of vascular seed points, a diffusion tracking algorithm is used to obtain the distribution of vascular branches in the medical image and obtain vascular characteristics. S3.4. Based on the medical images and the organ features obtained in step S3.2, extract a morphological feature group, a texture feature group, and a functional feature group using a feature extraction method. The extracted morphological feature group, texture feature group, and functional feature group are integrated and a deep learning method is used to output organ-specific features. S3.5. Based on the acquired organ features, an anatomical graph neural network method is used to construct graph structures, and graph convolution operations are performed to output spatial enhancement features to represent the spatial relationship between organs, thereby deriving organ spatial features; S3.6. Integrate organ features, vascular features, organ-specific features, and organ spatial features to derive a model input feature vector.

[0031] In a specific embodiment, Figure 2 As shown, the coarse segmentation method in the organ localization unit is expressed as , and then the probabilities of the body trunk, limbs and head and neck are obtained, and then the body regions corresponding to the medical images are respectively determined; after the AI ​​image-assisted diagnosis platform performs preliminary screening based on the body systems contained in the body region, it further positions and screens the medical images, which is expressed as , and then obtain and identify the body system in which the medical image is located, including 8 major systems such as circulation / respiratory / digestive / nervous; After distinguishing the body regions and body systems corresponding to the medical images, the AI ​​image-assisted diagnosis platform, based on the AI ​​big model, extracts the organ feature reference parameters corresponding to the body regions and body systems through historical case data. Based on the organ feature reference parameters, the organs in the medical images are precisely segmented, which is expressed as , and then the organ characteristics are expressed as ,The extracted organ features include volume, density mean (e.g., lung average CT value -780HU), morphological indices (sphericity, surface curvature), and tissue heterogeneity (entropy); By using anisotropic diffusion tracking method to track vascular branches on medical images, the vascular seed points are first detected and expressed as , where H represents the Hessian matrix, =0.05 is expressed as the vascular similarity parameter. Based on the extracted vascular seed points, the diffusion tracking algorithm is used to obtain the vascular branch distribution map after diffusion, and post-processing optimization is used. ,in It is represented as morphological opening and closing operation to remove false positive noise and extract blood vessel features, which is expressed as , vascular characteristics include calcification score (Agatston unit), vascular density (length / volume), and tortuosity (actual length / straight-line distance); In organ-specific analysis, the organ mask in the input organ feature is And the original medical image, first extract the morphological feature set, including volume: , sphericity: , surface curvature: , and then extract the texture feature set, including contrast: , correlation: , and then extract the functional feature group (dynamic enhancement), including the enhancement curve: , used to extract parameters such as peak time and enhancement rate; through the above feature group, deep learning of historical data is performed to output organ-specific features, which are expressed as , including peak reinforcement rate, time to peak, and reinforcement amplitude; In the analysis of organ spatial relationships, by inputting all organ features, an anatomical graph neural network is used, including graph structure construction, where the nodes of the graph structure are organ feature vectors, and the edges of the graph structure are represented as ,in It is represented by the centroid coordinates of organ i, and represents a specific organ. The data of anatomical correlation adopts Gray's Anatomy knowledge base. Based on the constructed graph structure, the graph convolution operation is performed, which is represented as , the output organ spatial features are expressed as , including the distance from the organ's center of mass (mm), the contact surface angle (degrees), and the deformation vector field (displacement); Integrate the features calculated above to obtain the model input feature vector .

[0032] Optionally, the specific implementation process of step S4 is as follows, including: S4.1. Filter the received model input feature vector for abnormal features and then perform preprocessing to filter out irrelevant factors; S4.2. Screen and identify the preprocessed model input feature vector, and determine the body system to which the model input feature vector belongs based on the organ features in the model input feature vector; S4.3. Based on the body system in which the model input feature vector is located, the corresponding system disease analysis sub-model is used, and the multi-system sign joint reasoning method is adopted to identify the disease and obtain the auxiliary diagnosis result.

[0033] In the specific implementation process, the organ features in the received model input feature vector are filtered for abnormal features.

[0034] in and It is expressed as the distribution parameter of organ characteristics in healthy people. The reference distribution is established based on 10,000 healthy images, and the characteristics are weighted based on medical data, which is expressed as ,in It is represented as a clinical feature vector, including age, gender, symptoms, and medical history. For example, if the clinical feature of medical data is coughing for two weeks, the weighted feature is that the weight of lung features is increased by 80% and the weight of bone features is reduced by 80%, so as to achieve more accurate body system judgment. After judgment, cross-organ feature fusion is performed, which is represented as , for example, respiratory system = {lung, trachea, bronchi, pleura}, and then the body system where the model input feature vector is located is determined by the disease screening model; Based on the body system where the model input feature vector is located, the clinically guided attention mechanism is used to express it as ,At the same time, a feature screening threshold is set. Only when the feature value exceeds the feature screening threshold of the corresponding body system can it be determined that a specific body system has a disease; like Figure 3 As shown, the corresponding system disease analysis sub-model is used to calculate and compare the various features in the model input feature vector: Based on organ characteristics, preliminary judgments are mainly made for systemic disease screening. For example, when screening for lung system diseases, a decrease in lung volume parameters and an increase in lung density can be judged as interstitial lung disease. For example, when screening for liver diseases, a decrease in liver density and an increase in tissue heterogeneity can be judged as fatty liver. The screening comparison data reference can be a database formed by historical case data and a reference database after training the AI ​​big data model. Vascular characteristics can be used as either a primary factor or a hidden factor in determining disease. For example, when used as a primary factor, a preliminary diagnosis of the disease can be made: coronary artery calcification >100 AU can be considered coronary heart disease in the circulatory system; for example, aortic tortuosity >1.8 can be considered aortic dissection. When used as a hidden factor, the corresponding data can be used as a partial reference factor in the subsequent multi-system sign joint reasoning method. Based on organ-specific functional characteristics, system-specific disease screening is mainly carried out, with peak enhancement rate, peak time and enhancement amplitude as reference factors. For example, when judging liver-specific diseases, >25 and <25, it can be diagnosed as hepatic hemangioma; or <20 and When the value is >60, it can be diagnosed as hepatocellular carcinoma. The judgment data reference can be a database formed by historical case data and a reference database trained by the AI ​​big data model. Based on the above judgment, a preliminary judgment of organ-specific diseases can be achieved. Based on the spatial characteristics of organs, anatomical relationship analysis is performed by judging data references. Based on the anatomical relationship results, if anatomical relationship abnormalities are found, a preliminary disease diagnosis is made based on the database. For example, if the distance between the liver and stomach is on a downward trend, it can be diagnosed as portal hypertension. For example, if the peripancreatic fat is intermittently blurred, it can be diagnosed as acute pancreatitis. The judgment data reference can be a database formed by historical case data and a reference database trained by the AI ​​big data model. Based on the above judgment, a preliminary diagnosis of the disease can be achieved. Based on the above preliminary judgment results, a multi-system sign joint reasoning method is used to take the disease types preliminarily judged by the above characteristics as the data basis; taking the characteristic manifestations of the corresponding disease types as the benchmark, feature screening is performed, anatomical driving feature screening is performed, and the influence weight of each feature on the probability of diagnosis is calculated. , expressed as the dependency weight of disease d on organ k, and cross-modal feature fusion is performed. , and then calculate the diagnosis probability of the analyzed disease type; Based on the above method, when diagnosing respiratory diseases, the initial judgment result is that pneumonia exists, and the characteristic manifestations of pneumonia are screened. In this embodiment, they include: pleural distance, pleural thickening and vascular signs. Based on the characteristic manifestations, each feature is matched one by one with the calculated features and confirmed to confirm whether the characteristic manifestation exists. Based on the anatomical driven feature screening, the weight of each corresponding feature image is determined, and the sign reasoning is performed to calculate the expression as follows: , in the detailed and accurate classification of diseases, such as bacterial pneumonia: pleural sign + uniform consolidation, viral pneumonia: vascular sign + multiple GGO, fungal infection: cavity + halo sign.

[0035] A computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned AI image-assisted diagnosis method when the computer program is executed.

[0036] A computer program product, comprising a computer program stored on a non-transitory computer-readable storage medium, wherein the computer program comprises program instructions, which, when executed by a computer, cause the computer to perform an AI image-assisted diagnosis method.

[0037] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0038] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0039] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0040] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0041] The present invention provides an AI-based image-assisted diagnosis system. Medical personnel can quickly and easily upload original medical images and medical data to the AI ​​image-assisted diagnosis platform through the medical-assisted diagnosis sub-terminal. The AI ​​image-assisted diagnosis platform performs a preliminary diagnosis of the patient's disease based on the uploaded medical images and medical data, thereby facilitating the provision of auxiliary diagnosis effects for the final diagnosis results. The AI ​​image-assisted diagnosis platform can further plan and provide treatment plans for reference by medical personnel based on the diagnosis results.

[0042] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiment. All technical solutions based on the concept of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. An AI-based image-assisted diagnosis system, characterized in that: include: The medical auxiliary diagnosis sub-terminal is installed on the equipment of each medical staff and is used for medical staff to upload and input original medical images and patient medical data; The regional Internet of Things transmission platform is used to receive the data transmitted by the medical auxiliary diagnosis sub-terminal, classify it and assign serial numbers to it, and then upload and transmit the data bound to the serial numbers through the Internet of Things; An AI image-assisted diagnosis platform is provided at the system terminal, and is used to receive data transmitted by the regional IoT transmission platform, pre-process the initial data, extract image features from the processed data, and then confirm the auxiliary diagnosis results and provide treatment planning solutions based on the auxiliary diagnosis results; The AI ​​image-assisted diagnosis platform includes: a data preprocessing module, configured to receive the original medical images transmitted by the regional IoT transmission platform, remove noise and preprocess them, fit them with the patient's medical data to obtain multimodal image data, and then transmit the multimodal image data; an anatomical feature extraction module, configured to receive the multimodal image data transmitted by the data preprocessing module, extract basic features, convert them into model input feature vectors, and transmit them; A disease adaptive judgment engine is used to construct a solution screening adaptation model, which is based on the model input feature vector transmitted by the anatomical feature extraction module to obtain an auxiliary diagnosis result; Big data management module, used to store patient historical case data; The intelligent diagnosis and treatment decision module calculates the treatment plan based on the diagnosis and treatment results obtained by the disease adaptive judgment engine and the data stored in the big data management module; The visual interface module is used to integrate and display various data and synchronously transmit the data to the medical auxiliary diagnosis sub-terminal for medical staff to view and make auxiliary judgments.

2. The AI-based image-assisted diagnosis system according to claim 1, characterized in that: The anatomical feature extraction module includes: Image preprocessing unit, used to preprocess multimodal image data to solve noise and standardization problems before transmission; an organ localization unit, configured to sequentially perform body region localization, body system differentiation, and organ segmentation on the medical image of the multimodal image data processed by the image preprocessing unit, so as to extract organ features; a blood vessel tree segmentation and extraction unit, configured to extract blood vessel features based on the medical image of the multimodal image data processed by the image preprocessing unit; an organ-specific feature extraction unit, configured to extract organ-specific features representing organ specificity based on the organ features extracted by the organ positioning unit and the multimodal image data; A spatial relationship modeling unit, which uses an anatomical graph neural network to perform spatial modeling based on the organ features extracted by the organ positioning unit, thereby obtaining the organ spatial features; The feature integration unit integrates the organ features extracted by the organ positioning unit, the vascular features extracted by the vascular tree segmentation and extraction unit, the organ-specific features extracted by the organ-specific feature extraction unit, and the organ spatial features extracted by the spatial relationship modeling unit to generate a model input feature vector.

3. The AI-based image-assisted diagnosis system according to claim 2, characterized in that: The disease adaptive judgment engine includes: Feature preprocessing unit, based on the model input feature vector, filters abnormal features and then transmits them; The disease screening unit uses a clinically guided attention mechanism based on the model input feature vector to screen the body systems involved in the model input feature vector and then calculate the body systems involved in the auxiliary diagnosis results; The disease identification and diagnosis unit, based on the body systems involved in the disease calculated by the disease screening unit, uses the model input feature vector as the model input quantity in the preset multi-system disease identification sub-model to obtain the auxiliary diagnosis result.

4. The AI-based image-assisted diagnosis system according to claim 3, characterized in that: The intelligent diagnosis and treatment decision module includes: A plan-feature driven screening unit is used to build a plan screening adaptation model. Based on the calculated auxiliary diagnosis results, the plan screening adaptation model is used to screen suitable medical plans; the medical plans include: drug treatment plans and surgical treatment plans; A drug regimen calculation unit calculates the amount of each drug required by the patient based on the drug regimen selected by the regimen-feature driven screening unit and a pharmacokinetic adaptive model, and further optimizes the amount of drug required by the patient based on a multi-dimensional risk warning model based on the mutual resistance of each drug, thereby obtaining an auxiliary judgment result of the drug regimen; The surgical plan simulation calculation unit constructs a pathological model based on the model input feature vector. Then, based on the constructed pathological model and the surgical plan selected by the plan-feature driven screening unit, it performs virtual surgery based on the physical engine, simulates the surgical effect, and then provides auxiliary judgment results for the surgical plan. The intelligent diagnosis and treatment judgment unit is used to calculate the treatment plan based on the auxiliary judgment results of the drug plan calculation unit and the auxiliary judgment results of the surgical plan simulation calculation unit, and synchronously combine the patient historical case data stored in the big data management module.

5. The AI-based image-assisted diagnosis system according to claim 4, characterized in that: The intelligent diagnosis and treatment judgment unit also includes: a patient risk management block, which is used to build a dynamic risk adaptive model, import the treatment plan as a model input into the dynamic risk adaptive model, and then calculate the subsequent follow-up time and review plan.

6. An AI image-assisted diagnosis method based on the AI-based image-assisted diagnosis system according to any one of claims 1 to 5, characterized in that: include, Step S1: The regional IoT transmission platform receives the original medical images and patient medical data transmitted by the medical auxiliary diagnosis sub-terminal, classifies them and assigns corresponding serial numbers, binds the transmitted data to the serial numbers, and then uploads them; Step S2: The AI ​​image-assisted diagnosis platform receives data transmitted by the regional IoT transmission platform, removes noise and preprocesses the transmitted data, and then obtains multimodal image data; Step S3: The AI ​​image-assisted diagnosis platform extracts basic features based on the multimodal image data and converts them into model input feature vectors; Step S4: The AI ​​image-assisted diagnosis platform makes a diagnosis and treatment judgment based on the model input feature vector, and then obtains an auxiliary diagnosis result; Step S5: The AI ​​image-assisted diagnosis platform calculates a treatment plan based on the auxiliary diagnosis results and the patient's historical case data; Step S6: The AI ​​image-assisted diagnosis platform transmits the treatment plan to the medical auxiliary diagnosis sub-terminal, so that medical personnel can perform medical diagnosis based on the auxiliary diagnosis results and the treatment plan.

7. The AI ​​image-assisted diagnosis method according to claim 6, characterized in that: The specific implementation process of step S3 is as follows, including: S3.

1. After receiving the pre-processed medical image, a coarse segmentation method is used to sequentially output probability items for the trunk, limbs, and head and neck, and the involved body regions are identified based on the probability items. S3.

2. Based on the body region determined in step S3.1, further locate the body system of the medical image, perform precise organ segmentation based on the corresponding body system feature items, output an organ mask of the corresponding organ on the medical image, and thereby derive the organ features of the medical image; S3.

3. Based on the medical image, anisotropic diffusion tracking is used to obtain a number of vascular seed points in the medical image. Based on the number of vascular seed points, a diffusion tracking algorithm is used to obtain the distribution of vascular branches in the medical image and obtain vascular characteristics. S3.

4. Based on the medical images and the organ features obtained in step S3.2, extract a morphological feature group, a texture feature group, and a functional feature group using a feature extraction method. The extracted morphological feature group, texture feature group, and functional feature group are integrated and a deep learning method is used to output organ-specific features. S3.

5. Based on the acquired organ features, an anatomical graph neural network method is used to construct graph structures, and graph convolution operations are performed to output spatial enhancement features to represent the spatial relationship between organs, thereby deriving organ spatial features; S3.

6. Integrate organ features, vascular features, organ-specific features, and organ spatial features to derive a model input feature vector.

8. The AI ​​image-assisted diagnosis method according to claim 6, characterized in that: The specific implementation process of step S4 is as follows, including: S4.

1. Filter the received model input feature vector for abnormal features and then perform preprocessing to filter out irrelevant factors; S4.

2. Screen and identify the preprocessed model input feature vector, and determine the body system to which the model input feature vector belongs based on the organ features in the model input feature vector; S4.

3. Based on the body system in which the model input feature vector is located, the corresponding system disease analysis sub-model is used, and the multi-system sign joint reasoning method is adopted to identify the disease and obtain the auxiliary diagnosis result.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, the AI ​​image-assisted diagnosis method according to any one of claims 6 to 8 is implemented.

10. A computer program product, characterized in that The computer program product includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions, wherein, when the program instructions are executed by a computer, the computer executes the AI ​​image-assisted diagnosis method according to any one of claims 6 to 8.

Images