Multi-modal medical image fusion system based on deep learning

Abstract

The invention discloses a multi-modal medical image fusion system based on deep learning, and belongs to the technical field of deep learning, and the system comprises a medical image obtaining module which is used for obtaining a multi-modal CT image of a preset part of a preset patient, and carrying out the preprocessing of the multi-modal CT image; the preprocessing comprises the steps of normalizing the multi-modal CT image, removing noise and adjusting the size; and the data acquisition module is used for acquiring the edge contours of the multi-modal CT images, counting the number of the multi-modal CT images, acquiring at least 5-8 different edge contours from different multi-modal CT images, and analyzing the characteristics of the edge contours. According to the invention, through multiple technical means such as multi-modal medical image fusion, edge contour analysis, dynamic monitoring and personalized evaluation, the image fusion quality and the accuracy of disease monitoring and evaluation are improved, misdiagnosis and missed diagnosis are reduced, and help is provided for clinical diagnosis, treatment planning and medical research.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a multimodal medical image fusion system based on deep learning. Background Art

[0002] In recent years, with the continuous advancement of medical technology, breakthroughs have been made in the development of medical imaging technology, which plays a vital role in clinical medicine. However, due to differences in imaging principles, the content contained in images generated by different imaging devices also varies. To fully obtain lesion information, doctors need to read a large number of medical images from various modalities. This enormous workload poses a severe challenge to doctors' mental and physical strength, and misdiagnosis is sometimes inevitable. In response to this situation, multimodal medical imaging (such as CT, MRI, PET, etc.) fusion technology has emerged.

[0003] Among them, the application document with application number CN202310758369.8 provides a multimodal medical image fusion method based on deep learning. The method includes obtaining medical source image data and extracting the medical source image data to obtain medical source image basic data and medical source image detail data; performing fusion processing based on the medical source image basic data to obtain medical source image basic fusion data; and performing data cleaning, desensitization, and consistency checking on the medical source image detail data to obtain medical source image detail consistency data. This method fuses medical image information from different modalities, enriching multimodal image data to a certain extent.

[0004] Another application document with technical application number CN202410767495.4 provides a multimodal medical image quality inspection system based on deep learning. This technical solution uses semantic co-space mapping to map the multimodal image semantic features of the target area into a high-dimensional co-space so that the two can be directly compared and contrasted, thereby realizing intelligent medical image quality inspection.

[0005] However, medical images of different modalities (such as CT, MRI, PET, etc.) have inconsistent data features due to differences in imaging principles, resolution, size, etc., and require complex preprocessing and alignment operations. This increases the difficulty and complexity of data preprocessing, resulting in limited fusion effects and prone to incorrect segmentation or poor fusion effects. This may cause the model to misjudge when dealing with complex medical problems, thereby affecting the treatment of patients. Summary of the Invention

[0006] In view of the above-mentioned problems existing in the existing field of deep learning technology, the present invention is proposed.

[0007] Therefore, one of the objectives of the present invention is to provide a multimodal medical image fusion system based on deep learning, which improves the quality of image fusion, the accuracy of disease monitoring and evaluation, reduces misdiagnosis and missed diagnosis, and provides assistance for clinical diagnosis, treatment planning and medical research through various technical means such as multimodal medical image fusion, edge contour analysis, dynamic monitoring and personalized evaluation.

[0008] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0009] The present invention provides a multimodal medical image fusion system based on deep learning, comprising:

[0010] a medical image acquisition module, configured to acquire multimodal CT images of a preset patient and a preset part, and to preprocess the multimodal CT images; the preprocessing comprising normalizing, removing noise, and resizing the multimodal CT images;

[0011] a data acquisition module, configured to acquire edge contours of the multimodal CT images, count the number of the multimodal CT images, acquire at least 5 to 8 different edge contours from different multimodal CT images, and analyze characteristics of the edge contours;

[0012] a medical image fusion processing unit, the medical image fusion processing unit being responsive to edge contours collected from different multimodal CT images, for acquiring images related to the edge contours and marking the acquired images in the different multimodal CT images; the medical image fusion processing unit comprising a data processing module, an image matching module, a calculation module, and a determination module;

[0013] The data processing module responds to the marking of the image and is used to mark the edge contours in the form of Arabic numerals according to the order in which the edge contours are obtained;

[0014] The image matching module is used to match the marked images with each other. The matching method includes aligning the edges of each image. If the edges of any two images can be aligned, the system determines that the image matching and / or fusion is successful; otherwise, no determination is made.

[0015] The calculation module is responsive to any two images that are successfully matched and / or fused, and is used to calculate relevant data of the images, wherein the relevant data includes the length and width of the images;

[0016] The determination module responds to the calculated relevant data and is used to determine, based on the calculation result, that the condition of the preset patient is deteriorating if the relevant data in the future time period is greater than the calculated result; otherwise, no determination is made; and the calculated result is marked as a reference result;

[0017] As a preferred solution of the present invention, wherein: in the medical image acquisition module, the multimodal CT image is normalized, including removing outliers, and the outlier removal includes cutting background noise and eliminating overly bright values;

[0018] The background noise trimming method includes a region growing method, which starts from a known background seed point in the multimodal CT image and gradually expands to adjacent pixels based on the similarity of pixel values until the entire background area is covered. The operation steps include:

[0019] Selecting a background seed point in the multimodal CT image;

[0020] Set growing conditions;

[0021] Use region growing algorithm to gradually mark the background area;

[0022] Set the pixel values marked as background to 0;

[0023] The method of eliminating overbright values includes setting a threshold based on a histogram, and the steps include:

[0024] Calculate the histogram of the image and obtain the distribution of pixel values;

[0025] A threshold is given according to the histogram, where the threshold is the starting point of the overbright value in the histogram;

[0026] Adjusting pixel values exceeding a threshold to the threshold;

[0027] As a preferred solution of the present invention, wherein: in the judgment module, if the system does not determine that the condition of the preset patient is in a worsening state, multiple object points are preset in the image along the edge of the image, the total number of object points among the multiple object points is counted, and distance calculation is performed based on any two opposite object points, and any two opposite object points are marked as an object group; if the calculation result is greater than the reference result, and the number of object points represented by the calculation result reaches half of the total number, then the system determines that the condition of the preset patient is in a worsening state; otherwise, no judgment is made.

[0028] As a preferred solution of the present invention, according to the judgment result, if the system determines that the condition of the preset patient is in a worsening state, images between two object points in each object group are collected based on at least 3 object groups, and the edge contour of the image is obtained, and the edge contour is marked as a reference edge contour; if the edge contour of the image in the future time period is the same as the reference edge contour, the system determines that the condition of the preset patient is not in a worsening state, otherwise, no judgment is made.

[0029] As a preferred solution of the present invention, according to the judgment result, if the system determines that the condition of the preset patient is not deteriorating, at least two image points are given between two object points in each object group, and a monitoring period is preset for any image point. After the monitoring period, the image changes of the corresponding image points are collected, the image points in which changes have occurred are obtained, and the changes in the condition of the preset patient are evaluated based on the changed image points.

[0030] As a preferred embodiment of the present invention, the changes in the condition of the preset patient are evaluated, and the evaluation method includes obtaining the height difference between the image point and the edge contour of the image during the preset monitoring period, and marking the height difference as a critical value, and obtaining the height difference after the monitoring period. If the obtained height difference is lower than and / or the same as the critical value, the system determines that the condition of the preset patient has not worsened; otherwise, no determination is made.

[0031] As a preferred solution of the present invention, the number of days corresponding to the monitoring period is obtained, and based on the number of days, the monitoring period is divided into three equal monitoring time periods, and based on the height difference obtained after the monitoring period, the change pattern of the height difference within the three monitoring time periods is calculated, and the calculation is obtained according to the following formula:

[0032] Among them, p j represents the p-th height difference obtained for the j-th time within a certain monitoring period;

[0033] Wherein, w represents the minimum height difference obtained in the corresponding monitoring period, r represents the number of times the minimum height difference is exceeded in other monitoring periods, and x represents the average height difference in the monitoring period corresponding to the monitoring period that exceeds the minimum height difference.

[0034] As a preferred solution of the present invention, wherein: until the average height difference is obtained in the monitoring period, the number of height differences obtained in the corresponding monitoring period is counted, and among the number, the change of the average height difference is continued to be obtained based on the last height difference, and the last height difference is marked as the reference height difference. If the average height difference shows an increasing trend, the system determines that the condition of the preset patient is deteriorating; otherwise, no determination is made.

[0035] As a preferred solution of the present invention, wherein: the height difference generated continues to be obtained based on the reference height difference in the corresponding monitoring period until the end of the monitoring period, the total number of height differences generated from the reference height difference to the end of the monitoring period is counted, the maximum height difference is obtained from the number, the proportion of the height difference in the number is calculated, and the proportion is marked as the risk proportion; when the change in the height difference is obtained in the corresponding monitoring period in the future period, based on the number of height differences obtained, if the proportion of the maximum height difference in the number is lower than the risk proportion, the system determines that the condition of the preset patient has not deteriorated; otherwise, no determination is made.

[0036] Beneficial effects:

[0037] 1. The present invention collects the edge contours of multimodal CT images and performs marking and alignment operations on images of different modalities, thereby ensuring the accuracy and consistency of image fusion and reducing artifacts or information loss caused by poor alignment;

[0038] 2. The present invention can dynamically monitor changes in a patient's condition by calculating data such as the length and width of an image and comparing these data changes over future time periods. If the relevant data increases, the system can determine that the condition may worsen, thereby providing doctors with early warnings.

[0039] 3. The present invention combines multiple methods such as edge contour feature analysis, object point distance calculation, and image point change monitoring to comprehensively assess the patient's condition changes from multiple dimensions, thereby improving the accuracy and reliability of the assessment;

[0040] 4. By calculating the risk ratio, the system can quantitatively assess the risk of worsening of the patient's condition and provide judgment support for clinical decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort. Among them:

[0042] Figure 1 Schematic diagram of the modular structure of a multimodal medical image fusion system based on deep learning according to an embodiment of the present invention;

[0043] Figure 2 A schematic diagram of the process structure of an embodiment of the present invention;

[0044] Numbers in the figure: 110 - medical image acquisition module; 120 - data acquisition module; 130 - medical image fusion processing unit; 1301 - data processing module; 1302 - image matching module; 1303 - calculation module; 1304 - determination module. DETAILED DESCRIPTION

[0045] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the described embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of the present invention.

[0046] Due to differences in imaging principles, resolution, size, etc., medical images of different modalities (such as CT, MRI, PET, etc.) have inconsistent data features and require complex preprocessing and alignment operations. This increases the difficulty and complexity of data preprocessing, resulting in limited fusion effects and prone to incorrect segmentation or poor fusion effects. This may cause the model to misjudge when dealing with complex medical problems, thereby affecting the treatment of patients.

[0047] Based on this, the present invention proposes a multimodal medical image fusion system based on deep learning. It comprehensively evaluates the patient's condition changes from multiple dimensions through various technical means such as multimodal image fusion, edge contour analysis, object point distance calculation, image point monitoring, and height difference change rules, significantly improving the image fusion quality, the accuracy of condition monitoring and evaluation, reducing misdiagnosis and missed diagnosis, and improving work efficiency.

[0048] The present invention will be further described in detail below through embodiments and in conjunction with the accompanying drawings.

[0049] Reference Figures 1 to 2 , is an embodiment of the present invention, which provides a multimodal medical image fusion system based on deep learning, including:

[0050] The medical image acquisition module 110 is used to acquire multimodal CT images of a preset patient and a preset part, and preprocess the multimodal CT images; the preprocessing includes normalizing, removing noise, and resizing the multimodal CT images;

[0051] By performing preprocessing operations such as normalization, noise removal, and size adjustment on multimodal CT images, the quality and consistency of the images can be improved, providing a better foundation for subsequent fusion processing;

[0052] Normalization operation can unify image data collected from different sources and different devices into a standardized range, which is convenient for subsequent analysis and processing;

[0053] The data acquisition module 120 is used to acquire edge contours of multimodal CT images, count the number of multimodal CT images, acquire at least 5 to 8 different edge contours in different multimodal CT images, and analyze the characteristics of the edge contours;

[0054] By collecting the edge contours of multimodal CT images and marking and aligning the images of different modalities, the advantages of different modalities can be effectively integrated to generate high-quality fused images.

[0055] In this embodiment, obtaining edge contours of CT images can assist in disease diagnosis and analysis, including assisting in lesion detection, differentiating between benign and malignant lesions, and monitoring disease progression. In assisting lesion detection, edge contours can help doctors more clearly identify the boundaries of lesion areas. For example, in tumor diagnosis, edge contours can clearly define the boundaries of the tumor and help doctors determine the extent and nature of the lesion.

[0056] In differentiating benign from malignant lesions, malignant tumors usually have irregular margins, while benign lesions have relatively smooth margins. Analyzing the shape and complexity of the margin contours can assist in differentiating between benign and malignant lesions.

[0057] In disease progression monitoring, by comparing the edge contours of CT images at different stages, the progression of the disease or the effect of treatment can be evaluated. For example, in pulmonary fibrosis or chronic obstructive pulmonary disease (COPD), changes in the edge contour can reflect the degree of fibrosis in the lung tissue.

[0058] At the same time, edge contour acquisition can also be used for surgical planning and navigation, as well as provide precise organ boundary information. For example, during surgery, edge contours can provide the surgical team with accurate organ boundary information and contour positioning, thereby providing more precise guidance for surgical operations, which helps reduce surgical risks and improve surgical success rates. Accurate edge contours are the basis for generating precise 3D models, which facilitates surgical planning and navigation.

[0059] Therefore, this embodiment obtains edge contours and performs feature analysis, which has practical significance;

[0060] The medical image fusion processing unit 130 is responsive to edge contours collected from different multimodal CT images, is used to obtain images related to the edge contours, and to mark the obtained images in the different multimodal CT images. The medical image fusion processing unit 130 includes a data processing module 1301, an image matching module 1302, a calculation module 1303, and a determination module 1304.

[0061] The data processing module 1301 responds to the marking of the image and is used to mark the image in Arabic numerals according to the order in which the edge contours are obtained;

[0062] The image matching module 1302 is used to match the marked images with each other. The matching method includes aligning the edges of each image. If the edges of any two images can be aligned, the system determines that the image matching and / or fusion is successful; otherwise, no determination is made.

[0063] In this embodiment, in CT images, the edges of different images are aligned, which is of reference significance for the diagnosis of the disease, including improving diagnostic accuracy, assisting in disease progression monitoring, and supporting multimodal image fusion;

[0064] To improve diagnostic accuracy, it can accurately locate lesions, enhance lesion identification, and reduce misdiagnosis and missed diagnosis. In precise lesion localization, edge alignment can ensure the consistency of the spatial position of the lesion area in images at different time points or in different modalities. For example, in contrast-enhanced CT and non-enhanced CT images, edge alignment can more accurately locate the position and extent of the tumor.

[0065] To assist in monitoring disease progression, images at different time points can be compared and lesion changes can be quantified. For example, during long-term follow-up of patients, edge alignment can help doctors more intuitively compare images from different periods and assess lesion progression or treatment efficacy. Furthermore, the aligned edges can be used to calculate parameters such as the area and volume of the lesion area, providing a basis for quantitative disease analysis.

[0066] It can also support multimodal image fusion, including integrating information from different modalities and improving the quality of fused images. In integrating information from different modalities, edge alignment is one of the key steps in multimodal image fusion. By aligning images from different modalities (such as CT and MRI), the advantages of different modalities can be integrated to provide more comprehensive diagnostic information.

[0067] Furthermore, the aligned edges can ensure the accuracy and consistency of the fused image, avoiding artifacts or information loss caused by poor alignment;

[0068] Therefore, this embodiment aligns the edges of the images, which has practical significance.

[0069] The calculation module 1303 is responsive to any two images that are successfully matched and / or fused, and is used to calculate image-related data, including the length and width of the images;

[0070] In this embodiment, by calculating data such as the length and width of the image and comparing the data changes in future time periods, it is possible to dynamically monitor the patient's condition changes and provide early warnings to doctors;

[0071] The determination module 1304 responds to the calculated relevant data and is used to determine, based on the calculation result, that the condition of the preset patient is deteriorating if the relevant data in the future period is greater than the calculated result; otherwise, no determination is made and the calculated result is marked as a reference result;

[0072] In this embodiment, if the length and width of a CT image of a certain part of the body increase in size, it indicates a change in the condition. For example, in the case of inflammatory lesions, some inflammatory diseases may cause swelling and volume increase of local tissues. In CT images of pneumonia, the scope of the lesion area will expand as the inflammation progresses. Therefore, the determination method of this embodiment has practical significance.

[0073] Of course, in other feasible solutions, the progression of the disease cannot be determined solely based on this. Instead, a comprehensive assessment must be conducted based on the patient's clinical symptoms, contrast-enhanced scan results, dynamic monitoring, and other information. For example, certain non-pathological factors may also cause changes in the size of local areas on the image, including changes in the patient's body position and respiratory movement, which may affect the image display.

[0074] In the medical image acquisition module, the multimodal CT images are normalized, including removing outliers. Outlier removal includes clipping background noise and eliminating overly bright values.

[0075] The background noise trimming method includes a region growing method. In a multimodal CT image, starting from a known background seed point, the method gradually expands to adjacent pixels based on the similarity of pixel values until the entire background area is covered. The steps include:

[0076] Selecting a background seed point in the multimodal CT image. It should be noted that the background seed point includes the corners of the CT image;

[0077] Setting growth conditions. It should be noted that the set growth conditions include a pixel value range;

[0078] Use region growing algorithm to gradually mark the background area;

[0079] Set the pixel values marked as background to 0;

[0080] In this embodiment, in CT images, background noise clipping is one of the important steps in preprocessing, the purpose of which is to remove irrelevant background areas, reduce noise interference, and highlight the target area;

[0081] Removing background noise through the region growing algorithm can effectively reduce the interference of irrelevant information, highlight the region of interest, and improve the signal-to-noise ratio of the image;

[0082] Region growing is the process of developing groups of pixels or regions into larger regions, starting from a set of seed points, and growing the region from these points by merging adjacent pixels with similar properties to each seed point, such as intensity, grayscale, texture color, etc.

[0083] Eliminating overly bright values, including setting a threshold based on a histogram, involves the following steps:

[0084] Calculate the histogram of the image and obtain the distribution of pixel values;

[0085] A threshold is given according to the histogram, and the threshold is the starting point of the overbright value in the histogram;

[0086] Adjust the pixel values exceeding the threshold to the threshold;

[0087] In this embodiment, a histogram is a quality distribution graph, also a statistical report graph, which is composed of a series of vertical stripes or line segments of varying heights to represent the data distribution. Generally, the horizontal axis represents the data type, and the vertical axis represents the distribution.

[0088] The histogram-based method gives a threshold value, which can effectively eliminate overly bright values, avoid the influence of highlight areas on image analysis, and improve the quality and usability of images.

[0089] At the same time, in CT images, eliminating overbright values is one of the important steps in preprocessing. The purpose is to remove abnormally bright areas caused by metal implants, high-density tissues or artifacts, so as to improve image quality and the accuracy of subsequent analysis;

[0090] In the determination module, if the system does not determine that the condition of the preset patient is worsening, a plurality of object points are preset along the edge of the image, the total number of object points in the plurality of object points is counted, a distance calculation is performed based on any two opposing object points, and the any two opposing object points are marked as an object group; if the calculated result is greater than the reference result, and the number of object points represented by the calculated result reaches half of the total number, then the system determines that the condition of the preset patient is worsening; otherwise, no determination is made;

[0091] In this embodiment, a combination of edge contour feature analysis, object point distance calculation, and image point change monitoring is used to comprehensively assess the patient's condition changes from multiple dimensions, thereby improving the accuracy and reliability of the assessment.

[0092] When the change in edge contour is not sufficient to determine the worsening of the disease, presetting multiple object points on the edge of the image and performing distance calculations provides a supplementary evaluation dimension, enhancing the comprehensiveness of the disease assessment.

[0093] By counting the total number of object points and calculating the distance between opposing object points, the changes in the condition can be quantified, providing doctors with a more specific and intuitive basis for disease assessment;

[0094] It should be noted that in this embodiment, the plurality of object points is at least 10 or more;

[0095] On the basis of the above, according to the determination result, if the system determines that the condition of the preset patient is worsening, then based on at least three object groups, an image between two object points in each object group is collected, and the edge contour of the image is obtained, and the edge contour is marked as a reference edge contour; if the edge contour of the image in the future time period is the same as the reference edge contour, then the system determines that the condition of the preset patient is not worsening; otherwise, no determination is made;

[0096] By collecting images between two object points in an object group and obtaining edge contours, these edge contours are marked as reference edge contours, providing an accurate reference benchmark for future image comparison;

[0097] If the edge contour of the image in the future period changes from the reference edge contour, it may indicate that the condition is worsening, thus providing early warning and helping to adjust the treatment plan in a timely manner;

[0098] In this embodiment, in CT images acquired in the future, by extracting radiomic features of edge contours (such as texture features, shape / contour features, etc.), it is possible to more comprehensively assess the characteristic changes of the lesion, thereby helping to predict future changes in the disease;

[0099] Furthermore, in this embodiment, if the system determines that the condition of the predetermined patient has not worsened based on the determination result, at least two image points are assigned between two object points in each object group, and a monitoring period is set for any image point. After the monitoring period, image changes of the corresponding image point are collected, and image points that have changed are obtained. The change in the condition of the predetermined patient is evaluated based on the changed image points.

[0100] In this embodiment, image points are given between two object points in each object group, and monitoring periods are set and changes are collected for these image points. This can monitor subtle changes in local areas and help to discover potential changes in the disease earlier.

[0101] By evaluating the changes in image points, the judgment of the patient's condition can be dynamically adjusted according to the changes, improving the flexibility and accuracy of disease monitoring;

[0102] It should be emphasized that in this embodiment, the assessment of the change in the condition of the predetermined patient includes obtaining a height difference between an image point and an edge contour of the image during a predetermined monitoring period, marking the height difference as a critical value, and obtaining the height difference again after the monitoring period. If the obtained height difference is lower than and / or equal to the critical value, the system determines that the condition of the predetermined patient has not worsened; otherwise, no determination is made.

[0103] This embodiment obtains the height difference between the image point and the edge contour and marks the height difference as a critical value, thereby providing a quantitative evaluation indicator for the change of the disease condition;

[0104] If the height difference obtained after the monitoring period exceeds the critical value, the condition is considered to have worsened. This clear standard helps reduce errors in subjective judgment and improves the objectivity and accuracy of condition assessment;

[0105] In this embodiment, research on medical images indicates that if the height difference between a certain image point and the edge contour gradually increases, it may indicate that the area corresponding to the image point is growing or invading surrounding tissue. For example, in some fibrotic or proliferative lesions, in chronic diseases such as pulmonary fibrosis, the boundaries of the fibrotic area may gradually become clearer, and the change in height difference may reflect the expansion of the fibrotic area. Therefore, the determination method of this embodiment has practical significance.

[0106] Furthermore, this embodiment obtains the number of days corresponding to the monitoring period, divides the monitoring period into three equal monitoring periods based on the number of days, and calculates the change pattern of the height difference within the three monitoring periods based on the height difference obtained after the monitoring period, and calculates it according to the following formula:

[0107] Among them, p j represents the pth height difference obtained for the jth time within a certain monitoring period;

[0108] Where w represents the minimum height difference obtained in the corresponding monitoring period, r represents the number of times the minimum height difference is exceeded in other monitoring periods, and x represents the average height difference in the monitoring period corresponding to the monitoring period that exceeds the minimum height difference;

[0109] This embodiment can analyze the trend of disease changes by calculating the change pattern of the height difference during the monitoring period, rather than just the changes at a single time point, which helps to more comprehensively understand the development direction of the disease.

[0110] At the same time, the changes in height difference during different monitoring periods are taken into account, which can cope with the complexity of disease changes and improve the system's adaptability to different disease change patterns;

[0111] It should be noted that in this embodiment, until the average height difference is obtained in the monitoring period, the number of height differences obtained in the corresponding monitoring period is counted, and the change in the average height difference is continuously obtained based on the last height difference. The last height difference is marked as the reference height difference. If the average height difference shows an increasing trend, the system determines that the condition of the preset patient is worsening; otherwise, no determination is made.

[0112] In this embodiment, the height difference within the monitoring period is continuously obtained until the end of the monitoring period, which can realize dynamic monitoring of the changes in the condition and capture any changes in the condition in a timely manner;

[0113] By analyzing the changing trend of the average height difference, if the average height difference shows an increasing trend, it is determined that the condition is worsening. This trend-based judgment method can more accurately reflect the dynamic changes of the condition;

[0114] On the basis of the above, the height differences generated are continuously acquired based on the reference height difference during the corresponding monitoring period until the end of the monitoring period, the total number of height differences generated from the reference height difference to the end of the monitoring period is counted, the largest height difference is acquired from the number, the proportion of the height differences in the number is calculated, and the proportion is marked as the risk proportion; when the change in the height difference is acquired during the corresponding monitoring period in the future, based on the number of acquired height differences, if the proportion of the largest height difference in the number is lower than the risk proportion, then the system determines that the condition of the preset patient has not deteriorated; otherwise, no determination is made;

[0115] In this embodiment, by calculating the proportion of height differences within a certain period and marking it as the risk proportion, a quantitative assessment of the risk of disease worsening is achieved;

[0116] In future monitoring periods, by comparing the new height difference with the risk ratio, future changes in the disease can be predicted, providing stronger support for clinical decision-making.

[0117] In summary, the present invention comprehensively evaluates the patient's condition changes from multiple dimensions through various technical means such as multimodal image fusion, edge contour analysis, object point distance calculation, image point monitoring, and height difference change rules, significantly improving the quality of image fusion, the accuracy of condition monitoring and evaluation, reducing misdiagnosis and missed diagnosis, and improving work efficiency.

[0118] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A multimodal medical image fusion system based on deep learning, characterized by: include: a medical image acquisition module, configured to acquire multimodal CT images of a preset patient and a preset part, and to preprocess the multimodal CT images; the preprocessing comprising normalizing, removing noise, and resizing the multimodal CT images; a data acquisition module, configured to acquire edge contours of the multimodal CT images, count the number of the multimodal CT images, acquire at least 5 to 8 different edge contours from different multimodal CT images, and analyze characteristics of the edge contours; a medical image fusion processing unit, the medical image fusion processing unit being responsive to edge contours collected from different multimodal CT images, for acquiring images related to the edge contours and marking the acquired images in the different multimodal CT images; the medical image fusion processing unit comprising a data processing module, an image matching module, a calculation module, and a determination module; The data processing module responds to the marking of the image and is used to mark the edge contours in the form of Arabic numerals according to the order in which the edge contours are obtained; The image matching module is used to match the marked images with each other. The matching method includes aligning the edges of each image. If the edges of any two images can be aligned, the system determines that the image matching and / or fusion is successful. Otherwise, no judgment is made; The calculation module is responsive to any two images that are successfully matched and / or fused, and is used to calculate relevant data of the images, wherein the relevant data includes the length and width of the images; The determination module responds to the calculated relevant data and is used to determine, based on the calculation result, that the condition of the preset patient is deteriorating if the relevant data in the future period is greater than the calculation result; Otherwise, no determination is made; and the calculation result is marked as a reference result.

2. The multimodal medical image fusion system based on deep learning according to claim 1, characterized in that: In the medical image acquisition module, the multimodal CT image is normalized, including removing outliers, wherein the outlier removal includes clipping background noise and eliminating overly bright values; The background noise trimming method includes a region growing method, which starts from a known background seed point in the multimodal CT image and gradually expands to adjacent pixels based on the similarity of pixel values until the entire background area is covered. The operation steps include: Selecting a background seed point in the multimodal CT image; Set growing conditions; Use region growing algorithm to gradually mark the background area; Set the pixel values marked as background to 0; The method of eliminating overbright values includes setting a threshold based on a histogram, and the steps include: Calculate the histogram of the image and obtain the distribution of pixel values; A threshold is given according to the histogram, where the threshold is the starting point of the overbright value in the histogram; Pixel values exceeding a threshold are adjusted to the threshold.

3. The multimodal medical image fusion system based on deep learning according to claim 1, characterized in that: In the determination module, if the system does not determine that the condition of the predetermined patient is worsening, a plurality of object points are preset along an edge of the image, a total number of object points is counted among the plurality of object points, a distance is calculated based on any two opposing object points, and the any two opposing object points are marked as an object group; If the calculated result is greater than the reference result, and the number of object points represented by the calculated result reaches half of the total number, the system determines that the condition of the preset patient is worsening; otherwise, no determination is made.

4. The multimodal medical image fusion system based on deep learning according to claim 3, characterized in that: According to the judgment result, if the system determines that the condition of the preset patient is in a worsening state, the image between two object points in each object group is collected based on at least 3 object groups, and the edge contour of the image is obtained, and the edge contour is marked as the reference edge contour; if the edge contour of the image in the future time period is the same as the reference edge contour, the system determines that the condition of the preset patient is not in a worsening state, otherwise, no judgment is made.

5. The multimodal medical image fusion system based on deep learning according to claim 4, characterized in that: According to the judgment result, if the system determines that the condition of the preset patient is not worsening, at least two image points are given between the two object points in each object group, and a monitoring period is preset for any image point. After the monitoring period, the image changes of the corresponding image points are collected, the image points that have changed are obtained, and the changes in the condition of the preset patient are evaluated based on the changed image points.

6. The multimodal medical image fusion system based on deep learning according to claim 5, characterized in that: evaluating changes in the condition of the predetermined patient, wherein the evaluation method includes obtaining a height difference between the image point and an edge contour of the image during a predetermined monitoring period, marking the height difference as a critical value, obtaining the height difference again after the monitoring period, and if the obtained height difference is lower than and / or the same as the critical value, the system determines that the condition of the predetermined patient has not worsened; Otherwise, no judgment is made.

7. The multimodal medical image fusion system based on deep learning according to claim 6, characterized in that: Obtain the number of days corresponding to the monitoring period, divide the monitoring period into three equal monitoring periods based on the number of days, and calculate the change pattern of the height difference within the three monitoring periods based on the height difference obtained after the monitoring period, and calculate it according to the following formula: Among them, p j represents the p-th height difference obtained for the j-th time within a certain monitoring period; Wherein, w represents the minimum height difference obtained in the corresponding monitoring period, r represents the number of times the minimum height difference is exceeded in other monitoring periods, and x represents the average height difference in the monitoring period corresponding to the monitoring period that exceeds the minimum height difference.

8. The multimodal medical image fusion system based on deep learning according to claim 7, characterized in that: Until the average height difference is obtained in the monitoring period, the number of height differences obtained in the corresponding monitoring period is counted, and among the number, the change of the average height difference is continued to be obtained based on the last height difference, and the last height difference is marked as the reference height difference. If the average height difference shows an increasing trend, the system determines that the condition of the preset patient is worsening; otherwise, no determination is made.

9. The multimodal medical image fusion system based on deep learning according to claim 8, characterized in that: Continue to obtain the generated height differences based on the reference height difference during the corresponding monitoring period until the end of the monitoring period, count the total number of height differences generated from the reference height difference to the end of the monitoring period, obtain the largest height difference among the number, calculate the proportion of the height difference in the number, and mark the proportion as the risk proportion; when obtaining the change in height difference during the corresponding monitoring period in the future, based on the number of height differences obtained, if the proportion of the largest height difference in the number is lower than the risk proportion, the system determines that the condition of the preset patient has not deteriorated; otherwise, no determination is made.

Images