A medical image intelligent processing method and system based on deep learning

Abstract

The present invention discloses a method and system for intelligent medical image processing based on deep learning, which relates to the field of image processing technology. The method comprises collecting and preprocessing multimodal medical image data, registering medical images of different modalities, fusing the registered multimodal images and optimizing them; segmenting the optimized fused images into image blocks and assigning a significance score to each block, marking candidate lesion areas based on the significance scores and generating regional masks; and performing time series change analysis on the regional masks to generate multi-time point lesion area heat maps and perform visual display. By combining deep learning with traditional registration and fusion technologies, the present invention can significantly improve registration accuracy, optimize fusion effects, and realize visual analysis of lesion changes at multiple time points through dynamic lesion heat map generation technology, providing accurate and dynamic support for the diagnosis and treatment of complex diseases.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method and system for intelligent medical image processing based on deep learning. Background Art

[0002] In recent years, with the rapid development of medical imaging technology, multimodal imaging techniques such as CT, MRI, and PET have been widely used in clinical diagnosis and disease detection. Each of these techniques has unique advantages. For example, CT can provide precise anatomical information, MRI can highlight soft tissue details, and PET can reveal the metabolic activity of lesions. Multimodal medical imaging combines the advantages of different imaging techniques, providing richer information for comprehensive diagnosis of complex diseases. However, due to differences in image sources and physical mechanisms, these multimodal images often exhibit significant differences in spatial resolution, contrast, and dynamic range, making accurate registration and fusion between images technically challenging. Traditional multimodal image processing methods mainly rely on algorithms based on rules or statistical models, such as registration methods based on maximizing mutual information and linear fusion techniques. However, these methods are generally limited to processing static features and have limited ability to describe the spatiotemporal characteristics of complex dynamic lesions. In addition, deep learning technology has made significant progress in medical image processing in recent years. Techniques such as convolutional neural networks (CNNs) and generative adversarial networks (GANs) have been widely applied to image enhancement, segmentation, and classification tasks. However, most of these technologies focus on the processing of single-modal images, and how to fuse, analyze and dynamically monitor multi-modal image data remains an unresolved problem. Summary of the Invention

[0003] In view of the problems existing in the above-mentioned existing medical image intelligent processing methods and systems based on deep learning, the present invention is proposed.

[0004] Therefore, the problem to be solved by the present invention is that most of these technologies focus on the processing of single-modality images, and how to fuse, analyze and dynamically monitor multi-modality image data is still an unresolved problem.

[0005] To solve the above technical problems, the present invention provides the following technical solutions: a medical image intelligent processing method based on deep learning, which includes collecting multimodal medical image data and preprocessing it, registering medical images of different modalities, fusing the registered multimodal images and optimizing them;

[0006] The optimized fused image is segmented into image blocks and a significance score is assigned to each block. The candidate lesion area is marked based on the significance score and a region mask is generated.

[0007] Time series change analysis of the regional mask was performed to generate a heat map of the lesion area at multiple time points and visualize it.

[0008] As a preferred solution of the medical image intelligent processing method based on deep learning described in the present invention, wherein: collecting multimodal medical image data and performing preprocessing, and registering medical images of different modalities refers to using medical imaging equipment to collect multimodal image data, adding a timestamp to each frame of the image, and normalizing the image pixel values ​​to adjust the image resolution to a uniform scale;

[0009] The multimodal imaging data includes CT, MRI and PET;

[0010] Select the CT modality as the reference image and the other modalities as the target images. Divide the pixel values ​​of the reference image and the target image into K intervals, and count the joint occurrence frequencies of the corresponding pixel values ​​of the reference image and the target image to construct a joint histogram.

[0011] Normalize the joint histogram into a joint probability distribution, calculate the marginal probability distribution of the reference image and the target image respectively, and calculate the mutual information F based on the joint probability distribution and the marginal probability distribution;

[0012] Define the optimization goal as maximizing mutual information, initialize the rotation parameters and translation parameters, construct the initial transformation matrix based on the initialization parameters, apply the initial transformation matrix to the target image, generate the transformed target image, and calculate the mutual information based on the pixel value distribution of the target image and the reference image;

[0013] Use the grid search method to update the transformation matrix parameters, and use the parameter combination with the maximum mutual information as the candidate for the next step. Iteratively optimize the parameter combination until the maximum number of iterations is reached. Based on the iteratively optimized parameter combination, the optimized optimal transformation matrix is ​​obtained.

[0014] The target image transformed by the optimal transformation matrix is ​​used as the rough registration result;

[0015] Use edge detection algorithm to extract the edges of reference image and target image, calculate the centroid of edge pixels of reference image and target image and select evenly distributed points to form reference point set and target point set;

[0016] Calculate the centroid of the reference point set and the target point set, decentralize the point set, use the decentralized point set to construct the covariance matrix, perform singular value decomposition on the covariance matrix and calculate the rotation matrix, and calculate the translation vector based on the centroid of the rotated point set;

[0017] Apply rotation and translation transformations to each point in the target point set to obtain an updated target point set;

[0018] Calculate the mean square error between the updated target point set and the reference point set, update the rotation matrix and translation vector through multiple iterations, set the maximum number of iterations, and stop the iteration when the maximum number of iterations is reached to obtain the final rotation matrix and translation vector;

[0019] The final rotation matrix and translation vector are used to fully align the target image to the reference image, generating the final registration result.

[0020] As a preferred solution of the deep learning-based intelligent processing method for medical images described in the present invention, fusing the registered multimodal images and performing optimization refers to extracting multimodal image pairs from the registered multimodal images, representing modality I1 and modality I2 respectively, dividing the image into image patches of size p*p using a sliding window method, extracting all patches and expanding them into vector form to form a feature matrix:

[0021]

[0022] Where X1 and X2 represent the patch feature matrices of modalities I1 and I2, respectively, and M is the total number of image patches. Dictionary matrices D1 and D2 and sparse representation matrices A1 and A2 are initialized for each modality.

[0023] The optimization objective function is defined as:

[0024]

[0025] Where, ‖·‖ F is the Frobenius norm;

[0026] The constraints that define the objective function are:

[0027] supp(A1)=supp(A2),

[0028] Where supp() represents the support set of the sparse matrix;

[0029] For each modality, fix D1 and D2, update A1 and A2, and solve the sparse representation matrix to obtain the updated sparse representation matrix A′ i ;

[0030] Fix A1 and A2, optimize D1 and D2, and update the dictionary matrix to obtain D′ i ;

[0031] After each update, the reconstruction error is calculated and the error threshold S is set. If the error is less than the threshold S, the iteration is stopped.

[0032] Based on the optimized dictionary and sparse representation, calculate the correlation features shared between modalities:

[0033] Z1=D1A1,Z2=D2A2,

[0034] Where Z1 and Z2 are correlation feature matrices;

[0035] Remove the correlation features from the original features to obtain the independence features:

[0036] E1=X1-Z1,E2=X2-Z2,

[0037] Where E1 and E2 are independence feature matrices, X1 and X2 represent the patch feature matrices of modalities I1 and I2 respectively;

[0038] Calculate the information entropy H(Z i );

[0039] Dynamically assign weights b1 and b2 based on information entropy;

[0040] According to the weight, Z1 and Z2 are fused to generate the low-frequency component F l (x,y);

[0041] The maximum selection rule is used to fuse the independent features, retain the significant eigenvalues ​​of each mode, and generate the fused high-frequency component F h (x,y);

[0042] Combine the fused low-frequency component and high-frequency component to generate the preliminary fused image F f (x,y) and normalize the fused image;

[0043] The global contrast intensity of the image is calculated by the grayscale histogram and the clarity is measured by the mean of the image gradient intensity. The quality evaluation index R of the fused image is constructed based on the contrast W and clarity X as the optimization target.

[0044] Use the Logistic Map model to initialize the contrast enhancement coefficient and noise suppression coefficient;

[0045] Map the current parameter value output by the Logistic Map model to the actual range of the optimized parameter;

[0046] The preliminary fusion image is dynamically optimized through pulse coupled neural network.

[0047] As a preferred solution of the medical image intelligent processing method based on deep learning described in the present invention, wherein: performing image block segmentation on the optimized fused image and assigning a significance score to each block refers to using a sliding window method to divide the optimized fused image into non-overlapping image blocks, performing linear interpolation on the edge area of ​​the image, and calculating the local mean of the image blocks. and local gradient

[0048] The saliency score e of each image patch is calculated based on the local mean and local gradient:

[0049]

[0050] Where N is the image block B j The number of pixels, is the image block B j The gradient of the jth pixel in , is the image block B j The local mean, μ max is the maximum local mean of all blocks.

[0051] As a preferred solution of the deep learning-based intelligent processing method for medical images described in the present invention, marking candidate lesion areas based on significance scores and generating regional masks refers to setting a significance threshold U, marking image blocks with significance scores higher than the threshold as significant, combining all blocks with significance scores higher than the threshold into candidate lesion areas, generating a lesion area mask based on the significance score and the set threshold U, and marking the candidate lesion areas.

[0052] As a preferred solution of the deep learning-based intelligent medical image processing method of the present invention, wherein: performing time series change analysis on the region mask to generate a multi-time point lesion region heat map refers to obtaining candidate lesion region mask data at multiple time points, extracting the lesion volume corresponding to each time point, and calculating the lesion volume by accumulating the total number of pixels in the lesion region;

[0053] Calculate the volume change rate G of the lesion area at each moment t t ;

[0054] Set a threshold m and select areas with a change rate greater than m as lesion areas with significant changes;

[0055] Initialize the heat map matrix with the same size as the image, initialize all pixel values ​​to 0, traverse all time points, gradually superimpose the significantly changed areas onto the heat map to generate a multi-time point lesion area heat map, and normalize the generated heat map.

[0056] As a preferred solution of the deep learning-based intelligent processing method for medical images described in the present invention, the visual display refers to color mapping the heat map, weighted superposition of the heat map and the original image, displaying the heat map area on the original image, marking and highlighting the heat map area, and saving the generated heat map image as an image file and storing it in a database.

[0057] Another object of the present invention is to provide a medical image intelligent processing system based on deep learning, which includes:

[0058] An acquisition module, used to acquire multimodal images using medical imaging equipment and perform preprocessing;

[0059] An image registration module is used to perform coarse registration and precise registration on the pre-processed images to obtain registered medical images;

[0060] The fusion optimization module is used to perform multimodal fusion and optimization on the registered images to obtain high-quality fused images;

[0061] The region marking module is used to calculate the significance score of each image block after image segmentation of the fused image, mark the candidate lesion area according to the significance score, and generate the region mask corresponding to the candidate lesion area;

[0062] An analysis module, used to perform time series analysis based on the mask and generate a heat map of the lesion area at multiple time points;

[0063] The storage module is displayed, which is used to perform color mapping on the generated heat map, perform weighted overlay with the original image, and store the heat map image in the database.

[0064] A computer device comprises: a memory and a processor; the memory stores a computer program, and the processor implements the steps of a medical image intelligent processing method based on deep learning when executing the computer program.

[0065] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a medical image intelligent processing method based on deep learning.

[0066] The beneficial effects of the present invention are as follows: by combining deep learning with traditional registration and fusion technologies, the present invention can significantly improve the registration accuracy and optimize the fusion effect, and realize the visualization analysis of lesion changes at multiple time points through dynamic lesion heat map generation technology, providing accurate and dynamic support for the diagnosis and treatment of complex diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0068] Figure 1 This is a flowchart of the medical image intelligent processing method based on deep learning.

[0069] Figure 2 Schematic diagram of the structure of the medical image intelligent processing system based on deep learning. DETAILED DESCRIPTION

[0070] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0071] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0072] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it constitute a separate or selective embodiment that is mutually exclusive with other embodiments.

[0073] Example 1, reference Figure 1 , which is the first embodiment of the present invention, provides a medical image intelligent processing method based on deep learning. The medical image intelligent processing method based on deep learning includes:

[0074] S1. Collect and preprocess multimodal medical imaging data, register medical images of different modalities, fuse and optimize the registered multimodal images;

[0075] Specifically, collecting multimodal medical imaging data and preprocessing it, and registering medical images of different modalities means using medical imaging equipment to collect multimodal imaging data, adding a timestamp to each frame of the image, and normalizing the image pixel values ​​to adjust the image resolution to a uniform scale;

[0076] Through standardization processing (such as pixel value normalization and resolution unification), the dynamic range differences of images produced by different devices are eliminated to ensure the consistency of input data. Timestamp information is added to the images, laying the foundation for subsequent time series analysis.

[0077] The multimodal imaging data includes CT, MRI and PET;

[0078] Select the CT modality as the reference image and the other modalities as the target images. Divide the pixel values ​​of the reference image and the target image into K intervals, and count the joint occurrence frequencies of the corresponding pixel values ​​of the reference image and the target image to construct a joint histogram.

[0079] Normalize the joint histogram to a joint probability distribution, calculate the marginal probability distribution of the reference image and the target image respectively, and calculate the mutual information F based on the joint probability distribution and the marginal probability distribution:

[0080]

[0081] Where P(z,c) is the joint probability distribution, which represents the probability that pixel values ​​z and c appear simultaneously in the reference image and the target image. P(z) is the edge probability of the grayscale value z in the reference image, and P(c) is the edge probability of the grayscale value c in the target image.

[0082] The joint histogram captures the global grayscale relationship between multimodal images, providing an intuitive statistical basis for the calculation of mutual information. Mutual information can effectively address nonlinear differences between different modalities by quantifying the pixel value dependencies between images.

[0083] Define the optimization goal as maximizing mutual information, initialize the rotation parameters and translation parameters, construct the initial transformation matrix based on the initialization parameters, apply the initial transformation matrix to the target image, generate the transformed target image, and calculate the mutual information based on the pixel value distribution of the target image and the reference image;

[0084] Use the grid search method to update the transformation matrix parameters, and use the parameter combination with the maximum mutual information as the candidate for the next step. Iteratively optimize the parameter combination until the maximum number of iterations is reached. Based on the iteratively optimized parameter combination, the optimized optimal transformation matrix is ​​obtained.

[0085] The target image transformed by the optimal transformation matrix is ​​used as the rough registration result;

[0086] The estimation of the initial transformation matrix directly affects the accuracy of the registration results. Global parameter optimization is performed through the grid search method to ensure that a high initial accuracy can be achieved in the coarse registration stage. Combined with the mutual information evaluation function, the grid search can significantly improve the efficiency of the initial alignment of the inter-modal structure.

[0087] Use edge detection algorithm to extract the edges of reference image and target image, calculate the centroid of edge pixels of reference image and target image and select evenly distributed points to form reference point set and target point set;

[0088] Calculate the centroid of the reference point set and the target point set, decentralize the point set, use the decentralized point set to construct the covariance matrix, perform singular value decomposition on the covariance matrix and calculate the rotation matrix, and calculate the translation vector based on the centroid of the rotated point set;

[0089] The reference and target point sets constructed using edge detection and centroid extraction algorithms significantly reduce complexity, and the extracted point sets have high geometric consistency. Singular value decomposition accurately captures the main directions in the covariance matrix and can efficiently calculate the rotation matrix. Combined with the centroid translation vector calculation, precise alignment of the target image is achieved.

[0090] Apply rotation and translation transformations to each point in the target point set to obtain an updated target point set;

[0091] The mean square error (MSE) between the updated target point set and the reference point set is calculated. The mean square error (MSE) is used as the evaluation indicator for iterative optimization to ensure the robustness and accuracy of the registration result. The rotation matrix and translation vector are updated through multiple iterations to reduce the registration error. The maximum number of iterations is set. When the maximum number of iterations is reached, the iteration is stopped to obtain the final rotation matrix and translation vector. The rotation matrix and translation vector are optimized through multiple iterations to reduce the error between the target point set and the reference point set in each round of calculation. By setting the maximum number of iterations and the error threshold, the computational cost and accuracy requirements can be flexibly controlled.

[0092] The final rotation matrix and translation vector are used to fully align the target image to the reference image to generate the final registration result. The combination of coarse registration and fine registration fully utilizes global and local features to achieve high-precision alignment of the target image to the reference image. The final registration result provides high-quality input data for subsequent image fusion and lesion significance analysis.

[0093] Existing medical image processing technologies have achieved some success in multimodal image registration, fusion, and lesion detection, but they still have limitations. This paper overcomes the shortcomings of existing technologies in terms of accuracy, efficiency, and robustness by introducing a registration algorithm that maximizes mutual information, a fusion optimization method based on sparse representation, and a lesion detection technique based on significance scores. In particular, it provides a more efficient and accurate solution for multimodal image fusion and lesion area analysis.

[0094] Furthermore, fusing the registered multimodal images and optimizing them refers to extracting multimodal image pairs from the registered multimodal images, representing modality I1 and modality I2 respectively, and using the sliding window method to divide the image into image patches of size p*p. The sliding window method ensures that the local information of the image is fully captured. Each patch contains local structure and texture features. All patches are extracted and expanded into vector form to form a feature matrix:

[0095]

[0096] Where X1 and X2 represent the patch feature matrices of modalities I1 and I2, respectively, and M is the total number of image patches;

[0097] The construction of the feature matrix provides a standardized data representation of multimodal images, which facilitates subsequent feature extraction, decomposition and optimization;

[0098] Initialize dictionary matrices D1 and D2 and sparse representation matrices A1 and A2 for each modality;

[0099] The optimization objective function is defined as:

[0100]

[0101] Where, ‖·‖ F is the Frobenius norm, which is used to measure the error of dictionary reconstruction;

[0102] By optimizing the objective function, we ensure that the dictionary and sparse representation can accurately reconstruct the original image features while preserving the differences between the modalities;

[0103] In order to extract the shared information between the modalities, the sparse representations A1 and A2 of the two modalities are constrained to have the same non-zero position. The constraint conditions of the objective function are defined as:

[0104] supp(A1)=supp(A2),

[0105] Where supp() represents the support set of the sparse matrix, that is, the index position of the non-zero elements in the matrix;

[0106] This constraint ensures the consistency of shared features between the two modalities. The introduction of sparse representation reduces the data dimension and improves the efficiency of feature processing.

[0107] For each modality, fix D1 and D2, update A1 and A2, and solve the sparse representation matrix to obtain the updated sparse representation matrix A′ i :

[0108]

[0109] Where λ is the sparse regularization parameter, which controls the sparsity, and A i is the sparse representation matrix of mode I1 and mode I2;

[0110] Fix A1 and A2, optimize D1 and D2, and update the dictionary matrix to obtain D′ i :

[0111]

[0112] Where, X i is the input feature matrix of mode I1 and mode I2, D i is the dictionary matrix of mode I1 and mode I2;

[0113] After each update, the reconstruction error is calculated and the error threshold S is set. If the error is less than the threshold S, the iteration is stopped.

[0114] Based on the optimized dictionary and sparse representation, calculate the correlation features shared between modalities:

[0115] Z1=D1A1,Z2=D2A2,

[0116] Where Z1 and Z2 are correlation feature matrices;

[0117] Remove the correlation features from the original features to obtain the independence features:

[0118] E1=X1-Z1,E2=X2-Z2,

[0119] Where E1 and E2 are independence feature matrices, X1 and X2 represent the patch feature matrices of modalities I1 and I2 respectively;

[0120] Shared feature extraction captures the common information of multimodal images and enhances the overall consistency of image fusion. Independent feature extraction retains the unique information of each modality and supports the local enhancement of high-frequency features.

[0121] Calculate the information entropy H(Z i ):

[0122] H(Z i )=-∑P(Z i )·logP(Z i ),i=1,2,

[0123] Where, P(Z i ) is the characteristic matrix Z i Gray value probability distribution;

[0124] Dynamically assign weights based on information entropy:

[0125]

[0126] Where H(Z1) and H(Z2) are the information entropy of the correlation feature matrices Z1 and Z2;

[0127] Information entropy measures the importance of different modal features. By dynamically assigning weights, it ensures a more balanced fusion result of low-frequency components. Dynamic weight assignment adapts to the complexity of multimodal image features and enhances the clarity and contrast of the fused image.

[0128] According to the weight, Z1 and Z2 are fused to generate the low-frequency component F l (x,y):

[0129] F l(x,y)=b1×Z1(x,y)+b2×Z2(x,y),

[0130] Where Z1(x,y), Z2(x,y) are the values ​​of the correlation feature matrices Z1 and Z2 at the pixel position (x,y);

[0131] The maximum selection rule is used to fuse the independent features, retain the significant eigenvalues ​​of each mode, and generate the fused high-frequency component F h (x,y):

[0132] F h (x,y)=max(E1(x,y),E2(x,y)),

[0133] Where E1(x,y) and E2(x,y) are the values ​​of the independence feature matrices E1 and E2 at the pixel position (x,y).

[0134] Combine the fused low-frequency component and high-frequency component to generate the preliminary fused image F f (x,y) and normalize the fused image;

[0135] The fusion of low-frequency components improves the contrast of the image and makes the overall structural information more prominent, while the fusion of high-frequency components retains the detailed information of independent features and enhances the sharpness of the image;

[0136] The global contrast intensity of the image is calculated by the grayscale histogram and the clarity is measured by the mean of the image gradient intensity. Based on the contrast W and clarity X, the quality evaluation index R of the fused image is constructed as the optimization target:

[0137] R=w1×W+w2×X,

[0138] Where w1 and w2 are the weights of contrast and clarity respectively;

[0139] Use the Logistic Map model to initialize the contrast enhancement coefficient and noise suppression coefficient:

[0140] x a+1 =r×x a ×(1-x a ),

[0141] Where x a+1 Is the output value of Logistic Map at the current iteration step, x a is the current parameter value, r is the chaos control parameter, and a is the current number of iterations;

[0142] Map the current parameter value output by the Logistic Map model to the actual range of the optimized parameter:

[0143] α=xa+1 ·(α max -α min )+α min ,

[0144] β=x a+1 ·(β max -β min )+β min ,

[0145] Where x a+1 is the output value of Logistic Map at the current iteration step, α min , α max is the range of contrast enhancement coefficient, β min , β max is the range of the clarity adjustment coefficient, α is the contrast enhancement coefficient, and β is the clarity adjustment coefficient;

[0146] Dynamic optimization of the preliminary fused image via pulse coupled neural network:

[0147]

[0148] Where, F o (x,y) is the optimized image pixel value, F f (x,y) is the pixel value of the initial fused image, is the second-order gradient of the Laplace operator output, α is the contrast enhancement coefficient, and β is the clarity adjustment coefficient;

[0149] By calculating the grayscale histogram and gradient intensity, the quality evaluation index R can accurately measure the contrast and clarity of the image. Combined with the optimization of the Logistic Map model and pulse coupled neural network, the dynamic response ability and detail expression of the fused image are improved.

[0150] By extracting patch feature matrices from multimodal images and introducing sparse representation, the team successfully achieved dynamic separation of shared and independent features between modalities. This approach, combined with information entropy weighting and high- and low-frequency fusion optimization, generated high-quality imagery. By combining the sliding window method, Frobenius norm constraints, and pulse-coupled neural network optimization, the team significantly improved image contrast, clarity, and dynamic characteristics.

[0151] S2. Segment the optimized fused image into image blocks and assign a significance score to each block. Mark the candidate lesion area based on the significance score and generate a region mask.

[0152] Specifically, the optimized fusion image is segmented into image blocks and a significance score is assigned to each block. The optimized fusion image is divided into non-overlapping image blocks using a sliding window method. By performing block processing, local details of the image, especially important information such as edges and textures, can be focused on. Compared with global processing, block processing can effectively reduce the computational complexity of significance analysis. The feature calculation of each image block can be completed independently, which is suitable for parallel computing and improves processing efficiency. Linear interpolation is performed on the edge area of ​​the image to ensure that the segmentation is not affected by the image boundary, and then the local mean of the image block is calculated. and local gradient The local mean reflects the brightness change of the image block and helps to distinguish areas with different light intensities. The lesion area is usually different in brightness from the surrounding area. The local mean can be used as a basis for preliminary screening of lesions. Areas with high gradients usually correspond to image edges or texture features and can help locate important areas.

[0153] The saliency score e of each image patch is calculated based on the local mean and local gradient:

[0154]

[0155] Where N is the image block B j The number of pixels, is the image block B j The gradient of the jth pixel in , is the image block B j The local mean, μ max is the maximum local mean of all blocks;

[0156] The saliency score combines brightness features (local mean) with edge features (local gradient) to provide a more comprehensive assessment of region importance.

[0157] By segmenting the image into blocks using a sliding window method and combining local means and local gradients to calculate significance scores, this method not only accurately extracts features of the lesion region but also effectively reduces the computational complexity of global processing. This method's advantage lies in its comprehensive consideration of both image brightness and edge features, ensuring the reliability and accuracy of the significance score calculation results, providing a solid foundation for subsequent lesion region labeling and dynamic analysis. This saliency score-based segmentation method can significantly improve the accuracy and efficiency of lesion detection.

[0158] Furthermore, marking the candidate lesion region based on the significance score and generating the region mask refers to setting the significance threshold U through Otsu's method. Dynamically adjusting the significance threshold can find a balance between sensitivity and specificity. For example, in high-risk scenarios, the threshold U can be lowered to reduce the risk of missed diagnosis; in routine screening, the threshold can be increased to reduce the false detection rate. The threshold setting can be optimized based on the imaging characteristics of different modalities (such as CT, MRI, and PET), thereby achieving good recognition performance under different imaging conditions. Image blocks with saliency scores above the threshold are marked as salient, and all blocks with saliency scores above the threshold are combined into candidate lesion regions to significantly reduce the number of mislabeled non-lesion areas and provide a more reliable data foundation for subsequent analysis. Candidate lesion regions are generated by combining salient blocks and have a clear visual representation, providing medical practitioners with a more intuitive auxiliary diagnostic tool. A lesion region mask is generated based on the saliency score and the set threshold U, and candidate lesion regions are marked. Positions with a median value of 1 in the region mask correspond to salient regions. The lesion region mask is a binary data that is easy to store and manipulate, and also facilitates the automation of subsequent tasks such as time series change analysis and heat map generation. The mask format unifies the representation of lesion regions, laying the foundation for modeling the dynamic changes of lesion regions at multiple time points, further improving the spatiotemporal accuracy of the analysis.

[0159] The present invention's technique for labeling candidate lesion regions based on significance scores and generating region masks provides an efficient and robust method for medical image processing. Its significance score calculation ensures accurate lesion region identification, dynamic threshold setting improves the system's adaptability, and lesion region mask generation standardizes and visualizes lesion region expression, significantly improving the efficiency and reliability of clinical image processing.

[0160] S3. Perform time series change analysis on the regional mask to generate a heat map of the lesion area at multiple time points and perform visual display;

[0161] Specifically, performing time series change analysis on the regional mask to generate a multi-time point lesion region heat map means obtaining candidate lesion region mask data at multiple time points, extracting the lesion volume corresponding to each time point, and calculating the lesion volume by accumulating the total number of pixels in the lesion region:

[0162]

[0163] Where V t is the volume of the lesion at time point t, J(x,y) is the pixel value, the pixel value of the lesion area is 1, and the pixel value of the background area is 0, R t is the lesion area mask at time point t. This area consists of pixels (2D) or voxels (3D) and represents the location of the lesion area.

[0164] This process provides quantitative information on lesion development, helps transform qualitative analysis into quantitative analysis, and provides a basis for quantitative assessment of disease progression or treatment efficacy. By comparing volume change trends under multimodal imaging, it supports accurate disease diagnosis and optimization of treatment plans.

[0165] Calculate the volume change rate G of the lesion area at each moment t t :

[0166]

[0167] Where V t-1 is the lesion volume at time point t-1;

[0168] The volume change rate is used to quantify the growth or shrinkage of lesions over time and screen for areas with significant changes. The introduction of a dynamic change rate can effectively eliminate the influence of noise, capture the true dynamic characteristics of lesions, quickly locate lesion areas with significant changes, prioritize areas of clinical significance, accurately distinguish between disease progression and stable states, and optimize physicians' diagnostic processes.

[0169] A threshold m is set based on the statistical distribution of the lesion volume change rate in the historical case data, and regions with a change rate greater than m are selected as lesion regions with significant changes.

[0170] Initialize a heat map matrix with the same size as the image, initialize all pixel values ​​to 0, traverse all time points, gradually superimpose the significant change areas on the heat map to generate a multi-time point lesion area heat map, and normalize the generated heat map to generate a heat map matrix that intuitively displays the dynamic change characteristics of the lesion over time, provides spatial distribution information of disease progression, and facilitates physicians to quickly locate key change areas in complex lesions. Normalize the generated heat map to make the distribution of significant change values ​​more balanced. The normalization process eliminates the difference in numerical dimensions, improves the visualization effect of the heat map, and increases the contrast of dynamic change information at multiple time points, making subtle changes more obvious, making it easier to superimpose the heat map with the original image to generate clearer visualization results.

[0171] By analyzing the time series changes of regional masks and generating heat maps of lesion regions at multiple time points, this method enables precise quantification and intuitive visualization of lesion dynamics. This method not only plays an important role in dynamic monitoring and significance screening, but also provides a powerful tool for clinical diagnosis through the generated heat maps.

[0172] Furthermore, performing visual display refers to color mapping of the heat map. The heat map converts the lesion significance information into an intuitive color gradient image, providing physicians with an efficient visualization tool for dynamically changing areas. The application of color mapping can significantly enhance the contrast of the lesion change area, facilitate the rapid location and identification of lesions in complex backgrounds, and improve diagnostic efficiency. The heat map is weightedly superimposed with the original image, and the heat map area is displayed on the original image. The weighted superposition provides physicians with a more complete diagnostic view by integrating the significance information of the heat map and the anatomical structure details of the original image. The flexible setting of weights can highlight the heat map information or retain more original image details according to actual needs, realizing targeted data display. The heat map area is marked and highlighted. The marking and highlighting functions can help physicians further focus on the core area of ​​the lesion and avoid missing key information in complex images. Highlighting can enhance the visualization of the lesion through color or borders, so that it maintains consistent readability on different display devices. The generated heat map image is saved as an image file and stored in a database. The storage of the heat map makes the diagnostic results of the dynamically changing area persistent, facilitating subsequent case tracking and analysis.

[0173] Through technologies such as color mapping, weighted overlay, labeling, and highlighting, this solution provides more efficient and intuitive support for interpreting medical imaging data. The dynamic generation and overlay of heat maps not only improves the physician's diagnostic experience but also provides a more information-dense visualization of dynamically changing lesion areas. Furthermore, database storage ensures data security, traceability, and long-term value. This solution not only improves the efficiency of medical image processing, but also provides strong technical support for clinical diagnosis and research.

[0174] Example 2, reference Figure 2 , which is the second embodiment of the present invention. This embodiment is different from the previous embodiment and provides a medical image intelligent processing system based on deep learning, which includes:

[0175] An acquisition module, used to acquire multimodal images using medical imaging equipment and perform preprocessing;

[0176] An image registration module is used to perform coarse registration and precise registration on the pre-processed images to obtain registered medical images;

[0177] The fusion optimization module is used to perform multimodal fusion and optimization on the registered images to obtain high-quality fused images;

[0178] The region marking module is used to calculate the significance score of each image block after image segmentation of the fused image, mark the candidate lesion area according to the significance score, and generate the region mask corresponding to the candidate lesion area;

[0179] An analysis module, used to perform time series analysis based on the mask and generate a heat map of the lesion area at multiple time points;

[0180] The storage module is displayed, which is used to perform color mapping on the generated heat map, perform weighted overlay with the original image, and store the heat map image in the database.

[0181] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0182] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0183] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0184] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

Claims

1. A medical image intelligent processing method based on deep learning, characterized by: include, Collect and preprocess multimodal medical imaging data, register medical images of different modalities, fuse and optimize the registered multimodal images; The optimized fused image is segmented into image blocks and a significance score is assigned to each block. The candidate lesion area is marked based on the significance score and a region mask is generated. Performing time series change analysis on the regional mask to generate a multi-time point lesion region heat map and performing visual display; said collecting multimodal medical imaging data and performing preprocessing, and performing registration on medical images of different modalities refers to using medical imaging equipment to collect multimodal imaging data, adding a timestamp to each frame of the image, and normalizing the image pixel values ​​to adjust the image resolution to a uniform scale; The multimodal imaging data includes CT, MRI and PET; Select the CT modality as the reference image and the other modalities as the target images. Divide the pixel values ​​of the reference image and the target image into K intervals, and count the joint occurrence frequencies of the corresponding pixel values ​​of the reference image and the target image to construct a joint histogram. Normalize the joint histogram into a joint probability distribution, calculate the marginal probability distribution of the reference image and the target image respectively, and calculate the mutual information F based on the joint probability distribution and the marginal probability distribution; Define the optimization goal as maximizing mutual information, initialize the rotation parameters and translation parameters, construct the initial transformation matrix based on the initialization parameters, apply the initial transformation matrix to the target image, generate the transformed target image, and calculate the mutual information based on the pixel value distribution of the target image and the reference image; Use the grid search method to update the transformation matrix parameters, and use the parameter combination with the maximum mutual information as the candidate for the next step. Iteratively optimize the parameter combination until the maximum number of iterations is reached. Based on the iteratively optimized parameter combination, the optimized optimal transformation matrix is ​​obtained. The target image transformed by the optimal transformation matrix is ​​used as the rough registration result; Use edge detection algorithm to extract the edges of reference image and target image, calculate the centroid of edge pixels of reference image and target image and select evenly distributed points to form reference point set and target point set; Calculate the centroid of the reference point set and the target point set, decentralize the point set, use the decentralized point set to construct the covariance matrix, perform singular value decomposition on the covariance matrix and calculate the rotation matrix, and calculate the translation vector based on the centroid of the rotated point set; Apply rotation and translation transformations to each point in the target point set to obtain an updated target point set; Calculate the mean square error between the updated target point set and the reference point set, update the rotation matrix and translation vector through multiple iterations, set the maximum number of iterations, and stop the iteration when the maximum number of iterations is reached to obtain the final rotation matrix and translation vector; The final rotation matrix and translation vector are used to fully align the target image to the reference image, generating the final registration result.

2. The deep learning-based intelligent medical image processing method according to claim 1, wherein: The said fusing registered multimodal images and optimizing them refers to extracting multimodal image pairs from the registered multimodal images, representing the modalities and modal , use the sliding window method to divide the image into image patches of size p*p, extract all patches and expand them into vector form to form a feature matrix: ， ， Where, and Represents the mode and Patch feature matrix, M is the total number of image patches; initialize the dictionary matrix for each modality and and sparse representation matrix and ; The optimization objective function is defined as: ， Where, is the Frobenius norm; The constraints that define the objective function are: ， Where, represents the support set of the sparse matrix; For each mode, fix and ,renew and , solve the sparse representation matrix to get the updated sparse representation matrix ; fixed and ,optimization and , update the dictionary matrix to get ; After each update, the reconstruction error is calculated and the error threshold S is set. If the error is less than the threshold S, the iteration is stopped. Based on the optimized dictionary and sparse representation, calculate the correlation features shared between modalities: ， ， Where, and is the correlation feature matrix; Remove the correlation features from the original features to obtain the independence features: ， ， Where, and is the independence characteristic matrix, and Represents the mode and The patch feature matrix of Calculate the correlation feature matrix and Information entropy ; Dynamically assign weights based on information entropy and ; According to the weight and Fusion generates low-frequency components ; The maximum selection rule is used to fuse the independent features, retain the significant eigenvalues ​​of each mode, and generate the fused high-frequency components. ; Combine the fused low-frequency component and high-frequency component to generate a preliminary fused image And normalize the fused image; The global contrast intensity of the image is calculated by the grayscale histogram and the clarity is measured by the mean of the image gradient intensity. The quality evaluation index R of the fused image is constructed based on the contrast W and clarity X as the optimization target. Use the Logistic Map model to initialize the contrast enhancement coefficient and noise suppression coefficient; Map the current parameter value output by the Logistic Map model to the actual range of the optimized parameter; The preliminary fusion image is dynamically optimized through pulse coupled neural network.

3. The deep learning-based intelligent medical image processing method according to claim 2, wherein: The segmentation of the optimized fused image into image blocks and the assignment of a significance score to each block refers to dividing the optimized fused image into non-overlapping image blocks using a sliding window method, performing linear interpolation on the edge areas of the image, and calculating the local mean of the image blocks. and local gradient ; The saliency score e of each image patch is calculated based on the local mean and local gradient: ， Where N is the image block The number of pixels, The gradient of the jth pixel in , is an image block The local mean of is the maximum local mean of all blocks.

4. The deep learning-based intelligent medical image processing method according to claim 3, wherein: The marking of candidate lesion regions based on the saliency score and generating a region mask refers to setting a saliency threshold U, marking image blocks with saliency scores higher than the threshold as saliency, combining all blocks with saliency scores higher than the threshold into candidate lesion regions, generating a lesion region mask based on the saliency score and the set threshold U, and marking the candidate lesion regions.

5. The deep learning-based intelligent medical image processing method according to claim 4, wherein: The generating of a multi-time-point lesion region heat map by performing a time series change analysis on the region mask refers to obtaining candidate lesion region mask data at multiple time points, extracting the lesion volume corresponding to each time point, and calculating the lesion volume by accumulating the total number of pixels in the lesion region; Calculate the volume change rate of the lesion area at each time t ; Set a threshold m and select areas with a change rate greater than m as lesion areas with significant changes; Initialize the heat map matrix with the same size as the image, initialize all pixel values ​​to 0, traverse all time points, gradually superimpose the significantly changed areas onto the heat map to generate a multi-time point lesion area heat map, and normalize the generated heat map.

6. The deep learning-based intelligent medical image processing method according to claim 5, wherein: The visual display refers to color mapping the heat map, weighted superposition of the heat map and the original image, displaying the heat map area on the original image, marking and highlighting the heat map area, saving the generated heat map image as an image file and storing it in a database.

7. A deep learning-based intelligent medical image processing system based on the deep learning-based intelligent medical image processing method according to any one of claims 1 to 6, characterized in that: include, An acquisition module, used to acquire multimodal images using medical imaging equipment and perform preprocessing; An image registration module is used to perform coarse registration and precise registration on the pre-processed images to obtain registered medical images; The fusion optimization module is used to perform multimodal fusion and optimization on the registered images to obtain high-quality fused images; The region marking module is used to calculate the significance score of each image block after image segmentation of the fused image, mark the candidate lesion area according to the significance score, and generate the region mask corresponding to the candidate lesion area; An analysis module, used to perform time series analysis based on the mask and generate a heat map of the lesion area at multiple time points; The storage module is displayed, which is used to perform color mapping on the generated heat map, perform weighted overlay with the original image, and store the heat map image in the database.

8. A computer device comprising: memory and processor; The memory stores a computer program, characterized in that when the processor executes the computer program, the steps of the medical image intelligent processing method based on deep learning described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the medical image intelligent processing method based on deep learning described in any one of claims 1 to 6 are implemented.

Images