Medical image intelligent processing method and system based on deep learning

Abstract

The invention discloses a medical image intelligent processing method and system based on deep learning, and relates to the technical field of image processing, and the method comprises the steps: collecting and preprocessing multi-modal medical image data, carrying out the registration of medical images of different modals, and fusing and optimizing the registered multi-modal images. Carrying out image block segmentation on the optimized fused image, distributing a significance score of each block, marking a candidate lesion region based on the significance scores, and generating a region mask; and carrying out time sequence change analysis on the region mask to generate a multi-time-point focus region heat map, and carrying out visual display. According to the method, deep learning is combined with a traditional registration and fusion technology, the registration precision can be remarkably improved, the fusion effect can be optimized, multi-time-point focus change visual analysis is achieved through a dynamic focus heat map generation technology, and accurate and dynamic support is provided for diagnosis and treatment of complex diseases.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a medical image intelligent processing method and system based on deep learning. Background Art

[0002] In recent years, with the rapid development of medical imaging technology, multimodal imaging technologies such as CT, MRI and PET have been widely used in clinical diagnosis and disease detection. Each of these technologies has its own unique advantages. For example, CT can provide precise anatomical information, MRI can highlight the details of soft tissues, and PET can present the metabolic activity of lesions. Multimodal medical imaging combines the advantages of different imaging technologies and provides richer information for the comprehensive diagnosis of complex diseases. However, due to the different sources and physical mechanisms of the images, these multimodal images usually have significant differences in spatial resolution, contrast and dynamic range, making accurate registration and fusion between images a technical difficulty. Traditional multimodal image processing methods mainly rely on algorithms based on rules or statistical models, such as registration methods based on mutual information maximization and linear fusion techniques. However, these methods can usually only process static features and have limited ability to describe the spatiotemporal characteristics of complex dynamic lesions. In addition, deep learning technology has made significant progress in the field of medical image processing in recent years. Technologies such as convolutional neural networks (CNN) and generative adversarial networks (GAN) have been widely used in image enhancement, segmentation and classification tasks. However, most of these technologies focus on the processing of single-modality images, and how to fuse, analyze and dynamically monitor multi-modality image data remains an unresolved problem. Summary of the invention

[0003] In view of the problems existing in the above-mentioned existing medical image intelligent processing methods and systems based on deep learning, the present invention is proposed.

[0004] Therefore, the problem to be solved by the present invention is that most of these technologies focus on the processing of single-modality images, and how to fuse, analyze and dynamically monitor multi-modality image data is still an unresolved problem.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: a medical image intelligent processing method based on deep learning, which includes collecting multimodal medical image data and preprocessing it, registering medical images of different modalities, fusing the registered multimodal images and optimizing them;

[0006] The optimized fused image is segmented into image blocks and a significance score is assigned to each block. The candidate lesion area is marked based on the significance score and a regional mask is generated.

[0007] Time series change analysis of regional masks was performed to generate heat maps of lesion areas at multiple time points and visualize them.

[0008] As a preferred solution of the medical image intelligent processing method based on deep learning described in the present invention, wherein: collecting multimodal medical image data and preprocessing, registering medical images of different modalities refers to using medical imaging equipment to collect multimodal image data, adding a timestamp to each frame of the image, and adjusting the image resolution to a uniform scale after normalizing the image pixel values;

[0009] The multimodal imaging data includes CT, MRI and PET;

[0010] Select CT modality as reference image and other modalities as target image, divide the pixel values ​​of reference image and target image into K intervals, count the joint occurrence frequency of corresponding pixel values ​​of reference image and target image to construct joint histogram;

[0011] Normalize the joint histogram to a joint probability distribution, calculate the marginal probability distribution of the reference image and the target image respectively, and calculate the mutual information F based on the joint probability distribution and the marginal probability distribution;

[0012] Define the optimization goal as maximizing mutual information, initialize the rotation parameters and translation parameters, construct the initial transformation matrix based on the initialization parameters, apply the initial transformation matrix to the target image, generate the transformed target image, and calculate the mutual information based on the pixel value distribution of the target image and the reference image;

[0013] Use the grid search method to update the transformation matrix parameters, take the parameter combination with the maximum mutual information as the candidate for the next step, iteratively optimize the parameter combination until the maximum number of iterations is reached, and obtain the optimized optimal transformation matrix based on the iteratively optimized parameter combination;

[0014] The target image transformed by the optimal transformation matrix is ​​used as the rough registration result;

[0015] Use edge detection algorithm to extract the edges of reference image and target image, calculate the centroid of edge pixels of reference image and target image and select evenly distributed points to form reference point set and target point set;

[0016] Calculate the centroid of the reference point set and the target point set, de-centroid the point set, use the de-centroided point set to construct the covariance matrix, perform singular value decomposition on the covariance matrix and calculate the rotation matrix, and calculate the translation vector based on the centroid of the rotated point set;

[0017] Apply rotation and translation transformations to each point in the target point set to obtain an updated target point set;

[0018] Calculate the mean square error between the updated target point set and the reference point set, update the rotation matrix and translation vector through multiple iterations, set the maximum number of iterations, and stop the iteration when the maximum number of iterations is reached to obtain the final rotation matrix and translation vector;

[0019] The final rotation matrix and translation vector are used to fully align the target image to the reference image, generating the final registration result.

[0020] As a preferred solution of the medical image intelligent processing method based on deep learning described in the present invention, wherein: fusing the registered multimodal images and optimizing them refers to extracting multimodal image pairs from the registered multimodal images, representing modality I1 and modality I2 respectively, using a sliding window method to divide the image into image patches of size p*p, extracting all patches and expanding them into vector form to form a feature matrix:

[0021]

[0022] Where X1 and X2 represent the patch feature matrices of modalities I1 and I2, respectively, and M is the total number of image patches; initialize the dictionary matrices D1 and D2 and the sparse representation matrices A1 and A2 for each modality;

[0023] The optimization objective function is defined as:

[0024]

[0025] In the formula, ‖·‖ F is the Frobenius norm;

[0026] The constraints for defining the objective function are:

[0027] supp(A1)=supp(A2),

[0028] Where supp() represents the support set of the sparse matrix;

[0029] For each mode, fix D1 and D2, update A1 and A2, and solve the sparse representation matrix to obtain the updated sparse representation matrix A′ i ;

[0030] Fix A1 and A2, optimize D1 and D2, and update the dictionary matrix to get D′ i ;

[0031] After each update, the reconstruction error is calculated and the error threshold S is set. If the error is less than the threshold S, the iteration is stopped;

[0032] Based on the optimized dictionary and sparse representation, calculate the correlation features shared between modalities:

[0033] Z1=D1A1,Z2=D2A2,

[0034] Where Z1 and Z2 are correlation feature matrices;

[0035] Remove the correlation features from the original features to obtain the independence features:

[0036] E1=X1-Z1,E2=X2-Z2,

[0037] Where E1 and E2 are the independence feature matrices, X1 and X2 represent the patch feature matrices of modalities I1 and I2 respectively;

[0038] Calculate the information entropy H(Z) of the correlation feature matrices Z1 and Z2 i );

[0039] Dynamically assign weights b1 and b2 based on information entropy;

[0040] According to the weight, Z1 and Z2 are fused to generate the low-frequency component F l (x,y);

[0041] The maximum selection rule is used to fuse the independent features, retain the significant eigenvalues ​​of each mode, and generate the fused high-frequency component F h (x,y);

[0042] The fused low-frequency component and high-frequency component are combined to generate a preliminary fused image F f (x,y) and normalize the fused image;

[0043] The global contrast intensity of the image is calculated by the grayscale histogram and the clarity is measured by the mean value of the image gradient intensity. The quality evaluation index R of the fused image is constructed based on the contrast W and clarity X as the optimization target.

[0044] Use the Logistic Map model to initialize the contrast enhancement coefficient and noise suppression coefficient;

[0045] Map the current parameter value output by the Logistic Map model to the actual range of the optimized parameter;

[0046] The preliminary fusion image is dynamically optimized through pulse coupled neural network.

[0047] As a preferred solution of the medical image intelligent processing method based on deep learning described in the present invention, wherein: segmenting the optimized fused image into image blocks and assigning a significance score to each block means using a sliding window method to divide the optimized fused image into non-overlapping image blocks, performing linear interpolation on the edge area of ​​the image, and calculating the local mean of the image blocks. and local gradient

[0048] The saliency score e of each image patch is calculated based on the local mean and local gradient:

[0049]

[0050] Where N is the image block B j The number of pixels, is the image block B j The gradient of the jth pixel in , is the image block B j The local mean, μ max is the maximum local mean of all blocks.

[0051] As a preferred solution of the medical image intelligent processing method based on deep learning described in the present invention, wherein: marking the candidate lesion area based on the significance score and generating a regional mask refers to setting a significance threshold U, marking the image blocks with significance scores higher than the threshold as significant, combining all blocks with significance scores higher than the threshold into candidate lesion areas, generating a lesion area mask according to the significance score and the set threshold U, and marking the candidate lesion area.

[0052] As a preferred solution of the medical image intelligent processing method based on deep learning described in the present invention, wherein: performing time series change analysis on the regional mask to generate a multi-time point lesion region heat map refers to obtaining candidate lesion region mask data at multiple time points, extracting the lesion volume corresponding to each time point, and calculating the lesion volume by accumulating the total number of pixels in the lesion region;

[0053] Calculate the volume change rate G of the lesion area at each time t t ;

[0054] Set a threshold m, and select areas with a change rate greater than m as lesion areas with significant changes;

[0055] Initialize the heat map matrix with the same size as the image, initialize all pixel values ​​to 0, traverse all time points, gradually superimpose the significantly changed areas onto the heat map to generate a multi-time point lesion area heat map, and normalize the generated heat map.

[0056] As a preferred solution of the medical image intelligent processing method based on deep learning described in the present invention, wherein: visual display refers to color mapping of the heat map, weighted superposition of the heat map and the original image, displaying the heat map area on the original image, marking and highlighting the heat map area, saving the generated heat map image as an image file and storing it in a database.

[0057] Another object of the present invention is to provide a medical image intelligent processing system based on deep learning, which comprises:

[0058] An acquisition module, used to acquire multimodal images using medical imaging equipment and perform preprocessing;

[0059] An image registration module is used to perform rough registration and precise registration on the pre-processed image to obtain a registered medical image;

[0060] The fusion optimization module is used to perform multi-modal fusion and optimization on the registered images to obtain high-quality fused images;

[0061] The region marking module is used to calculate the significance score of each image block after image segmentation of the fused image, mark the candidate lesion region according to the significance score, and generate a region mask corresponding to the candidate lesion region;

[0062] An analysis module, used to perform time series analysis based on the mask and generate a heat map of the lesion area at multiple time points;

[0063] The storage module is displayed, which is used to perform color mapping on the generated heat map, weighted overlay it with the original image, and store the heat map image in the database.

[0064] A computer device comprises: a memory and a processor; the memory stores a computer program, and the processor implements the steps of a medical image intelligent processing method based on deep learning when executing the computer program.

[0065] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a medical image intelligent processing method based on deep learning.

[0066] The beneficial effects of the present invention are as follows: by combining deep learning with traditional registration and fusion technologies, the present invention can significantly improve the registration accuracy and optimize the fusion effect, and through the dynamic lesion heat map generation technology, it can realize the visualization analysis of lesion changes at multiple time points, providing accurate and dynamic support for the diagnosis and treatment of complex diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0068] Figure 1 The figure is a flowchart of the intelligent processing method of medical images based on deep learning.

[0069] Figure 2 This is a schematic diagram of the structure of the medical image intelligent processing system based on deep learning. DETAILED DESCRIPTION

[0070] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.

[0071] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0072] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it a separate or selective embodiment that is mutually exclusive with other embodiments.

[0073] Example 1, reference Figure 1 , is the first embodiment of the present invention, which provides a medical image intelligent processing method based on deep learning, and the medical image intelligent processing method based on deep learning includes:

[0074] S1. Collect and preprocess multimodal medical imaging data, register medical images of different modalities, fuse and optimize the registered multimodal images;

[0075] Specifically, collecting multimodal medical imaging data and preprocessing them, registering medical images of different modalities means using medical imaging equipment to collect multimodal imaging data, adding a timestamp to each frame of the image, and adjusting the image resolution to a uniform scale after normalizing the image pixel values;

[0076] Through standardization processing (such as pixel value normalization and resolution unification), the dynamic range differences of images produced by different devices are eliminated to ensure the consistency of input data, and timestamp information is added to the images, laying the foundation for subsequent time series analysis;

[0077] The multimodal imaging data includes CT, MRI and PET;

[0078] Select CT modality as reference image and other modalities as target image, divide the pixel values ​​of reference image and target image into K intervals, count the joint occurrence frequency of corresponding pixel values ​​of reference image and target image to construct joint histogram;

[0079] Normalize the joint histogram to a joint probability distribution, calculate the marginal probability distribution of the reference image and the target image respectively, and calculate the mutual information F based on the joint probability distribution and the marginal probability distribution:

[0080]

[0081] Where P(z,c) is the joint probability distribution, which indicates the probability of the pixel values ​​z and c appearing simultaneously in the reference image and the target image. P(z) is the edge probability of the gray value z in the reference image, and P(c) is the edge probability of the gray value c in the target image.

[0082] The joint histogram captures the global grayscale relationship between multimodal images and provides an intuitive statistical basis for the calculation of mutual information. Mutual information can effectively deal with the nonlinear differences between different modalities by quantifying the pixel value dependence between images.

[0083] Define the optimization goal as maximizing mutual information, initialize the rotation parameters and translation parameters, construct the initial transformation matrix based on the initialization parameters, apply the initial transformation matrix to the target image, generate the transformed target image, and calculate the mutual information based on the pixel value distribution of the target image and the reference image;

[0084] Use the grid search method to update the transformation matrix parameters, take the parameter combination with the maximum mutual information as the candidate for the next step, iteratively optimize the parameter combination until the maximum number of iterations is reached, and obtain the optimized optimal transformation matrix based on the iteratively optimized parameter combination;

[0085] The target image transformed by the optimal transformation matrix is ​​used as the rough registration result;

[0086] The estimation of the initial transformation matrix directly affects the accuracy of the registration result. The global parameter optimization is performed through the grid search method to ensure that a high initial accuracy can be achieved in the rough registration stage. Combined with the mutual information evaluation function, the grid search can significantly improve the efficiency of the initial alignment of the inter-modal structure.

[0087] Use edge detection algorithm to extract the edges of reference image and target image, calculate the centroid of edge pixels of reference image and target image and select evenly distributed points to form reference point set and target point set;

[0088] Calculate the centroid of the reference point set and the target point set, de-centroid the point set, use the de-centroided point set to construct the covariance matrix, perform singular value decomposition on the covariance matrix and calculate the rotation matrix, and calculate the translation vector based on the centroid of the rotated point set;

[0089] The reference point set and target point set constructed using edge detection and centroid extraction algorithms significantly reduce complexity, and the extracted point set has high geometric consistency. The singular value decomposition accurately captures the main direction in the covariance matrix and can efficiently calculate the rotation matrix. Combined with the centroid translation vector calculation, accurate alignment of the target image is achieved;

[0090] Apply rotation and translation transformations to each point in the target point set to obtain an updated target point set;

[0091] The mean square error between the updated target point set and the reference point set is calculated. The mean square error (MSE) is used as the evaluation index for iterative optimization to ensure the robustness and accuracy of the registration result. The rotation matrix and translation vector are updated through multiple iterations to reduce the registration error. The maximum number of iterations is set. When the maximum number of iterations is reached, the iteration is stopped to obtain the final rotation matrix and translation vector. The rotation matrix and translation vector are optimized through multiple iterations to reduce the error between the target point set and the reference point set in each round of calculation. By setting the maximum number of iterations and the error threshold, the calculation cost and accuracy requirements can be flexibly controlled.

[0092] The final rotation matrix and translation vector are used to fully align the target image to the reference image to generate the final registration result. The combination of coarse registration and precise registration makes full use of global and local features to achieve high-precision alignment of the target image to the reference image. The final registration result provides high-quality input data for subsequent image fusion and lesion significance analysis.

[0093] Existing medical image processing technologies have achieved certain results in multimodal image registration, fusion, lesion detection, etc., but there are still certain limitations. The present invention overcomes the shortcomings of the existing technologies in terms of accuracy, efficiency and robustness by introducing a registration algorithm that maximizes mutual information, a fusion optimization method for sparse representation, and a lesion detection technology based on significance scores, and provides a more efficient and accurate solution in multimodal image fusion and lesion area analysis.

[0094] Furthermore, fusing the registered multimodal images and optimizing them refers to extracting multimodal image pairs from the registered multimodal images, representing modality I1 and modality I2 respectively, and using the sliding window method to divide the images into image patches of size p*p. The sliding window method ensures that the local information of the image is fully captured. Each patch contains local structure and texture features. All patches are extracted and expanded into vector form to form a feature matrix:

[0095]

[0096] Where X1 and X2 represent the patch feature matrices of modalities I1 and I2, respectively, and M is the total number of image patches;

[0097] The construction of feature matrix provides a standardized data representation of multimodal images, which is convenient for subsequent feature extraction, decomposition and optimization;

[0098] Initialize dictionary matrices D1 and D2 and sparse representation matrices A1 and A2 for each modality;

[0099] The optimization objective function is defined as:

[0100]

[0101] In the formula, ‖·‖ F is the Frobenius norm, which is used to measure the error of dictionary reconstruction;

[0102] By optimizing the objective function, we ensure that the dictionary and sparse representation can accurately reconstruct the original image features while retaining the differences between the modalities;

[0103] In order to extract the shared information between the modalities, the sparse representations A1 and A2 of the two modalities are constrained to have the same non-zero position, and the constraint condition of the objective function is defined as:

[0104] supp(A1)=supp(A2),

[0105] In the formula, supp() represents the support set of the sparse matrix, that is, the index position of the non-zero elements in the matrix;

[0106] This constraint ensures the consistency of the shared features of the two modalities. The introduction of sparse representation reduces the data dimension and improves the efficiency of feature processing.

[0107] For each mode, fix D1 and D2, update A1 and A2, and solve the sparse representation matrix to obtain the updated sparse representation matrix A′ i :

[0108]

[0109] Where λ is the sparse regularization parameter, which controls sparsity, and A i is the sparse representation matrix of mode I1 and mode I2;

[0110] Fix A1 and A2, optimize D1 and D2, and update the dictionary matrix to get D′ i :

[0111]

[0112] In the formula, X i is the input feature matrix of mode I1 and mode I2, D i is the dictionary matrix of modality I1 and modality I2;

[0113] After each update, the reconstruction error is calculated and the error threshold S is set. If the error is less than the threshold S, the iteration is stopped;

[0114] Based on the optimized dictionary and sparse representation, calculate the correlation features shared between modalities:

[0115] Z1=D1A1,Z2=D2A2,

[0116] Where Z1 and Z2 are correlation feature matrices;

[0117] Remove the correlation features from the original features to obtain the independence features:

[0118] E1=X1-Z1,E2=X2-Z2,

[0119] Where E1 and E2 are the independence feature matrices, X1 and X2 represent the patch feature matrices of modalities I1 and I2 respectively;

[0120] Shared feature extraction captures the common information of multi-modal images and enhances the overall consistency of image fusion. Independent feature extraction retains the unique information of each modality and provides support for local enhancement of high-frequency features.

[0121] Calculate the information entropy H(Z) of the correlation feature matrices Z1 and Z2 i ):

[0122] H(Z i )=-∑P(Z i )·logP(Z i ),i=1,2,

[0123] In the formula, P(Z i ) is the feature matrix Z i The gray value probability distribution of ;

[0124] Dynamically assign weights based on information entropy:

[0125]

[0126] Where H(Z1) and H(Z2) are the information entropy of the correlation feature matrices Z1 and Z2;

[0127] Information entropy measures the importance of different modal features. By dynamically allocating weights, it ensures that the fusion results of low-frequency components are more balanced. Dynamic weight allocation adapts to the complexity of multi-modal image features and enhances the clarity and contrast of the fused image.

[0128] According to the weight, Z1 and Z2 are fused to generate the low-frequency component F l (x,y):

[0129] F l(x,y)=b1×Z1(x,y)+b2×Z2(x,y),

[0130] Where Z1(x,y), Z2(x,y) are the values ​​of the correlation feature matrices Z1 and Z2 at the pixel position (x,y);

[0131] The maximum selection rule is used to fuse the independent features, retain the significant eigenvalues ​​of each mode, and generate the fused high-frequency component F h (x,y):

[0132] F h (x,y)=max(E1(x,y),E2(x,y)),

[0133] Where E1(x,y), E2(x,y) are the values ​​of the independence feature matrices E1 and E2 at the pixel position (x,y);

[0134] The fused low-frequency component and high-frequency component are combined to generate a preliminary fused image F f (x,y) and normalize the fused image;

[0135] The fusion of low-frequency components improves the contrast of the image and makes the overall structural information more prominent, while the fusion of high-frequency components retains the detailed information of independent features and enhances the sharpening effect of the image;

[0136] The global contrast intensity of the image is calculated by the grayscale histogram and the clarity is measured by the mean value of the image gradient intensity. The quality evaluation index R of the fused image is constructed based on the contrast W and clarity X as the optimization target:

[0137] R=w1×W+w2×X,

[0138] In the formula, w1 and w2 are the weights of contrast and clarity respectively;

[0139] Use the Logistic Map model to initialize the contrast enhancement coefficient and noise suppression coefficient:

[0140] x a+1 =r×x a ×(1-x a ),

[0141] In the formula, x a+1 is the output value of Logistic Map at the current iteration step, x a is the current parameter value, r is the chaos control parameter, and a is the current iteration number;

[0142] Map the current parameter values ​​output by the Logistic Map model to the actual range of the optimized parameters:

[0143] α=xa+1 ·(α max -α min )+α min ,

[0144] β=x a+1 ·(β max -β min )+β min ,

[0145] In the formula, x a+1 is the output value of Logistic Map at the current iteration step, α min , α max is the range of contrast enhancement factor, β min , β max is the range of the clarity adjustment coefficient, α is the contrast enhancement coefficient, and β is the clarity adjustment coefficient;

[0146] Dynamic optimization of the preliminary fusion image through pulse coupled neural network:

[0147]

[0148] In the formula, F o (x, y) is the optimized image pixel value, F f (x,y) is the pixel value of the initial fused image, is the second-order gradient of the Laplace operator output, α is the contrast enhancement coefficient, and β is the clarity adjustment coefficient;

[0149] Through the calculation of grayscale histogram and gradient intensity, the quality evaluation index R can accurately measure the contrast and clarity of the image. Combined with the optimization of the Logistic Map model and pulse coupled neural network, the dynamic response ability and detail expression of the fused image are improved.

[0150] By extracting patch feature matrices from multimodal images and introducing sparse representation, the dynamic separation of shared features and independent features between modalities was successfully achieved, and high-quality image results were generated by combining information entropy weight allocation and high- and low-frequency fusion optimization. The contrast, clarity and dynamic characteristics of the image were significantly improved through the joint optimization of the sliding window method, Frobenius norm constraint and pulse coupled neural network.

[0151] S2, segmenting the optimized fused image into image blocks and assigning a significance score to each block, marking the candidate lesion area based on the significance score and generating a regional mask;

[0152] Specifically, the optimized fused image is segmented into image blocks and the significance score of each block is assigned, which means that the optimized fused image is divided into non-overlapping image blocks using a sliding window method. By block processing, the local details of the image can be focused on, especially important information such as edges and textures. Compared with global processing, block processing can effectively reduce the amount of calculation for significance analysis. The feature calculation of each image block can be completed independently, which is suitable for parallel computing and improves processing efficiency. Linear interpolation is performed on the edge area of ​​the image to ensure that the segmentation is not affected by the image boundary, and then the local mean of the image block is calculated. and local gradient The local mean reflects the brightness change of the image block, which helps to distinguish areas with different light intensities. The lesion area is usually different in brightness from the surrounding area. The local mean can be used as a basis for preliminary screening of lesions. Areas with high gradients usually correspond to the edge or texture features of the image, which can help locate important areas.

[0153] The saliency score e of each image patch is calculated based on the local mean and local gradient:

[0154]

[0155] Where N is the image block B j The number of pixels, is the image block B j The gradient of the jth pixel in , is the image block B j The local mean, μ max is the maximum local mean of all blocks;

[0156] The saliency score combines brightness features (local mean) with edge features (local gradient) to provide a more comprehensive assessment of region importance.

[0157] The image is processed in blocks by the sliding window method, and the significance score is calculated by combining the local mean and local gradient. This method can not only accurately extract the characteristics of the lesion area, but also effectively reduce the computational complexity of global processing. The advantage of this method is that it comprehensively considers the brightness characteristics and edge characteristics of the image, ensuring that the calculation results of the significance score are highly reliable and accurate, providing a solid foundation for subsequent lesion area marking and dynamic analysis. This block processing method based on the significance score can significantly improve the accuracy and efficiency of lesion detection.

[0158] Furthermore, marking the candidate lesion region based on the significance score and generating the region mask refers to setting the significance threshold U by Otsu's method. Dynamically adjusting the significance threshold can find a balance between sensitivity and specificity. For example, in high-risk scenarios, the threshold U can be lowered to reduce the risk of missed diagnosis; in routine screening, the threshold can be increased to reduce the false detection rate. The setting of the threshold can be optimized according to the imaging characteristics of different modalities (such as CT, MRI, and PET), so that good recognition performance can be achieved under different imaging conditions. Image blocks with significance scores higher than the threshold are marked as significant, and all blocks with significance scores higher than the threshold are combined into candidate lesion regions to significantly reduce the mislabeled non-lesion regions, providing a more reliable data basis for subsequent analysis. Candidate lesion regions are generated by combining significant blocks and have clear visual expressions, providing medical practitioners with a more intuitive auxiliary diagnostic tool. Lesion region masks are generated according to the significance scores and the set threshold U, and candidate lesion regions are marked. The positions with a median value of 1 in the region mask correspond to significant regions. Lesion region masks are binary data that are easy to store and operate, and are also convenient for the automated processing of subsequent tasks such as time series change analysis and heat map generation. The mask form unifies the expression of lesion regions, lays the foundation for modeling the dynamic changes of lesion regions at multiple time points, and further improves the spatiotemporal accuracy of the analysis.

[0159] The technology of marking candidate lesion regions based on significance scores and generating regional masks in the present invention provides an efficient and robust medical image processing method. The calculation of the significance scores ensures the accurate identification of lesion regions, the dynamic threshold setting improves the adaptability of the system, and the generation of lesion region masks realizes the standardization and visualization of lesion region expression, which significantly improves the efficiency and reliability of clinical image processing.

[0160] S3, perform time series change analysis on the regional mask to generate a heat map of the lesion area at multiple time points and perform visual display;

[0161] Specifically, performing time series change analysis on the regional mask to generate a multi-time point lesion region heat map means obtaining candidate lesion region mask data at multiple time points, extracting the lesion volume corresponding to each time point, and calculating the lesion volume by accumulating the total number of pixels in the lesion region:

[0162]

[0163] Where V t is the volume of the lesion at time point t, J(x,y) is the pixel value, the pixel value of the lesion area is 1, and the pixel value of the background area is 0, R t is the lesion area mask at time point t. This area consists of pixels (2D) or voxels (3D) and indicates the location of the lesion area.

[0164] This process provides quantitative information on the development of lesions, helps transform qualitative analysis into quantitative analysis, provides a quantitative evaluation basis for disease development or treatment effects, compares volume change trends under multimodal imaging, and provides support for accurate diagnosis of diseases and optimization of treatment plans.

[0165] Calculate the volume change rate G of the lesion area at each time t t :

[0166]

[0167] Where V t-1 is the lesion volume at time point t-1;

[0168] The volume change rate is used to quantify the growth or shrinkage rate of the lesion over time and screen the areas with significant changes. The introduction of the dynamic change rate can effectively eliminate the influence of noise, capture the true dynamic characteristics of the lesion, quickly locate the lesion area with significant changes, prioritize the marking of areas of clinical significance, accurately distinguish between disease progression and stable state, and optimize the physician's diagnostic process;

[0169] A threshold m is set based on the statistical distribution of the lesion volume change rate in the historical case data, and regions with a change rate greater than m are selected as lesion regions with significant changes.

[0170] Initialize a heat map matrix with the same size as the image, initialize all pixel values ​​to 0, traverse all time points, gradually superimpose the significant change areas on the heat map to generate a multi-time point lesion area heat map, and normalize the generated heat map to generate a heat map matrix that intuitively displays the dynamic change characteristics of the lesion over time, provides spatial distribution information of disease progression, and facilitates physicians to quickly locate key change areas in complex lesions. Normalize the generated heat map to make the distribution of significant change values ​​more balanced. The normalization process eliminates the difference in numerical dimensions, improves the visualization effect of the heat map, and increases the contrast of dynamic change information at multiple time points, making subtle changes more obvious, making it easier to superimpose the heat map with the original image to generate clearer visualization results.

[0171] By analyzing the time series changes of the regional masks and generating heat maps of the lesion regions at multiple time points, the dynamic changes of the lesions can be accurately quantified and intuitively visualized. This method not only plays an important role in dynamic monitoring and significance screening, but also provides a powerful tool for clinical diagnosis through the generated heat maps.

[0172] Furthermore, performing visual display refers to color mapping of the heat map. The heat map converts the lesion significance information into an intuitive color gradient image, providing physicians with an efficient visualization tool for dynamically changing areas. The application of color mapping can significantly enhance the contrast of the lesion change area, facilitate rapid positioning and identification of lesions in complex backgrounds, and improve diagnostic efficiency. The heat map is weightedly superimposed with the original image to display the heat map area on the original image. The weighted superposition provides physicians with a more complete diagnostic view by integrating the significance information of the heat map and the anatomical structure details of the original image. The flexible setting of weights can highlight the heat map information or retain more original image details according to actual needs to achieve targeted data display. The heat map area is marked and highlighted. The marking and highlighting functions can help physicians further focus on the core area of ​​the lesion to avoid missing key information in complex images. Highlighting can enhance the visualization of the lesion through color or borders to maintain consistent readability on different display devices. The generated heat map image is saved as an image file and stored in a database. The storage of the heat map persists the diagnostic results of the dynamically changing area, which is convenient for subsequent case tracking and analysis.

[0173] Through color mapping, weighted overlay, labeling and highlighting technologies, more efficient and intuitive support is provided for the interpretation of medical imaging data. The dynamic generation and overlay of heat maps not only improves the physician's diagnostic experience, but also provides a more information-dense visual representation of dynamically changing lesion areas. At the same time, database storage ensures the security, traceability and long-term utilization value of data. While improving the efficiency of medical image processing, this solution also provides strong technical support for clinical diagnosis and research.

[0174] Example 2, reference Figure 2 , which is the second embodiment of the present invention, which is different from the previous embodiment and provides a medical image intelligent processing system based on deep learning, comprising:

[0175] An acquisition module, used to acquire multimodal images using medical imaging equipment and perform preprocessing;

[0176] An image registration module is used to perform rough registration and precise registration on the pre-processed image to obtain a registered medical image;

[0177] The fusion optimization module is used to perform multi-modal fusion and optimization on the registered images to obtain high-quality fused images;

[0178] The region marking module is used to calculate the significance score of each image block after image segmentation of the fused image, mark the candidate lesion region according to the significance score, and generate a region mask corresponding to the candidate lesion region;

[0179] An analysis module, used to perform time series analysis based on the mask and generate a heat map of the lesion area at multiple time points;

[0180] The storage module is displayed, which is used to perform color mapping on the generated heat map, weighted overlay it with the original image, and store the heat map image in the database.

[0181] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0182] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0183] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0184] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

Claims

1. A medical image intelligent processing method based on deep learning, characterized in that: include, Collect and preprocess multimodal medical imaging data, register medical images of different modalities, fuse and optimize the registered multimodal images; The optimized fused image is segmented into image blocks and a significance score is assigned to each block. The candidate lesion area is marked based on the significance score and a regional mask is generated. Time series change analysis of regional masks was performed to generate heat maps of lesion areas at multiple time points and visualize them.

2. The medical image intelligent processing method based on deep learning according to claim 1, characterized in that: The collecting and preprocessing of multimodal medical imaging data and registering medical images of different modalities refers to using medical imaging equipment to collect multimodal imaging data, adding a timestamp to each frame of the image, and adjusting the image resolution to a uniform scale after normalizing the image pixel values; The multimodal imaging data includes CT, MRI and PET; Select CT modality as reference image and other modalities as target image, divide the pixel values ​​of reference image and target image into K intervals, count the joint occurrence frequency of corresponding pixel values ​​of reference image and target image to construct joint histogram; Normalize the joint histogram to a joint probability distribution, calculate the marginal probability distribution of the reference image and the target image respectively, and calculate the mutual information F based on the joint probability distribution and the marginal probability distribution; Define the optimization goal as maximizing mutual information, initialize the rotation parameters and translation parameters, construct the initial transformation matrix based on the initialization parameters, apply the initial transformation matrix to the target image, generate the transformed target image, and calculate the mutual information based on the pixel value distribution of the target image and the reference image; Use the grid search method to update the transformation matrix parameters, take the parameter combination with the maximum mutual information as the candidate for the next step, iteratively optimize the parameter combination until the maximum number of iterations is reached, and obtain the optimized optimal transformation matrix based on the iteratively optimized parameter combination; The target image transformed by the optimal transformation matrix is ​​used as the rough registration result; Use edge detection algorithm to extract the edges of reference image and target image, calculate the centroid of edge pixels of reference image and target image and select evenly distributed points to form reference point set and target point set; Calculate the centroid of the reference point set and the target point set, de-centroid the point set, use the de-centroided point set to construct the covariance matrix, perform singular value decomposition on the covariance matrix and calculate the rotation matrix, and calculate the translation vector based on the centroid of the rotated point set; Apply rotation and translation transformations to each point in the target point set to obtain an updated target point set; Calculate the mean square error between the updated target point set and the reference point set, update the rotation matrix and translation vector through multiple iterations, set the maximum number of iterations, and stop the iteration when the maximum number of iterations is reached to obtain the final rotation matrix and translation vector; The final rotation matrix and translation vector are used to fully align the target image to the reference image, generating the final registration result.

3. The medical image intelligent processing method based on deep learning as claimed in claim 2, characterized in that: The fusion and optimization of the registered multimodal images refers to extracting multimodal image pairs from the registered multimodal images, representing modality I1 and modality I2 respectively, dividing the images into image patches of size p*p using a sliding window method, extracting all patches and expanding them into vector form to form a feature matrix: Where X1 and X2 represent the patch feature matrices of modalities I1 and I2, respectively, and M is the total number of image patches; initialize the dictionary matrices D1 and D2 and the sparse representation matrices A1 and A2 for each modality; The optimization objective function is defined as: In the formula, ‖·‖ F is the Frobenius norm; The constraints for defining the objective function are: supp(A1)=supp(A2), Where supp() represents the support set of the sparse matrix; For each mode, fix D1 and D2, update A1 and A2, and solve the sparse representation matrix to obtain the updated sparse representation matrix A′ i ; Fix A1 and A2, optimize D1 and D2, and update the dictionary matrix to get D′ i ; After each update, the reconstruction error is calculated and the error threshold S is set. If the error is less than the threshold S, the iteration is stopped; Based on the optimized dictionary and sparse representation, calculate the correlation features shared between modalities: Z1=D1A1,Z2=D2A2, Where Z1 and Z2 are correlation feature matrices; Remove the correlation features from the original features to obtain the independence features: E1=X1-Z1,E2=X2-Z2, Where E1 and E2 are the independence feature matrices, X1 and X2 represent the patch feature matrices of modalities I1 and I2 respectively; Calculate the information entropy H(Z) of the correlation feature matrices Z1 and Z2 i ); Dynamically assign weights b1 and b2 based on information entropy; According to the weight, Z1 and Z2 are fused to generate the low-frequency component F l (x,y); The maximum selection rule is used to fuse the independent features, retain the significant eigenvalues ​​of each mode, and generate the fused high-frequency component F h (x,y); The fused low-frequency component and high-frequency component are combined to generate a preliminary fused image F f (x,y) and normalize the fused image; The global contrast intensity of the image is calculated by the grayscale histogram and the clarity is measured by the mean value of the image gradient intensity. The quality evaluation index R of the fused image is constructed based on the contrast W and clarity X as the optimization target. Use the Logistic Map model to initialize the contrast enhancement coefficient and noise suppression coefficient; Map the current parameter value output by the Logistic Map model to the actual range of the optimized parameter; The preliminary fusion image is dynamically optimized through pulse coupled neural network.

4. The medical image intelligent processing method based on deep learning as claimed in claim 3, characterized in that: The segmentation of the optimized fused image into image blocks and the allocation of a significance score to each block refers to using a sliding window method to divide the optimized fused image into non-overlapping image blocks, performing linear interpolation on the edge area of ​​the image, and calculating the local mean of the image blocks. and local gradient The saliency score e of each image patch is calculated based on the local mean and local gradient: Where N is the image block B j The number of pixels, is the image block B j The gradient of the jth pixel in , is the image block B j The local mean, μ max is the maximum local mean of all blocks.

5. The medical image intelligent processing method based on deep learning according to claim 4, characterized in that: The marking of the candidate lesion region based on the significance score and generating a region mask refers to setting a significance threshold U, marking image blocks with a significance score higher than the threshold as significant, combining all blocks with a significance score higher than the threshold into a candidate lesion region, generating a lesion region mask according to the significance score and the set threshold U, and marking the candidate lesion region.

6. The medical image intelligent processing method based on deep learning according to claim 5, characterized in that: The generating of the multi-time point lesion region heat map by performing time series change analysis on the region mask refers to obtaining candidate lesion region mask data at multiple time points, extracting the lesion volume corresponding to each time point, and calculating the lesion volume by accumulating the total number of pixels in the lesion region; Calculate the volume change rate G of the lesion area at each time t t ; Set a threshold m, and select areas with a change rate greater than m as lesion areas with significant changes; Initialize the heat map matrix with the same size as the image, initialize all pixel values ​​to 0, traverse all time points, gradually superimpose the significantly changed areas onto the heat map to generate a multi-time point lesion area heat map, and normalize the generated heat map.

7. The medical image intelligent processing method based on deep learning according to claim 6, characterized in that: The visual display refers to color mapping the heat map, weighted superposition of the heat map and the original image, displaying the heat map area on the original image, marking and highlighting the heat map area, saving the generated heat map image as an image file and storing it in a database.

8. A medical image intelligent processing system based on deep learning based on the medical image intelligent processing method based on deep learning according to any one of claims 1 to 7, characterized in that: include, An acquisition module, used to acquire multimodal images using medical imaging equipment and perform preprocessing; An image registration module is used to perform rough registration and precise registration on the pre-processed image to obtain a registered medical image; The fusion optimization module is used to perform multi-modal fusion and optimization on the registered images to obtain high-quality fused images; A region marking module is used to calculate the significance score of each image block after image segmentation of the fused image, mark the candidate lesion region according to the significance score, and generate a region mask corresponding to the candidate lesion region; An analysis module, used to perform time series analysis based on the mask and generate a heat map of the lesion area at multiple time points; The storage module is displayed, which is used to perform color mapping on the generated heat map, weighted overlay it with the original image, and store the heat map image in the database.

9. A computer device comprising: A memory and a processor; the memory stores a computer program, characterized in that: when the processor executes the computer program, the steps of the medical image intelligent processing method based on deep learning described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the medical image intelligent processing method based on deep learning described in any one of claims 1 to 7 are implemented.

Images