Multimodal image threshold segmentation preprocessing method based on convolutional neural network

Abstract

This invention relates to a multimodal image threshold segmentation preprocessing method based on a convolutional neural network. The method unifies images into a standard space, performs pixel value mapping and resampling to generate high- and low-frequency subbands. High-frequency subbands are subjected to soft-threshold denoising, low-frequency subbands are contrast-enhanced, and then fused. Image quality is assessed and labeled, an optimized VGGnet framework is constructed, and a noise adversarial network is generated to perform active learning and cyclic training of the convolutional neural network model. The image is input into the model for prediction. Local entropy and gradient amplitude are calculated based on the prediction results. The fusion weights are dynamically adjusted to obtain an adaptive threshold, and pixel-level binary segmentation is performed. Optimal segmentation is achieved by setting a two-layer matrix for the feature label-segmentation method. Gray matter Dice calculation is performed on the segmented images. For unqualified images, preprocessing parameters are optimized using a dynamic parameter adjustment module based on a Gaussian process regression model. This method effectively improves the segmentation accuracy and processing efficiency of multimodal images and enhances the model's adaptability to complex images.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a multimodal image threshold segmentation preprocessing method based on a convolutional neural network. Background Art

[0002] Preprocessing of multimodal medical imaging data, including 3D-T1, DTI, and BOLD-fMRI, is crucial in neuroimaging research. Currently, commonly used fixed-threshold segmentation tools, such as FSL's FAST, SPM's VBM, and FreeSurfer, can perform tissue segmentation in most high-quality images. However, these MRI sequences, which are sensitive to magnetic field uniformity, have inherently low signal-to-noise ratios and long scan times, can increase the likelihood of localized noise interference and image quality degradation, leading to various effects such as head motion artifacts and geometric distortion.

[0003] Fixed threshold segmentation, however, cannot adapt to variations such as local noise and geometric distortion, resulting in blurred segmentation boundaries and prone to misclassification, leading to data loss and increased research costs. Furthermore, traditional methods rely on manual threshold adjustment or empirical parameter settings, lacking real-time feedback on image quality. While some improved approaches exist, such as adaptive thresholding and morphological post-processing, they all have their limitations, are low-level, and lack robustness.

[0004] This method provides an adaptive, real-time feedback solution. It uses a convolutional neural network to build an autonomous closed-loop optimization system. By training the model to process graphics under different quality conditions and deep learning capabilities, it provides an optimization solution to meet the accuracy required for scientific research and improve image quality. Summary of the Invention

[0005] The purpose of the present invention is to provide a multimodal image threshold segmentation preprocessing method based on convolutional neural networks, use convolutional neural networks to build an autonomous closed-loop optimization system, and provide an optimization solution by training the model to process graphics under different quality conditions and deep learning capabilities, thereby meeting the accuracy required for scientific research and improving image quality.

[0006] In order to solve the above technical problems, the technical solutions adopted by the present invention are as follows:

[0007] The multimodal image threshold segmentation preprocessing method based on convolutional neural network includes the following steps:

[0008] S1: Unify the multimodal medical imaging data into a standard space, map the multimodal medical imaging pixel values ​​to a specified interval, resample them, and generate a low-frequency sub-band and multiple high-frequency sub-bands through multi-scale decomposition;

[0009] S2: Perform soft threshold denoising on the high-frequency sub-band, divide the low-frequency sub-band into multiple sub-regions and calculate the cumulative distribution function value for each sub-region, generate a grayscale mapping function and apply it to the original pixel value, transform and fuse the low-frequency sub-band with the high-frequency sub-band;

[0010] S3: Perform quality assessment and labeling of multimodal medical images, calculate the K coefficient of each image, agree on the labeling of images with K coefficients within a first preset range, conduct third-party arbitration for images with K coefficients within a second preset range, and output the labeling as the arbitration result;

[0011] S4: Build a convolutional neural network model and perform optimization and modification, generate a noise adversarial network for active learning cycles, and train the convolutional neural network model;

[0012] S5: Input a specified portion of multimodal medical images into the trained convolutional neural network model for prediction, divide the prediction results into multiple local regions, calculate the local entropy and gradient amplitude of each region, dynamically adjust the fusion weight of the local entropy and gradient, and use an adaptive weighted fusion algorithm to calculate the adaptive threshold;

[0013] S6: Based on the adaptive threshold, perform pixel-level binary segmentation on the multimodal medical image, set the two-layer matrix of feature label-segmentation method, and complete the threshold segmentation of the original image according to the optimal solution;

[0014] S7: Perform gray matter Dice calculation on the multimodal medical image after threshold segmentation, and judge whether the multimodal medical image is qualified based on the gray matter Dice result. The qualified image is directly output, and the unqualified image is input into the dynamic parameter adjustment module for processing.

[0015] Preferably, the specific process of unifying the multimodal medical image data into a standard space and mapping the multimodal medical image pixel values ​​into a specified interval and then performing resampling in step S1 is as follows:

[0016] S11: Acquire multimodal medical imaging data including 3D-T1, DTI, and fMRI, and unify them into a standard space;

[0017] S12: setting the target floating-point coordinates of the multimodal medical image, including the integer part and the decimal part, and performing bilinear interpolation processing on the image;

[0018] When the target point is located between two pixels, the pixels in the target image are mapped to the floating-point coordinates of the original image, and a 4×4 area centered on the specified pixel in the original image is taken;

[0019] S13: For each surrounding point, calculate its offset distance in the x and y directions, where the surrounding points refer to the four nearest neighboring pixels around the target floating point coordinate in the original image when performing bilinear interpolation;

[0020] S14: Calculate weights using the cubic convolution function.

[0021] Preferably, the specific process of generating a low-frequency sub-band and multiple high-frequency sub-bands by multi-scale decomposition in step S1 is as follows:

[0022] S15: Create a two-dimensional image matrix of the multimodal medical image, and perform convolution operations using a low-pass filter and a high-pass filter in the row direction. After the convolution, two matrices are obtained, one containing low-frequency information in the row direction and the other containing high-frequency information in the horizontal direction;

[0023] S16: The result of the row-wise convolution is convolved with the low-pass filter and the high-pass filter in the column direction to obtain four sub-bands:

[0024] Low-frequency subband: contains the low-frequency information and contours of the image;

[0025] Horizontal high-frequency sub-band: contains the vertical edge information of the image;

[0026] Vertical high-frequency sub-band: contains the horizontal edge information of the image;

[0027] Diagonal high-frequency subband: contains high-frequency details and noise in the diagonal direction of the image;

[0028] S17: Repeat the steps of single-layer two-dimensional DWT decomposition for the obtained low-frequency subband to obtain subbands of more scales. With each layer of decomposition, the size of the low-frequency subband becomes 1 / 4 of the size of the previous layer of low-frequency subband, and the size of the high-frequency subband remains the same as before the decomposition of the corresponding low-frequency subband.

[0029] Among them, single-layer two-dimensional DWT refers to the process of two-dimensional discrete wavelet transform of low-pass filter and high-pass filter;

[0030] S18: Based on the single-layer two-dimensional DWT decomposition, a filtering operation in the depth direction is added. Low-pass filters and high-pass filters are used for convolution in the row direction, column direction, and depth direction respectively. One three-dimensional DWT decomposition can obtain 8 sub-bands, including 1 low-frequency sub-band and 7 high-frequency sub-bands. Multi-layer three-dimensional DWT decomposition continuously repeats the S18 operation on the low-frequency sub-band to obtain sub-band information of more scales.

[0031] Among them, the three-dimensional DWT refers to adding a depth-direction filtering operation on the basis of a single-layer two-dimensional DWT. It is essentially a cube decomposition of separable filtering. Through triple filtering of rows, columns, and depth, the 3D data is decomposed into 8 sub-bands with complementary directional features.

[0032] Multi-layer 3D DWT refers to recursively performing 3D DWT decomposition on the low-frequency subbands of the previous layer to form a pyramid-like multi-scale feature representation.

[0033] Preferably, the specific process of step S2 is as follows:

[0034] S21: Based on Calculate the threshold, where σ is the noise standard deviation, σ≈ 1.4826 · MAD, MAD is the median absolute deviation of the high-frequency subband, N is the total number of pixels in the high-frequency subband, and for each coefficient in the high-frequency subband, a preset soft threshold function is applied to process it, and the coefficients with absolute values ​​less than the threshold are set to zero, and the coefficients greater than the threshold are shrunk to zero. λ ;

[0035] S22: Divide the low-frequency sub-band into multiple non-overlapping sub-regions, where the size of the sub-regions is set to 8×8. For each sub-region, count the distribution of pixel values ​​therein and calculate the cumulative distribution function value.

[0036] S23: Multiplying the cumulative distribution function value by the total number of gray levels and performing a rounding operation to obtain a mapped gray value, thereby obtaining a mapping relationship from the original gray value to the new gray value, thereby forming a gray mapping function;

[0037] S24: traverse each sub-region of the low-frequency sub-band, and replace the original grayscale value of each pixel in the sub-region with the mapped grayscale value according to the generated grayscale mapping function, thereby completing the contrast enhancement of the low-frequency sub-band;

[0038] S25: For the two-dimensional image, perform inverse filtering in the column direction and then inverse filtering in the row direction. For the three-dimensional image, perform inverse filtering operations in the three dimensional directions in sequence to obtain a preliminary fused image.

[0039] S26: According to local features of the image including local variance and gradient, different weights are assigned to the low-frequency sub-band and the high-frequency sub-band, and fusion optimization is performed using a weighted fusion method.

[0040] Preferably, the specific process of step S3 is as follows:

[0041] S31: Perform quality assessment on multimodal medical images and add labels including noise level and artifact type. The noise level is set to 5 dimensions and the artifact type is set to N types.

[0042] S32: Each image was quality-assessed and classified by two experts, and the K coefficient was calculated. The calculation included the observed consistency Po, that is, the actual observed agreement ratio, and the random consistency Pe;

[0043] When 0.8 < K < 1: Agree with the corresponding label. That is, when the assessment of the image quality by two experts falls within the first preset range of 0.8 < K < 1, the system automatically adopts the label marked by the expert as the final result without third-party arbitration. The K coefficient is the Kappa coefficient.

[0044] When -1 < K < 0.8: Introduce a third expert for arbitration, and the label output is the arbitration result.

[0045] Preferably, in step S4, the specific process of constructing a convolutional neural network model and optimizing and modifying it to generate a noise adversarial network for active learning loop is as follows:

[0046] S41: Construct an improved VGGnet framework for optimization and modification:

[0047] Input the multi-modal stacked data into the multi-scale feature extraction module of the improved VGGnet framework to perform multi-scale feature extraction to obtain multi-scale feature maps;

[0048] Input the multi-scale feature maps into the channel attention mechanism module of the improved VGGnet framework, perform global average pooling and global max pooling operations on the multi-scale feature maps, obtain the statistical information of the channel dimension, and use a multi-layer perceptron containing two fully connected layers for non-linear transformation to generate channel attention weights;

[0049] Perform two-dimensional convolution operation through the spatial attention mechanism module to generate a spatial attention map;

[0050] S42: Use GAN to generate a noise adversarial network for active learning loop:

[0051] For pixel-level loss: Constrain the local consistency between the generated noise and the real noise, and set the adversarial loss formula and gradient penalty formula.

[0052] For frequency-domain loss: Constrain the noise spectrum distribution through FFT, and the perceptual loss uses the pre-trained and improved VGGnet framework to extract features to maintain semantic consistency;

[0053] Import the generated noise image into the training set, simulate various noise patterns in the training data, and enhance the generalization ability of the model;

[0054] Perform model training, set the training samples, initial learning rate and number of iterations, construct a complete CNN large model, and when inputting an image picture, output the feature label corresponding to the image picture.

[0055] Preferably, the specific process of step S5 is as follows:

[0056] S51: Input a specified portion of the multimodal medical image into the trained convolutional neural network model for prediction. Based on the prediction results, the multimodal medical image is divided into multiple non-overlapping local regions. When the image size cannot be divided by the region size, the edge region is zero-filled or mirrored to ensure that all pixels are contained in a certain region.

[0057] S52: For each local area, a histogram of pixel values ​​is counted, and local entropy is calculated based on the histogram;

[0058] S53: using the Soble operator to calculate the gradient magnitude of each pixel in the region;

[0059] S54: Calculate the mean and standard deviation of the gradient amplitude in the region as the gradient feature of the corresponding region;

[0060] S55: Evaluate the feature complexity of the corresponding region based on local entropy and gradient information;

[0061] S56: Adjust the fusion weight of local entropy and gradient according to the complexity of regional features;

[0062] S57: For each region, calculate the threshold based on local entropy, use the mean and standard deviation of the gradient amplitude in the region to calculate the threshold, and perform weighted fusion to obtain the final adaptive threshold. Gaussian filtering is performed on all regional thresholds to eliminate threshold mutations between regions. The regional thresholds are mapped back to the original image coordinates to obtain the final adaptive threshold map.

[0063] Preferably, the specific process of step S6 is as follows:

[0064] S61: for each pixel in the multimodal medical image, compare its pixel value with the adaptive threshold value. For a pixel with coordinates (x, y) in the multimodal medical image, if its pixel value I(x, y) is greater than or equal to the adaptive threshold value T(x, y), mark the pixel as foreground and assign a value of 1; if the pixel value I(x, y) is less than the adaptive threshold value T(x, y), mark the pixel as background and assign a value of 0.

[0065] S62: Based on the noise level and artifact type, a first-layer 5×N matrix is ​​constructed, where each coordinate of the matrix corresponds to a specified noise-artifact combination feature label, and the image is mapped to the corresponding coordinate position of the feature label matrix based on the added label;

[0066] S63: constructing a second-layer 5×N matrix of the same dimension as the first-layer 5×N matrix, wherein each coordinate of the matrix corresponds one-to-one to a coordinate of the first-layer 5×N matrix, and each coordinate position stores a segmentation method for the corresponding noise-artifact combination, each segmentation method including a different threshold adjustment strategy or a combination of morphological operations or a post-processing algorithm based on machine learning;

[0067] S64: Determine, based on the multimodal medical image label, a corresponding coordinate position in the first layer 5×N matrix, and obtain, based on the coordinate position determined in the first layer 5×N matrix, a corresponding segmentation method at the same coordinate position in the second layer 5×N matrix;

[0068] S65: Apply the acquired segmentation method to the binarized image:

[0069] If the segmentation method is threshold fine-tuning, the adaptive threshold used in binarization is adjusted according to the rules set in the method, and then the binarization operation is performed again;

[0070] If the segmentation method includes morphological operations, the specified structure element is used to process the binary image to remove small noise areas, fill internal holes, and optimize the segmentation results;

[0071] If the segmentation method is a post-processing algorithm based on machine learning, the binary image is input into the corresponding model to obtain the final segmentation result;

[0072] Multimodal medical images are segmented into pixel-level binarization based on adaptive thresholds, a two-layer matrix of feature labels and segmentation methods is set, and image threshold segmentation is completed according to the optimal solution.

[0073] Preferably, the specific process of inputting the unqualified image into the dynamic parameter adjustment module for processing in step S7 is as follows:

[0074] S71: Import the unqualified image into the Gaussian process regression model for nonlinear mapping calculation;

[0075] S72: Simplify the regression model by setting the mean function to zero. The observation data y is generated by the nonlinear mapping function f(x) between the input features and the parameters plus Gaussian noise: , ;

[0076] in, y represents observation data; Represents the nonlinear mapping function between input features and parameters; x is the input feature vector; represents expert annotation error or inherent image noise; Refers to the robustness of the control model to noise;

[0077] S73: Kernel Function Implicitly defines a mapping from the input space to a high-dimensional feature space ,make Select a nonlinear kernel function so that the model can capture the nonlinear relationship between inputs;

[0078] Among them, x and are two eigenvectors in the input space; It is a nonlinear mapping function that maps the input vector from the original space to the high-dimensional feature space; It is a kernel function that directly calculates the two mapped vectors and The inner product of

[0079] S74: Construct the covariance matrix:

[0080] ;

[0081] in, Refers to the sample x i and x j is the kernel function of the independent variable, used to calculate the input sample x i and x j The similarities between refers to the robustness of the control model to noise, is the Kronecker delta function, when i = j 1 when it is, otherwise 0;

[0082] Compute the marginal likelihood:

[0083] ;

[0084] in, y represents observation data; X is the input feature matrix, each row corresponds to the eigenvector of a sample, and K is the covariance matrix; n is the sample size, related to the normalization of the log-likelihood;

[0085] S75: Command input x ∈ R d , the RBF kernel is implicitly mapped to infinite-dimensional space;

[0086] S76: Maximizing the Log-Marginal Likelihood via Gradient Descent:

[0087] ;

[0088] in, y represents the observed data, θ is the hyperparameter vector to be optimized, K is the covariance matrix; is the partial derivative of the covariance matrix with respect to the hyperparameters, and tr is the summation of all elements on the main diagonal of the matrix;

[0089] Adaptively select and adjust the parameters of the preprocessing method, including dynamic adjustment of the segmentation threshold, smoothing kernel size, and number of registration iterations.

[0090] The beneficial effects of the present invention include:

[0091] The present invention provides a multimodal image threshold segmentation preprocessing method based on a convolutional neural network. The method unifies images into a standard space, maps pixel values, and resamples them to generate high- and low-frequency subbands. The high-frequency subbands are subjected to soft-threshold denoising, and the low-frequency subbands are contrast-enhanced and then fused. Image quality is assessed and labeled, an optimized VGGnet framework is constructed, a noise adversarial network is generated for an active learning loop, and a convolutional neural network model is trained. The image is input into the model for prediction. Based on the prediction results, local entropy and gradient amplitude are calculated. The fusion weights are dynamically adjusted to obtain an adaptive threshold, and pixel-level binary segmentation is performed. Optimal segmentation is achieved by setting a two-layer matrix of feature labels and segmentation methods. Gray matter Dice calculation is performed on the segmented images, and preprocessing parameters for unqualified images are optimized using a dynamic parameter adjustment module based on a Gaussian process regression model. This method effectively improves the segmentation accuracy and processing efficiency of multimodal medical images and enhances the model's adaptability to complex images.

[0092] First, by adopting a resampling technology that combines bilinear interpolation and bicubic interpolation, interpolation artifacts are reduced while maintaining image resolution. In particular, the weight calculation of the cubic convolution function, including parameter adaptive selection, significantly improves the fidelity of image details after resampling.

[0093] Second, through low-frequency sub-band contrast enhancement + high-frequency sub-band soft threshold denoising, an organic combination of spatial domain contrast enhancement and frequency domain noise suppression is achieved. Compared with the traditional single preprocessing method, it can more effectively preserve edge details and suppress complex noise.

[0094] Third, we introduced two independent experts to independently label the data, coupled with a K-factor verification mechanism. This ensured the reliability of noise level and artifact type labels by calculating quantitative metrics of observational consistency and random consistency. When consistency was insufficient, a third expert was introduced to arbitrate, effectively reducing the subjective bias of manual labeling and providing high-quality labeled data for subsequent model training.

[0095] Fourth, the VGGnet model is improved by combining a multi-scale feature extraction module with a dual-channel attention mechanism (channel attention CAM + spatial attention SAM), enabling the model to capture structural features of different scales in multimodal medical images, and focus on key areas through weight distribution to enhance the richness of feature expression.

[0096] Fifth, GAN generates an active learning loop of the noise adversarial network. By designing a joint optimization objective of pixel-level loss, adversarial loss, gradient penalty, frequency domain loss, and perceptual loss, it forces the generator to simulate the spatial distribution, spectral characteristics, and semantic consistency of real noise, significantly enhancing the model's generalization ability in low-quality images, especially for clinical data with diverse noise types.

[0097] Sixth, a dynamic weighted fusion algorithm based on local entropy and gradient amplitude automatically adjusts the fusion weight through the regional feature complexity evaluation index (combining grayscale variance, texture entropy, and edge density), so that the threshold calculation can adapt to the local characteristics of the image (such as high-texture areas vs. smooth areas), avoiding the one-size-fits-all defect of the global threshold and improving segmentation accuracy.

[0098] Seventh, the design of a two-layer matrix of feature labels and segmentation methods directly links the noise level and artifact type to the optimal segmentation strategy, realizing the intelligent matching of "image features and processing methods". Compared with fixed parameter segmentation, it effectively improves processing efficiency by more than 30%, and the Dice coefficient of the segmentation results is improved by an average of 15%.

[0099] Eighth, based on the nonlinear mapping mechanism of the Gaussian process regression model, the implicit correlation between preprocessing parameters and segmentation quality indicators is captured through kernel functions (such as the RBF kernel), and the processing parameters of unqualified images are automatically optimized to form a closed-loop feedback of "segmentation-evaluation-parameter adjustment", which greatly reduces the cost of manual parameter adjustment and is particularly suitable for large-scale image batch processing scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0100] Figure 1 Schematic diagram of the process of multimodal image threshold segmentation preprocessing method based on convolutional neural network of the present invention.

[0101] Figure 2 Schematic diagram of the improved VGGnet framework of the present invention. DETAILED DESCRIPTION

[0102] The following is combined with Figures 1 - 2 The present invention is further described in detail. In the embodiment, brain medical images of three modalities, 3D-T1, DTI, and fMRI, are processed as the processing objects, and the specific implementation process of the multimodal medical image threshold segmentation preprocessing method based on convolutional neural network is elaborated in detail.

[0103] Example 1

[0104] See attached Figure 1 As shown in FIG, the multimodal image threshold segmentation preprocessing method based on convolutional neural network includes the following steps:

[0105] S1: Acquire multimodal medical imaging data from the hospital's PACS system, unify the multimodal medical images into a preset standard space, and resample them. Map the pixel values ​​in the multimodal medical images to a specified interval and perform multi-scale decomposition to generate a low-frequency sub-band and multiple high-frequency sub-bands.

[0106] S11: Acquire multimodal medical imaging data including 3D-T1, DTI, and fMRI, and unify them into a standard space using the ITK library;

[0107] S12: Set the target floating-point coordinates of the multimodal medical image to (x, y), where the integer part is ( i , j ), the decimal part is ( u , v ), u , v ∈[0,1), perform bilinear interpolation processing on the image, and the interpolation formula is as follows:

[0108] ;

[0109] in,( i , j ) is the integer part, which represents the upper left corner reference coordinates of the 4 pixels around the target point; ( u, v ) is the decimal part, which represents the offset between the target point and the reference coordinate. ; Indicates that the coordinates in the original image are ( i , j ) pixel value. is the upper left pixel The weight of is the upper right pixel The weight of is the lower left pixel The weight of is the lower right pixel The weight of .

[0110] When the target point is located between two pixels, the pixel in the target image is mapped to the floating point coordinates (x, y) of the original image, where x = i + u , y = j + v , take the original image with ( i ,j ) as the center of the 4 × 4 area, that is, from ( i −1, j −1) to ( i +2, j +2) 16 pixels;

[0111] S13: For each surrounding point ( i + m , j + n ), m, n∈{−1,0,1,2}, calculate its x and y Direction offset distance dx and dy , dx = m - u , dy = n - v ;

[0112] S14: Use the cubic convolution function to calculate the weights w(dx) and w(dy),

[0113] ;

[0114] When w(dx)=0, a =-0.5, indicating a balance between smoothness and sharpness, a=-0.75 indicates enhanced edge contrast, and the pixel value of each surrounding point is multiplied by the corresponding weight w(dx)×w(dy), and the pixel value of the target point is accumulated. The pixel value P of the target point is the weighted sum of the 16 neighboring points:

[0115] , make sure the image pixel size is 2,192×192.

[0116] Set the target image size to 192×192 and perform a resampling operation combining bilinear interpolation and bicubic interpolation on the image. Taking a 2D slice in a 3D-T1 image as an example, for the target floating-point coordinates (50.3, 60.7), the integer part (i, j) is (50, 60) and the decimal part (u, v) is (0.3, 0.7). When performing bilinear interpolation, according to the formula Calculate the initial pixel value. Next, take a 4×4 region centered at (50, 60) in the original image and calculate the offset distance in the x and y directions for each surrounding point. Use a cubic convolution function (with a = −0.5) to calculate the weight. Multiply the pixel values ​​of the surrounding points by the corresponding weights and add them together to obtain the final pixel value of the target point, completing the resampling.

[0117] The specific process of generating a low-frequency sub-band and multiple high-frequency sub-bands through multi-scale decomposition in step S1 is as follows:

[0118] S15: Create a two-dimensional image matrix of the multimodal medical image and perform convolution operations on it using a low-pass filter and a high-pass filter in the row direction. After convolution, two matrices are obtained: one contains the low-frequency information in the row direction ILH (row low-pass, column high-pass), and the other contains the high-frequency information in the horizontal direction IHH (row high-pass, column high-pass);

[0119] S16: The result of the row-wise convolution is convolved with the low-pass filter and the high-pass filter in the column direction to obtain four sub-bands:

[0120] Low-frequency subband: contains the low-frequency information and contours of the image;

[0121] Horizontal high-frequency sub-band: contains the vertical edge information of the image;

[0122] Vertical high-frequency sub-band: contains the horizontal edge information of the image;

[0123] Diagonal high-frequency subband: Contains high-frequency details and noise in the diagonal direction of the image.

[0124] The resampled two-dimensional image matrix is ​​convolved with a low-pass filter and a high-pass filter in the row direction to obtain a matrix containing low-frequency information and high-frequency information in the row direction; then convolved in the column direction to obtain a low-frequency sub-band LL, a horizontal high-frequency sub-band LH, a vertical high-frequency sub-band HL, and a diagonal high-frequency sub-band HH.

[0125] S17: Repeat the above single-layer 2D DWT decomposition steps for the obtained low-frequency subbands to obtain subbands of multiple scales. With each decomposition layer, the size of the low-frequency subbands becomes 1 / 4 of the size of the previous low-frequency subband, while the size of the high-frequency subbands remains the same as before the decomposition. A single-layer 2D DWT refers to the 2D Discrete Wavelet Transform (DWT) process using low-pass and high-pass filters.

[0126] S18: Based on the single-layer two-dimensional DWT decomposition, a filtering operation in the depth direction is added. Low-pass filters and high-pass filters are used for convolution in the row direction, column direction, and depth direction respectively. One three-dimensional DWT decomposition can obtain 8 sub-bands, including 1 low-frequency sub-band and 7 high-frequency sub-bands. Multi-layer three-dimensional DWT decomposition continuously repeats the S18 operation on the low-frequency sub-band to obtain sub-band information of more scales.

[0127] The 3D DWT adds depth-wise filtering to a single-layer 2D DWT. Essentially, it's a cubic decomposition with separable filtering: through triple filtering at the row, column, and depth levels, 3D data is broken down into eight subbands with complementary directional features. A multi-layer 3D DWT recursively repeats the 3D DWT decomposition on the low-frequency subbands of the previous layer, forming a pyramid-like multi-scale feature representation.

[0128] A multi-layer 2D DWT decomposition is performed on the low-frequency subband. With each decomposition layer, the size of the low-frequency subband is reduced to 1 / 4 of the previous layer. For example, the initial low-frequency subband size is 192×192, and after one layer of decomposition, it becomes 96×96. For 3D images, a depth-wise filtering operation is added to the single-layer 2D DWT decomposition. Each 3D DWT decomposition yields 8 subbands, including 1 low-frequency subband and 7 high-frequency subbands. This operation is repeated on the low-frequency subband to obtain information on subbands of higher scales.

[0129] S2: Perform soft threshold denoising on the high-frequency subband, divide the low-frequency subband into non-overlapping sub-regions, calculate the cumulative distribution function value for each sub-region, generate a grayscale mapping function based on the cumulative distribution function value and apply it to the original pixel value, and perform inverse DWT transformation and fusion processing on the low-frequency subband and high-frequency subband;

[0130] S3: Perform quality assessment on the fused multimodal medical images and add labels by classification. Calculate the Cohen's Kappa coefficient (K coefficient) of each image and label the images if the K coefficient is within the first preset range. If the K coefficient is within the second preset range, a third-party arbitration is introduced and the label output is the arbitration result.

[0131] S4: Build a convolutional neural network model to generate a noise adversarial network for active learning cycles and train the convolutional neural network model;

[0132] S5: Input the multimodal medical image with the specified portion of the label into the trained convolutional neural network model. The convolutional neural network model predicts the multimodal medical image, divides the prediction results into multiple local regions, calculates the local entropy and gradient amplitude of each region, dynamically adjusts the fusion weight of the local entropy and gradient based on the regional feature complexity evaluation index, and uses an adaptive weighted fusion algorithm to calculate the adaptive threshold;

[0133] S6: Based on the adaptive threshold, perform pixel-level binary segmentation on the multimodal medical image data, set the two-layer matrix of feature label-segmentation method, and complete the threshold segmentation of the original image according to the optimal solution;

[0134] S7: Perform gray matter Dice calculation on the multimodal medical image after threshold segmentation, and judge whether the multimodal medical image is qualified based on the gray matter Dice result. The qualified image is directly output, and the unqualified image is input into the dynamic parameter adjustment module for processing.

[0135] Example 2

[0136] Based on Example 1, the specific process of step S2 is as follows:

[0137] S21: Based on Calculate the threshold, where σ is the noise standard deviation, σ≈ 1.4826 · MAD, MAD is the median absolute deviation of the high-frequency subband, N is the total number of pixels in the high-frequency subband, and for each coefficient in the high-frequency subband, a preset soft threshold function is applied to process it, and the coefficients with absolute values ​​less than the threshold are set to zero, and the coefficients greater than the threshold are shrunk to zero. λ ;

[0138] S22: Divide the low-frequency subband into multiple non-overlapping sub-regions. The size of the sub-region is set to 8×8. For each sub-region, the distribution of pixel values ​​is counted and the cumulative distribution function is calculated. The range of pixel values ​​in the sub-region is set to [0, L-1], where L is the total number of gray levels. For an 8-bit grayscale image, L=256 and the pixel value is e The number of pixels is ne , the total number of pixels in the sub-region is N, then the cumulative distribution function CDF( e ) is calculated as follows:

[0139] ;

[0140] S23: CDF( e ) multiplied by the total number of gray levels L−1 and rounded to get the mapped gray value ye ,Right now ye =[CDF( e )·(L−1)], the mapping relationship from the original grayscale value to the new grayscale value is obtained, forming a grayscale mapping function;

[0141] S24: traverse each sub-region of the low-frequency sub-band, and replace the original grayscale value of each pixel in the sub-region with the mapped grayscale value according to the generated grayscale mapping function, thereby completing the contrast enhancement of the low-frequency sub-band;

[0142] S25: For the two-dimensional image, perform inverse filtering in the column direction and then inverse filtering in the row direction. For the three-dimensional image, perform inverse filtering operations in the three dimensional directions in sequence to obtain a preliminary fused image.

[0143] S26: According to local features of the image including local variance and gradient, different weights are assigned to the low-frequency sub-band and the high-frequency sub-band, and fusion optimization is performed using a weighted fusion method.

[0144] Taking a high-frequency subband of the 3D-T1 modality (such as the horizontal edge subband HL) as an example, assuming its size is 96×96 pixels (after the first layer of decomposition), the calculation process is as follows: Calculate MAD: extract the absolute values ​​of all coefficients of the high-frequency subband, sort them and take the median. Assume that MAD = 8.5.

[0145] Estimated noise standard deviation: σ≈1.4826×8.5=12.6.

[0146] Calculate the threshold λ: N = 96 × 96 = 9216, λ = 12.6 × (2 × log (9216)) 1 / 2 ≈12.6×5.24≈66.

[0147] Apply a soft threshold function: For each coefficient c:

[0148] If |c|<66, set to 0;

[0149] If c≥66, adjust to c-66;

[0150] If c≤-66, adjust to c+66.

[0151] Processing results: About 78% of the coefficients are set to zero, effectively suppressing random noise and retaining significant edge features.

[0152] The low frequency subband LL96×96 of 3D-T1 is divided into 8×8 sub-regions, a total of 144. i , j )=(10,15) for analysis:

[0153] Statistical pixel value distribution: Assume that the pixel value range in the area is [45,180], and the number of pixel values ​​n is e=100 100 =12.

[0154] Calculate CDF: The total number of pixels in the sub-region is N=64, .

[0155] S23: Generate grayscale mapping function:

[0156] Gray value y after mapping 100 =[0.5×255]=128.

[0157] Full mapping table: linearly maps [45,180] to [23,240] to enhance contrast.

[0158] Traverse all 144 sub-regions and apply the corresponding grayscale mapping function to each sub-region.

[0159] For example, a point with an original pixel value of 80 is mapped to 72, and 150 is mapped to 198, significantly improving the contrast between the overall brain tissue and the background.

[0160] Take the three-dimensional data of the DTI modality as an example (assuming the size is 96×96×64):

[0161] Inverse filtering operation: Apply 2D inverse DWT along the Z-axis direction to process each axial slice (a total of 64 slices);

[0162] Apply 1D inverse DWT to the result along the Y-axis direction;

[0163] Finally, complete the inverse transformation along the X-axis direction to restore the 3D data.

[0164] Processing effect: The reconstructed image restores the original structural information while retaining the denoising and enhancement effects.

[0165] S26: Weighted fusion based on local features: Calculate local variance: Calculate the variance of pixel values for each 8×8 region. For example, the variance in the cerebrospinal fluid region is about 120, and in the white matter region is about 450. Gradient calculation: Use the Sobel operator to calculate the gradient magnitude. The average gradient magnitude in the gray matter and white matter boundary region is 85, and in the uniform region is about 15. Weight assignment: High variance / high gradient region: Assign a higher weight to the high-frequency sub-band to retain details; low variance / low gradient region: Assign a higher weight to the low-frequency sub-band to reduce noise interference.

[0166] The specific process of step S3 is as follows:

[0167] S31: Perform quality assessment on multimodal medical images and add labels including noise level: 5 dimensions, and the artifact type is encoded as N types;

[0168] S32: Each image is evaluated and classified by two experts, and the K coefficient is calculated. The calculation includes the observed agreement Po: the actual observed proportion of agreement, and the chance agreement Pe: the expected proportion of agreement when the evaluators classify randomly; the formula is ;

[0169] When 0.8 < K < 1: Agree on the corresponding label, that is, when the quality assessment of the image by the two experts falls within the first preset range 0.8 < K < 1, the system automatically adopts the label marked by the experts as the final result without third-party arbitration. The K coefficient is the Kappa coefficient;

[0170] When -1 < K < 0.8: Introduce a third expert for arbitration, and the label output is the arbitration result.

[0171] Example 3

[0172] Based on Example 1 or Example 2, the specific process of constructing a convolutional neural network model and optimizing and modifying it in step S4 to generate a noise adversarial network for active learning loop is as follows:

[0173] S41: Construct an improved VGGnet framework and optimize and modify it. The improved VGGnet framework is shown in Figure 2 as follows:

[0174] The 192×192×3 multimodal stacked data is input into the multi-scale feature extraction module of the improved VGGnet framework to extract multi-scale features and obtain a multi-scale feature map;

[0175] The multi-scale feature map is input into the channel attention mechanism module of the improved VGGnet framework, and global average pooling and global maximum pooling operations are performed on the multi-scale feature map to obtain statistical information of the channel dimension. A multi-layer perceptron containing two fully connected layers is used for nonlinear transformation to generate channel attention weights.

[0176] Perform two-dimensional convolution operations through the spatial attention mechanism module to generate a spatial attention map;

[0177] S42: Using GAN to generate noise adversarial networks for active learning loops:

[0178] For pixel-level loss: constrain the local consistency between generated noise and real noise, formula:

[0179] ;

[0180] in, is the synthetic noise output by the generator, y noise is a real noise sample; N is the total number of pixels, i is the sequence number; is the L1 norm.

[0181] Adversarial loss formula:

[0182] ;

[0183] in, Expressing hope, is the false noise image synthesized by the generator G according to the noise input z; noise is a real noise sample; D(·) is the discriminator’s score on the authenticity of the input noise, the larger the value, the more realistic it is. The discriminator’s target is the adversarial loss L adv Maximize,distinguish between real noise and generated noise.

[0184] Gradient penalty formula:

[0185] ;

[0186] in, Expressing hope, represents y noise and random interpolation points of G(z), , Express The operator for finding the gradient, D is the discriminator, Represents the discriminator's judgment Is it the result of a real sample or a generated sample, etc.? is the discriminator function; is the L2 norm.

[0187] Frequency domain loss: The noise spectrum distribution is constrained by FFT. The formula is as follows:

[0188] ;

[0189] in, represents the fast Fourier transform, which converts the spatial noise image into the frequency domain; is the noise image synthesized by the generator; y noise is a real noise sample; is the L2 norm, which measures the difference in frequency domain features.

[0190] Perceptual loss: Use pre-trained VGG to extract features and maintain semantic consistency. The specific formula is as follows:

[0191] ;

[0192] in, Represents the intermediate layer feature extractor of the pre-trained VGG19 network, outputting high-dimensional semantic features; is the noise image synthesized by the generator; y noise is a real noise sample.

[0193] Total loss:

[0194] ;

[0195] in, represents the total loss, arrive Represents the weight coefficient of each loss item; represents pixel-level reconstruction loss; It means fighting against loss; represents the gradient penalty; represents the frequency domain loss; Indicates perceptual loss.

[0196] Import the generated noise images into the training set to simulate various noise patterns in the training data to enhance the generalization ability of the model;

[0197] The model was trained with 32 training samples, an initial learning rate of 0.1, and 20,000 iterations. Finally, a complete CNN model was constructed. When an image is input, the corresponding feature label is output.

[0198] The specific process of step S5 is as follows:

[0199] S51: Input a specified portion of the multimodal medical image into the trained convolutional neural network model for prediction. Based on the prediction results, the multimodal medical image is divided into multiple non-overlapping local regions. When the image size cannot be divided by the region size, the edge region is zero-filled or mirrored to ensure that all pixels are contained in a certain region.

[0200] S52: For each local area, a histogram of pixel values ​​is counted, and local entropy is calculated based on the histogram;

[0201] S53: The Soble operator is used to calculate the gradient amplitude of each pixel in the region. The convolution kernels of the Soble operator in the x and y directions are:

[0202] ;

[0203] ;

[0204] For a pixel (x,y), the gradient magnitude is:

[0205] ;

[0206] in, is the Sobel convolution result of pixel (x,y) in the horizontal direction (x-axis); It is the Sobel convolution result of pixel (x,y) in the vertical direction (y axis).

[0207] S54: Calculate the mean of the gradient amplitude within the region μG and standard deviation σG , as the gradient feature of the corresponding area;

[0208] S55: Evaluate the feature complexity of the corresponding region based on local entropy and gradient information. The specific formula is as follows:

[0209] ;

[0210] in, C i For the region i The normalized complexity score of E i For the region i The local entropy mean of ; E max is the maximum value of the local entropy of the entire graph; For the region i Gradient energy; is the maximum value of the gradient energy of the entire image; is the weight coefficient of entropy and gradient.

[0211] S56: Adjust the fusion weight of local entropy and gradient according to regional feature complexity wi , the specific formula is as follows:

[0212] ;

[0213] in, C i For the region i The normalized complexity score of β is the slope of the control function, γ is the midpoint offset;

[0214] S57: For each region, calculate the threshold based on local entropy, using the mean of the gradient amplitude in the region μ G and standard deviation σ G Calculate the threshold:

[0215] ;

[0216] in, k is the adjustment coefficient, μ G is the mean value of the gradient amplitude in the region, σ G is the standard deviation ;

[0217] pass Perform weighted fusion, where T i For the region R i The final adaptive threshold is used for pixel-level binary segmentation; is a threshold based on local entropy, reflecting texture complexity; is the threshold based on gradient statistics; w i is the dynamic weight;

[0218] Gaussian filtering is performed on all regional thresholds to eliminate threshold mutations between regions, and the regional thresholds are mapped back to the original image coordinates to obtain the final adaptive threshold map.

[0219] Example 4

[0220] On the basis of Example 1 or Example 2 or Example 3, the specific process of step S6 is as follows:

[0221] S61: for each pixel in the multimodal medical image, compare its pixel value with the adaptive threshold value. For a pixel with coordinates (x, y) in the multimodal medical image, if its pixel value I(x, y) is greater than or equal to the adaptive threshold value T(x, y), mark the pixel as foreground and assign a value of 1; if the pixel value I(x, y) is less than the adaptive threshold value T(x, y), mark the pixel as background and assign a value of 0.

[0222] S62: Based on the noise level and artifact type, a first-layer 5×N matrix is ​​constructed, where each coordinate of the matrix corresponds to a specified noise-artifact combination feature label, and the image is mapped to the corresponding coordinate position of the feature label matrix based on the added label;

[0223] S63: constructing a second-layer 5×N matrix of the same dimension as the first-layer 5×N matrix, wherein each coordinate of the matrix corresponds one-to-one to a coordinate of the first-layer 5×N matrix, and each coordinate position stores a segmentation method for the corresponding noise-artifact combination, each segmentation method including a different threshold adjustment strategy or a combination of morphological operations or a post-processing algorithm based on machine learning;

[0224] S64: Determine, based on the multimodal medical image label, a corresponding coordinate position in the first layer 5×N matrix, and obtain, based on the coordinate position determined in the first layer 5×N matrix, a corresponding segmentation method at the same coordinate position in the second layer 5×N matrix;

[0225] S65: Apply the acquired segmentation method to the binarized image:

[0226] If the segmentation method is threshold fine-tuning, the adaptive threshold used in binarization is adjusted according to the rules set in the method, and then the binarization operation is performed again;

[0227] If the segmentation method includes morphological operations, the specified structure element is used to process the binary image to remove small noise areas, fill internal holes, and optimize the segmentation results;

[0228] If the segmentation method is a post-processing algorithm based on machine learning, the binarized image is input into the corresponding model to obtain the final segmentation result.

[0229] Based on the adaptive threshold, pixel-level binary segmentation is performed on multimodal medical images. A two-layer matrix of feature label-segmentation method is set, and image threshold segmentation is completed according to the optimal solution.

[0230] 3D-T1 processing:

[0231] For the adaptive threshold map T1(x,y) generated in step S5, the pixel values ​​I1(x,y) of the original 3D-T1 image are compared pixel by pixel:

[0232] If I1(50, 60) = 180 ≥ T1(50, 60) = 150, then it is labeled as foreground (1), corresponding to the gray matter region;

[0233] If I1(100, 80) = 90 < T1(100, 80) = 110, then it is labeled as background (0), corresponding to the white matter region.

[0234] Result: Generate a binary image B1(x, y), where the brain tissue region is 1 and the background is 0.

[0235] DTI processing:

[0236] For the pixel value I2(x, y) of the fractional anisotropy (FA) map, use the threshold map T2(x, y):

[0237] The fiber bundle regions with high FA values are labeled as 1;

[0238] Isotropic regions such as cerebrospinal fluid are labeled as 0.

[0239] Result: Generate a fiber bundle binary map B2(x, y).

[0240] fMRI processing:

[0241] Apply the threshold T3(x, y) to the pixel value I3(x, y) of the activation map:

[0242] The activation regions with Z-score > 2.3 are labeled as 1;

[0243] The resting regions are labeled as 0.

[0244] Result: Generate an activation region binary map B3(x, y).

[0245] Construct and apply a two-layer matrix:

[0246] The first layer 5×N matrix:

[0247] Noise level: 5-dimensional One-hot encoding (mild, moderate, severe, mixed noise, no noise).

[0248] Artifact type: Assume N = 8 categories (motion artifact, susceptibility artifact, truncation artifact, metal artifact, chemical shift artifact, wraparound artifact, radiofrequency artifact, no artifact).

[0249] A certain 3D-T1 image is labeled as "moderate noise + motion artifact", corresponding to the matrix coordinates (2, 1).

[0250] A certain DTI image is labeled as "severe noise + susceptibility artifact", corresponding to the coordinates (3, 2). [[ID=The second layer 5×N matrix:

[0252] The method corresponding to the coordinate (2,1) is: "threshold fine-tuning + morphological closing operation".

[0253] The coordinate (3,2) corresponds to the method of "Gaussian filtering + conditional random field (CRF) post-processing".

[0254] S65: Apply segmentation methods

[0255] 3D-T1 example (moderate noise + motion artifacts):

[0256] Threshold fine-tuning: Multiply the initial threshold T1(x,y) by a factor of 1.05 to compensate for edge diffusion caused by motion blur.

[0257] Morphological closing operation: using a 5×5 circular structure element:

[0258] First, it expands to fill small holes in the gray matter area;

[0259] Then erode to remove the small noise of edge adhesion.

[0260] Results: The Dice coefficient of gray matter segmentation increased from 0.78 to 0.85.

[0261] The specific process of inputting the unqualified image into the dynamic parameter adjustment module for processing in step S7 is as follows:

[0262] S71: Import the unqualified image into the Gaussian process regression model for nonlinear mapping calculation;

[0263] S72: Simplify the regression model by setting the mean function to zero. The observation data y is generated by the nonlinear mapping function f(x) between the input features and the parameters plus Gaussian noise: ;

[0264] Where y represents the observed data; Represents the nonlinear mapping function between input features and parameters; x is the input feature vector, including Dice deviation, local entropy skewness, and gradient kurtosis; Reflects expert annotation errors or inherent image noise; Indicates the robustness of the control model to noise;

[0265] S73: Kernel Function Implicitly defines a mapping from the input space to a high-dimensional feature space ,make Select a nonlinear kernel function so that the model can capture the nonlinear relationship between inputs;

[0266] Among them, x and are two eigenvectors in the input space; It is a nonlinear mapping function that maps the input vector from the original space to the high-dimensional feature space; It is a kernel function that directly calculates the two mapped vectors and The inner product of

[0267] S74: Construct the covariance matrix:

[0268] ;

[0269] in, Refers to the sample x i and x j is the kernel function of the independent variable, used to calculate the input sample x i and x j The similarities between represents the robustness of the control model to noise, is the Kronecker delta function, when i = j 1 when it is, otherwise 0;

[0270] Compute the marginal likelihood:

[0271] ;

[0272] in, y represents observation data; X is the input feature matrix, each row corresponds to the eigenvector of a sample, and K is the covariance matrix; n is the sample size, related to the normalization of the log-likelihood;

[0273] S75: Command input x ∈ R d , the RBF kernel is implicitly mapped to infinite-dimensional space;

[0274] S76: Maximizing the Log-Marginal Likelihood via Gradient Descent:

[0275] ;

[0276] in, y represents the observed data, θ is the hyperparameter vector to be optimized, K is the covariance matrix; is the partial derivative of the covariance matrix with respect to the hyperparameters, and tr is the summation of all elements on the main diagonal of the matrix;

[0277] Adaptively select and adjust the parameters of the preprocessing method, including dynamic adjustment of the segmentation threshold, smoothing kernel size, and number of registration iterations.

[0278] In summary, the multimodal image threshold segmentation preprocessing method based on convolutional neural network provided by the present invention unifies the image into standard space and pixel value mapping, resamples, generates high and low frequency subbands, performs soft threshold denoising on the high frequency subband, enhances the contrast of the low frequency subband, and then fuses them; evaluates the image quality and adds labels, builds an optimized VGGnet framework, generates a noise adversarial network for active learning loop, trains the convolutional neural network model, inputs the image into the model for prediction, calculates the local entropy and gradient amplitude based on the prediction results, dynamically adjusts the fusion weight to obtain the adaptive threshold, performs pixel-level binary segmentation, and achieves optimal segmentation by setting a double-layer matrix of feature label-segmentation method; performs gray matter Dice calculation on the segmented image, and optimizes the preprocessing parameters of unqualified images through a dynamic parameter adjustment module based on the Gaussian process regression model. It effectively improves the segmentation accuracy and processing efficiency of multimodal medical images and enhances the adaptability of the model to complex images.

Claims

1. A multimodal image threshold segmentation preprocessing method based on convolutional neural network, characterized in that: The following steps are involved: S1: Unify the multimodal medical imaging data into a standard space, map the multimodal medical imaging pixel values ​​to a specified interval, resample them, and generate a low-frequency sub-band and multiple high-frequency sub-bands through multi-scale decomposition; S2: Perform soft threshold denoising on the high-frequency sub-band, divide the low-frequency sub-band into multiple sub-regions and calculate the cumulative distribution function value for each sub-region, generate a grayscale mapping function and apply it to the original pixel value, transform and fuse the low-frequency sub-band with the high-frequency sub-band; S3: Perform quality assessment and labeling of multimodal medical images, calculate the K coefficient of each image, agree on the labeling of images with K coefficients within a first preset range, conduct third-party arbitration for images with K coefficients within a second preset range, and output the labeling as the arbitration result; S4: Build a convolutional neural network model and perform optimization and modification, generate a noise adversarial network for active learning cycles, and train the convolutional neural network model; S5: Input a specified portion of multimodal medical images into the trained convolutional neural network model for prediction, divide the prediction results into multiple local regions, calculate the local entropy and gradient amplitude of each region, dynamically adjust the fusion weight of the local entropy and gradient, and use an adaptive weighted fusion algorithm to calculate the adaptive threshold; S6: Based on the adaptive threshold, perform pixel-level binary segmentation on the multimodal medical image, set the two-layer matrix of feature label-segmentation method, and complete the threshold segmentation of the original image according to the optimal solution; S7: Perform gray matter Dice calculation on the multimodal medical image after threshold segmentation, and judge whether the multimodal medical image is qualified based on the gray matter Dice result. The qualified image is directly output, and the unqualified image is input into the dynamic parameter adjustment module for processing.

2. The multimodal image threshold segmentation preprocessing method based on convolutional neural network according to claim 1, characterized in that: The specific process of unifying the multimodal medical image data into the standard space and mapping the multimodal medical image pixel values ​​into the specified interval and then resampling in step S1 is as follows: S11: Acquire multimodal medical imaging data including 3D-T1, DTI, and fMRI, and unify them into a standard space; S12: setting the target floating-point coordinates of the multimodal medical image, including the integer part and the decimal part, and performing bilinear interpolation processing on the image; When the target point is located between two pixels, the pixels in the target image are mapped to the floating-point coordinates of the original image, and a 4×4 area centered on the specified pixel in the original image is taken; S13: For each surrounding point, calculate its offset distance in the x and y directions, where the surrounding points refer to the four nearest neighboring pixels around the target floating point coordinate in the original image when performing bilinear interpolation; S14: Calculate weights using the cubic convolution function.

3. The multimodal image threshold segmentation preprocessing method based on convolutional neural network according to claim 2, characterized in that: The specific process of generating a low-frequency sub-band and multiple high-frequency sub-bands through multi-scale decomposition in step S1 is as follows: S15: Create a two-dimensional image matrix of the multimodal medical image, and perform convolution operations using a low-pass filter and a high-pass filter in the row direction. After the convolution, two matrices are obtained, one containing low-frequency information in the row direction and the other containing high-frequency information in the horizontal direction; S16: The result of the row-wise convolution is convolved with the low-pass filter and the high-pass filter in the column direction to obtain four sub-bands: Low-frequency subband: contains the low-frequency information and contours of the image; Horizontal high-frequency sub-band: contains the vertical edge information of the image; Vertical high-frequency sub-band: contains the horizontal edge information of the image; Diagonal high-frequency subband: contains high-frequency details and noise in the diagonal direction of the image; S17: Repeat the steps of single-layer two-dimensional DWT decomposition for the obtained low-frequency subband to obtain subbands of more scales. With each layer of decomposition, the size of the low-frequency subband becomes 1 / 4 of the size of the previous layer of low-frequency subband, and the size of the high-frequency subband remains the same as before the decomposition of the corresponding low-frequency subband. Among them, single-layer two-dimensional DWT refers to the process of two-dimensional discrete wavelet transform of low-pass filter and high-pass filter; S18: Based on the single-layer two-dimensional DWT decomposition, a filtering operation in the depth direction is added. Low-pass filters and high-pass filters are used for convolution in the row direction, column direction, and depth direction respectively. One three-dimensional DWT decomposition can obtain 8 sub-bands, including 1 low-frequency sub-band and 7 high-frequency sub-bands. Multi-layer three-dimensional DWT decomposition continuously repeats the S18 operation on the low-frequency sub-band to obtain sub-band information of more scales. Among them, the three-dimensional DWT refers to adding a depth-direction filtering operation on the basis of a single-layer two-dimensional DWT. It is essentially a cube decomposition of separable filtering. Through triple filtering of rows, columns, and depth, the 3D data is decomposed into 8 sub-bands with complementary directional features. Multi-layer 3D DWT refers to recursively performing 3D DWT decomposition on the low-frequency subbands of the previous layer to form a pyramid-like multi-scale feature representation.

4. The multimodal image threshold segmentation preprocessing method based on convolutional neural network according to claim 1, characterized in that: The specific process of step S2 is as follows: S21: Based on Calculate the threshold, where σ is the noise standard deviation, σ≈ 1.4826 · MAD, MAD is the median absolute deviation of the high-frequency subband, N is the total number of pixels in the high-frequency subband, and for each coefficient in the high-frequency subband, a preset soft threshold function is applied to process it, and the coefficients with absolute values ​​less than the threshold are set to zero, and the coefficients greater than the threshold are shrunk to zero. λ ; S22: Divide the low-frequency sub-band into multiple non-overlapping sub-regions, where the size of the sub-regions is set to 8×8. For each sub-region, count the distribution of pixel values ​​therein and calculate the cumulative distribution function value. S23: Multiplying the cumulative distribution function value by the total number of gray levels and performing a rounding operation to obtain a mapped gray value, thereby obtaining a mapping relationship from the original gray value to the new gray value, thereby forming a gray mapping function; S24: traverse each sub-region of the low-frequency sub-band, and replace the original grayscale value of each pixel in the sub-region with the mapped grayscale value according to the generated grayscale mapping function, thereby completing the contrast enhancement of the low-frequency sub-band; S25: For the two-dimensional image, perform inverse filtering in the column direction and then inverse filtering in the row direction. For the three-dimensional image, perform inverse filtering operations in the three dimensional directions in sequence to obtain a preliminary fused image. S26: According to local features of the image including local variance and gradient, different weights are assigned to the low-frequency sub-band and the high-frequency sub-band, and fusion optimization is performed using a weighted fusion method.

5. The multimodal image threshold segmentation preprocessing method based on convolutional neural network according to claim 1, characterized in that: The specific process of step S3 is as follows: S31: Perform quality assessment on multimodal medical images and add labels including noise level and artifact type. The noise level is set to 5 dimensions and the artifact type is set to N types. S32: Each image was quality-assessed and classified by two experts, and the K coefficient was calculated. The calculation included the observed consistency Po, that is, the actual observed agreement ratio, and the random consistency Pe; When 0.8 < K < 1: Agree with the corresponding label. That is, when the assessment of the image quality by two experts falls within the first preset range of 0.8 < K < 1, the system automatically adopts the label marked by the expert as the final result without third-party arbitration. The K coefficient is the Kappa coefficient. When -1 < K < 0.8: Introduce a third expert for arbitration, and the label output is the arbitration result.

6. The multimodal image threshold segmentation preprocessing method based on convolutional neural network according to claim 1, characterized in that: In step S4, the specific process of constructing a convolutional neural network model and optimizing and modifying it to generate a noise adversarial network for active learning loop is as follows: S41: Construct an improved VGGnet framework for optimization and modification: Input the multi-modal stacked data into the multi-scale feature extraction module of the improved VGGnet framework to perform multi-scale feature extraction to obtain multi-scale feature maps; Input the multi-scale feature maps into the channel attention mechanism module of the improved VGGnet framework, perform global average pooling and global max pooling operations on the multi-scale feature maps to obtain statistical information in the channel dimension, and use a multi-layer perceptron containing two fully connected layers for non-linear transformation to generate channel attention weights; Perform two-dimensional convolution operation through the spatial attention mechanism module to generate a spatial attention map; S42: Use GAN to generate a noise adversarial network for active learning loop: For pixel-level loss: Constrain the local consistency between the generated noise and the real noise, and set the adversarial loss formula and gradient penalty formula. For frequency-domain loss: Constrain the noise spectrum distribution through FFT, and the perceptual loss uses the pre-trained and improved VGGnet framework to extract features to maintain semantic consistency; Import the generated noise image into the training set, simulate various noise patterns in the training data, and enhance the generalization ability of the model; Perform model training, set the training samples, initial learning rate and number of iterations, construct a complete CNN large model, and when an input image is fed in, output the feature label corresponding to the image.

7. The multimodal image threshold segmentation preprocessing method based on convolutional neural network according to claim 6, characterized in that: The specific process of step S5 is as follows: S51: Input the specified part of the multi-modal medical image into the trained convolutional neural network model for prediction. According to the prediction result, divide the multi-modal medical image into multiple non-overlapping local regions. When the image size cannot be evenly divided by the region size, perform zero-padding or mirror extension on the edge regions to ensure that all pixels are included in a certain region; S52: For each local region, count the histogram of its pixel values and calculate the local entropy according to the histogram; S53: Use the soble operator to calculate the gradient amplitude of each pixel in the region; S54: Calculate the mean and standard deviation of the gradient amplitudes in the region as the gradient features of the corresponding region; S55: Evaluate the complexity of the corresponding region features according to the local entropy and gradient information; S56: Adjust the fusion weights of the local entropy and gradient according to the region feature complexity; S57: For each region, calculate the threshold based on the local entropy, calculate the threshold using the mean and standard deviation of the gradient amplitudes in the region, and perform weighted fusion to obtain the final adaptive threshold. Perform Gaussian filtering on all region thresholds to eliminate the threshold mutation between regions, and map the region thresholds back to the original image coordinates to obtain the final adaptive threshold map.

8. The multimodal image threshold segmentation preprocessing method based on convolutional neural network according to claim 7, characterized in that: The specific process of step S6 is as follows: S61: for each pixel in the multimodal medical image, compare its pixel value with the adaptive threshold value. For a pixel with coordinates (x, y) in the multimodal medical image, if its pixel value I(x, y) is greater than or equal to the adaptive threshold value T(x, y), mark the pixel as foreground and assign a value of 1; if the pixel value I(x, y) is less than the adaptive threshold value T(x, y), mark the pixel as background and assign a value of 0. S62: Based on the noise level and artifact type, a first-layer 5×N matrix is ​​constructed, where each coordinate of the matrix corresponds to a specified noise-artifact combination feature label, and the image is mapped to the corresponding coordinate position of the feature label matrix based on the added label; S63: constructing a second-layer 5×N matrix of the same dimension as the first-layer 5×N matrix, wherein each coordinate of the matrix corresponds one-to-one to a coordinate of the first-layer 5×N matrix, and each coordinate position stores a segmentation method for the corresponding noise-artifact combination, each segmentation method including a different threshold adjustment strategy or a combination of morphological operations or a post-processing algorithm based on machine learning; S64: Determine, based on the multimodal medical image label, a corresponding coordinate position in the first layer 5×N matrix, and obtain, based on the coordinate position determined in the first layer 5×N matrix, a corresponding segmentation method at the same coordinate position in the second layer 5×N matrix; S65: Apply the acquired segmentation method to the binarized image: If the segmentation method is threshold fine-tuning, the adaptive threshold used in binarization is adjusted according to the rules set in the method, and then the binarization operation is performed again; If the segmentation method includes morphological operations, the specified structure element is used to process the binary image to remove small noise areas, fill internal holes, and optimize the segmentation results; If the segmentation method is a post-processing algorithm based on machine learning, the binary image is input into the corresponding model to obtain the final segmentation result; Multimodal medical images are segmented into pixel-level binarization based on adaptive thresholds, a two-layer matrix of feature labels and segmentation methods is set, and image threshold segmentation is completed according to the optimal solution.

9. The multimodal image threshold segmentation preprocessing method based on convolutional neural network according to claim 1, characterized in that: The specific process of inputting the unqualified image into the dynamic parameter adjustment module for processing in step S7 is as follows: S71: Import the unqualified image into the Gaussian process regression model for nonlinear mapping calculation; S72: Simplify the regression model by setting the mean function to zero. The observation data y is generated by the nonlinear mapping function f(x) between the input features and the parameters plus Gaussian noise: , ; in, y represents observation data; Represents the nonlinear mapping function between input features and parameters; x is the input feature vector; represents expert annotation error or inherent image noise; Refers to the robustness of the control model to noise; S73: Kernel Function Implicitly defines a mapping from the input space to a high-dimensional feature space ,make , select a nonlinear kernel function so that the model can capture the nonlinear relationship between inputs; Among them, x and are two eigenvectors in the input space; It is a nonlinear mapping function that maps the input vector from the original space to the high-dimensional feature space; It is a kernel function that directly calculates the two mapped vectors and The inner product of S74: Construct the covariance matrix: ； in, Refers to the sample x i and x j is the kernel function of the independent variable, used to calculate the input sample x i and x j The similarities between refers to the robustness of the control model to noise, is the Kronecker delta function, when i = j 1 when it is, otherwise 0; Compute the marginal likelihood: ； in, y represents observation data; X is the input feature matrix, each row corresponds to the eigenvector of a sample, and K is the covariance matrix; n is the sample size, related to the normalization of the log-likelihood; S75: Command input x ∈ R d , the RBF kernel is implicitly mapped to infinite-dimensional space; S76: Maximizing the Log-Marginal Likelihood via Gradient Descent: ； in, y represents the observed data, θ is the hyperparameter vector to be optimized, K is the covariance matrix; is the partial derivative of the covariance matrix with respect to the hyperparameters, and tr is the summation of all elements on the main diagonal of the matrix; Adaptively select and adjust the parameters of the preprocessing method, including dynamic adjustment of the segmentation threshold, smoothing kernel size, and number of registration iterations.

Images