Multi-sequence image segmentation method based on geometric iterative optimization fusion

Abstract

The present invention discloses a multi-sequence image segmentation method based on geometric iteration optimization fusion, which relates to the field of medical image processing technology. The method comprises: labeling the collected glioma scanning images by manual delineation, and preprocessing the glioma scanning images by using patch extraction technology; combining the processed glioma scanning images with the segmentation network architecture to construct a sequence image automatic segmentation model, and outputting the segmentation features of the glioma scanning images by using the sequence image automatic segmentation model; mapping the segmentation features of the glioma scanning images to a geometric high-dimensional space, and obtaining the segmentation results of the glioma scanning images based on the mapping results and the geometric iteration optimization algorithm. The present invention collects a number of glioma sequence scanning images, labels them with segmentation labels by experts manually delineating them, and calibrates the glioma image segmentation by using a geometric iteration optimization algorithm, so as to achieve the fusion of different image feature information.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a multi-sequence image segmentation method based on geometric iteration optimization fusion. Background Art

[0002] Glioma is a primary brain tumor that originates from the cancerous transformation of glial cells in the brain and spinal cord. 3D magnetic resonance imaging (MRI) can obtain glioma sequence information at different spatial locations through a single scan. Common glioma MR multi-sequence images include four sequences: T1-weighted (T1), T1-weighted contrast-enhanced (T1ce), T2-weighted (T2), and T2 fluid-attenuated inversion recovery (Flair). The T1 sequence is mainly used to observe anatomical structures. The T1ce sequence is to inject contrast agent into the blood before the MRI examination, which helps to enhance the display of areas with rich blood flow, such as the tumor site. The T2 sequence reflects the edema area around the lesion, and the lesion area can be seen through the T2 sequence. The Flair sequence is also called water suppression imaging technology. In this sequence, cerebrospinal fluid presents low signal, and substantial lesions and lesions containing bound water present high signal. At present, MRI has been widely used by radiologists as an important tool for early diagnosis and treatment of brain tumors. However, manual segmentation of glioma MRI images requires a lot of time, resources and expertise.

[0003] In the treatment of gliomas, lesion type is a very important indicator for evaluating the nature and severity of the lesion. Common lesion type annotations include necrotic tumor core, peritumoral edema and enhancing tumor. Currently, most glioma segmentation tasks are to segment the sub-regions formed by nesting three labels, including the enhancing tumor (ET) region, the tumor core (TC) region, and the whole tumor (WT) region. The ET region represents the area of ​​blood-brain barrier destruction and angiogenesis. The TC region is the main part of the glioma that usually needs to be resected, including the ET labeled region and the NCR labeled region. The WT region represents the complete range of the glioma and is composed of three labeled regions.

[0004] Medical image segmentation is an important part of medical imaging. It can help doctors better evaluate and diagnose image information. In recent years, with the rapid development of artificial intelligence, deep learning has begun to be widely used in the field of medical image segmentation. Deep learning is a machine learning technology based on neural networks. It learns the characteristics of input data through multi-layer neural networks to achieve tasks such as data classification, prediction and segmentation. In medical image processing, deep learning technology can help doctors automatically identify and segment different tissues, organs or lesion areas in images. The application of deep learning technology has achieved many successes in the field of medical image processing. For example, in medical image segmentation, the U-Net model is a classic deep learning model that can achieve high-precision organ and lesion area segmentation.

[0005] Although deep learning has made great progress in medical image processing, there are still some shortcomings. For example, the current method is limited in MR image data processing and high-resolution features are easily lost. In addition, traditional glioma MR image segmentation often only considers the impact of a single sequence and does not consider the joint impact between multiple sequences. This may lead to certain limitations in the final segmentation results of glioma MR multi-sequence images.

[0006] Currently, no effective solution has been proposed for the problems in the related technologies. Summary of the invention

[0007] In order to solve the above problems, the present invention proposes a multi-sequence image segmentation method based on geometric iterative optimization fusion, so as to achieve the purpose of calibrating glioma image segmentation by using a geometric iterative optimization algorithm.

[0008] In order to achieve the above object, the present invention adopts the following technical solution:

[0009] The present invention provides a multi-sequence image segmentation method based on geometric iterative optimization fusion, which comprises the following steps:

[0010] The collected glioma scan images were labeled by manual delineation, and the patch extraction technology was used to pre-process the glioma scan images;

[0011] The processed glioma scan images are combined with the segmentation network architecture to construct a sequence image automatic segmentation model, and the sequence image automatic segmentation model is used to output the segmentation features of the glioma scan images;

[0012] Map the segmentation features of the glioma scan image to a geometric high-dimensional space, and obtain the segmentation result of the glioma scan image based on the mapping result and the geometric iterative optimization algorithm;

[0013] The automatic segmentation model of sequential images is used to output the segmentation results of real-time glioma scan images, and the segmentation results are matched with the image labeling results to obtain glioma characteristics.

[0014] Preferably, labeling the collected glioma scan images by manual delineation, and preprocessing the glioma scan images by using patch extraction technology include the following steps:

[0015] Acquire a corresponding set of sequential glioma scanning images based on a preset magnetic resonance imaging parameter range value, and verify the quality level of the glioma scanning images;

[0016] Adjust the clarity and contrast of the glioma scan image according to the verification results, and perform image registration processing after the adjustment is completed;

[0017] The annotation protocol was defined based on the glioma type information, and the manual annotator combined the annotation protocol with the image processing software to manually outline the boundary information of the glioma;

[0018] Label the glioma lesion type based on the boundary information results, and combine all the labeling results to perform cross-validation to compare the accuracy of the labeling results;

[0019] If the accuracy is greater than or equal to the preset reference value, the lesion label type is saved in the specified file format, and the standard parameters generated by the annotation process are recorded. If the accuracy is less than the preset reference value, the boundary information delineation operation is re-executed;

[0020] The image pixels of the glioma scanning image are obtained, the glioma scanning image is normalized according to the image pixels, and the glioma scanning image is subjected to a hot encoding operation using a patch extraction technique.

[0021] Preferably, obtaining image pixels of a glioma scanning image, normalizing the glioma scanning image according to the image pixels, and performing a thermal encoding operation on the glioma scanning image using a patch extraction technique comprises the following steps:

[0022] Based on the voxel value of the glioma scan image, the redundant background with zero voxel value is cropped and removed, and the glioma scan image is normalized using the image pixels;

[0023] The nearest neighbor interpolation algorithm was used to scale the glioma scan images to the same size, and the glioma scan images with the same contrast were merged to form a channel three-dimensional image;

[0024] The channel 3D image and the corresponding mask are divided into several patches, and the corresponding patches are extracted by moving the sliding window on the channel 3D image;

[0025] A hot encoding operation is performed according to the lesion label type corresponding to each pixel in the patch to obtain the channel area corresponding to the lesion label type.

[0026] Preferably, based on the voxel value of the glioma scan image, cutting out the redundant background with a voxel value of zero, and normalizing the glioma scan image using the image pixels comprises the following steps:

[0027] Initialize a labeling matrix with the same dimension as the original glioma scan image based on the three-dimensional image matrix of the glioma scan image, and traverse the voxel values ​​of the glioma scan image;

[0028] Analyze the modal value of the voxel value in the lesion label type. If the modal value is zero, it is defined as a background voxel, and the corresponding position of the background voxel in the labeling matrix is ​​marked as zero;

[0029] If the modality value is non-zero, it is defined as a normal voxel, and the corresponding position of the normal voxel in the labeling matrix is ​​marked as one;

[0030] Calculate the minimum bounding box of non-zero voxels in the label matrix, crop to remove redundant background with zero voxel value, and retain the area of ​​non-zero voxels;

[0031] The normalization range is defined based on the image pixels, and the maximum and minimum pixel intensity values ​​are selected through the glioma scanning image, and the normalization processing is performed using the value selection results.

[0032] Preferably, the processed glioma scan image is combined with the segmentation network architecture to construct a sequence image automatic segmentation model, and the segmentation features of the glioma scan image are outputted by the sequence image automatic segmentation model, which includes the following steps:

[0033] The processed glioma scan images were divided into training set, validation set and test set according to the preset ratio by cross-validation method, and the segmentation network was used as the architecture of the automatic segmentation model of the sequence images;

[0034] Input the training set into the recognition and segmentation network, use the convolution layer to obtain the feature map of the training set, and input the feature result into the parallel layer;

[0035] In the parallel layer, the convolution block and the window self-attention block are used to repeatedly extract the features according to the preset extraction rules to obtain the underlying features;

[0036] The output layer is used to convert the underlying features into probability values ​​to obtain the segmentation features corresponding to the lesion label type channel area, and the composite loss function is used to verify the output error optimization segmentation network;

[0037] The optimized segmentation network is used as the automatic segmentation model for sequential images, and the automatic segmentation model for sequential images is used to output the segmentation features of glioma scan images.

[0038] Preferably, using the output layer to convert the underlying features into probability values ​​to obtain the segmentation features corresponding to the lesion label type channel area, and using the composite loss function to verify the output error to optimize the segmentation network includes the following steps:

[0039] The output layer receives the underlying features, converts the feature values ​​corresponding to the underlying features into probability values, and defines the pixel points corresponding to the lesion label type channel area based on the probability value conversion results;

[0040] The encoder and decoder of the segmentation network are connected using the skip connection technology. After each upsampling operation, the feature map of the current decoding layer is concatenated with the feature map of the corresponding layer in the encoding stage in the channel dimension.

[0041] The high-resolution features are combined with the low-resolution features to obtain the segmentation results of the glioma scan images, and the composite loss function is defined using the weighted cross entropy loss function and the dice loss function;

[0042] The predicted segmentation label of the segmentation network output is calculated based on the composite loss function, and the predicted segmentation label is compared with the true segmentation label to obtain the error result, and the error result is used to back propagate and update the optimized segmentation network parameters.

[0043] Preferably, the expression of the composite loss function is:

[0044]

[0045] In the formula, L represents the composite loss function, C represents the total number of classes, i represents the i-th class, and w i represents the weight of the i-th class, y i is the true segmentation label of the i-th category, p i is the predicted segmentation label of the i-th class, n represents the total number of pixels, P j represents the predicted probability of the jth pixel, g i represents the true probability of the jth pixel, and ε represents a constant.

[0046] Preferably, mapping the segmentation features of the glioma scanning image to a geometric high-dimensional space, and obtaining the segmentation result of the glioma scanning image based on the mapping result and the geometric iterative optimization algorithm includes the following steps:

[0047] The convolutional layer is used to convert the segmentation features of the glioma scan image into a feature vector, and the feature vector is mapped to a geometric high-dimensional space as the input of the geometric iterative optimization algorithm;

[0048] The confidence of the feature vector is updated iteratively and alternately using the geometric iterative optimization algorithm, the feature vector is updated based on the confidence result, and the optimal feature vector is found as the feature fusion result;

[0049] The feature fusion result is reshaped by using inverse convolution to obtain a feature map, and the feature map is input into the decoding stage of the segmentation network to obtain the fused calibrated segmentation result.

[0050] Preferably, using a geometric iterative optimization algorithm to iteratively and alternately update the confidence of the feature vector, updating the feature vector based on the confidence result, and finding the optimal feature vector as the feature fusion result includes the following steps:

[0051] After mapping the feature vector to a geometric high-dimensional space, the confidence of the high-dimensional segmentation vector is initialized in advance, and the predicted output of the feature vector under the confidence is determined;

[0052] The confidence of the feature vector and the high-dimensional segmentation vector is iteratively optimized using the loss function, and the feature vector with the best confidence is selected as the true feature vector;

[0053] Repeatedly updating the true feature vector within the range of iteration times based on the optimal confidence, and terminating the geometric iteration update of the true feature vector when the trend of the true feature vector is stable;

[0054] Based on the updated result, a true feature vector whose confidence is consistent with the true segmentation is obtained, and the true feature vector is used as the feature fusion result.

[0055] Preferably, using the loss function to iteratively optimize the confidence of the feature vector and the high-dimensional segmentation vector, and selecting the feature vector corresponding to the best confidence as the true feature vector includes the following steps:

[0056] The difference between the predicted output and the true segmentation label is calculated based on the loss function, the difference is used to measure the distance between the predicted distribution and the actual distribution, and the feature vector gradient is analyzed based on the distance result;

[0057] Based on the gradient of the feature vector, the confidence of the feature vector and the high-dimensional segmentation vector is repeatedly updated using a geometric iterative optimization algorithm, and the distance between the predicted distribution and the actual distribution is obtained based on the update result;

[0058] The feature vector corresponding to the minimum distance value and the confidence of the high-dimensional segmentation vector are selected as the true feature vector and the optimal confidence respectively.

[0059] The beneficial effects of the present invention are:

[0060] 1. The present invention collects a number of glioma sequence scanning images, which are manually segmented and labeled by experts, and preprocessed to standardize them to form a multi-sequence glioma MR image data set. The geometric iterative optimization algorithm is used to calibrate the glioma MR multi-sequence image segmentation to achieve the fusion of different sequence MR image feature information, which can solve the problem of ignoring the correlation of multi-sequence MR data feature information and provide auxiliary decision support for clinical diagnosis.

[0061] 2. The image segmentation model proposed in the present invention is based on the joint influence of multi-sequence images of gliomas. By improving the iterative optimization fusion algorithm and calibrating the feature information extracted by the model, a more accurate glioma segmentation result can be obtained, thereby improving the accuracy of glioma MR multi-sequence image segmentation, providing a more reliable basis for clinical diagnosis and treatment. At the same time, the parallel module is used to simultaneously extract richer local and global features of glioma MR images, providing an efficient and reliable solution for medical image segmentation.

[0062] 3. The automatic segmentation of glioma images using the segmentation network proposed in the present invention can extract richer and more comprehensive glioma MR image features, improve the segmentation accuracy of glioma MR images, and realize the calibration of the deep convolutional network model based on the geometric iterative optimization algorithm, thereby solving the problem of limited calibration of experimental results caused by the non-optimal fusion of multi-sequence MR image features by the traditional deep convolutional network model, thereby improving the performance of the model and the reliability of the experimental results. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0064] Figure 1 is a flow chart of a multi-sequence image segmentation method based on geometric iterative optimization fusion according to an embodiment of the present invention;

[0065] Figure 2 It is a labeled flow chart in a multi-sequence image segmentation method based on geometric iterative optimization fusion according to an embodiment of the present invention;

[0066] Figure 3 is a flowchart of glioma scan image preprocessing in a multi-sequence image segmentation method based on geometric iterative optimization fusion according to an embodiment of the present invention;

[0067] Figure 4 is an automatic segmentation flow chart in a multi-sequence image segmentation method based on geometric iterative optimization fusion according to an embodiment of the present invention;

[0068] Figure 5 It is a flow chart of a geometric iterative optimization fusion algorithm in a multi-sequence image segmentation method based on geometric iterative optimization fusion according to an embodiment of the present invention. DETAILED DESCRIPTION

[0069] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0070] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0071] It should be noted that the terms used herein are only for describing specific embodiments, and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should also be understood that the terms "include" and "have" and any of their variations are intended to cover non-exclusive inclusions, for example, the process, method, system, product or equipment comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0072] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0073] See also Figure 1 , a multi-sequence image segmentation method based on geometric iterative optimization fusion, the multi-sequence image segmentation method includes the following steps:

[0074] S1. Use manual outlining to label the collected glioma scan images, and use patch extraction technology to pre-process the glioma scan images.

[0075] It should be noted that the glioma scanning images mentioned here include several glioma multi-parameter MRI (mpMRI) multi-sequence scanning images, including T1-weighted (T1), T1-weighted contrast-enhanced (T1ce), T2-weighted (T2) and T2 fluid-attenuated inversion recovery (Flair).

[0076] See also Figure 2In this embodiment, the collected glioma scan images are labeled by manual outlining, and the patch extraction technology is selected to perform preprocessing on the glioma scan images. Based on the preset parameter range value of the magnetic resonance imaging, a corresponding number of groups of sequential glioma scan images can be obtained, and the quality level of the glioma scan images can be verified, and the quality of the image sequence can be evaluated to ensure that the image clarity and contrast are high enough to accurately identify and annotate the tissue structure; the clarity and contrast of the glioma scan images are adjusted according to the verification results, and image registration processing is performed after the adjustment is completed to ensure that they are aligned in space; the annotation protocol is defined based on the glioma type information, and the manual annotator combines the annotation protocol with the image processing software (specifically, 3DSlicer can be used) according to the manual outlining. The boundary information of the glioma is outlined by drawing; the lesion label type of the glioma (including the necrotic tumor core area, the peritumoral edema area and the enhanced tumor area) is annotated according to the boundary information results, and all the annotation results are combined to perform cross-validation to compare the accuracy of the annotation results; if the accuracy is greater than or equal to the preset reference value, the lesion label type is saved in a specified file format (which can be saved in NifTI file format), and the standard parameters generated by the annotation process (including the annotator, date, parameters used by the annotation software, etc.) are recorded; if the accuracy is less than the preset reference value, the boundary information outlining operation is re-executed; the image pixels of the glioma scanning image are obtained, the glioma scanning image is normalized according to the image pixels, and the glioma scanning image is subjected to thermal encoding operation using patch extraction technology.

[0077] See also Figure 3 Specifically, the image pixels of the glioma scanning image are obtained, the glioma scanning image is normalized according to the image pixels, and the glioma scanning image is subjected to a hot encoding operation using a patch extraction technique. Based on the voxel value of the glioma scanning image, the redundant background with a voxel value of zero can be cropped and removed, and the glioma scanning image is subjected to normalization using the image pixels; the glioma scanning image is scaled to the same size using a nearest neighbor interpolation algorithm, the specific purpose of which is to ensure that no new voxel value is generated during the scaling process, so as to alleviate the problem of excessive memory usage caused by the three-dimensional image, and the glioma scanning images of the same contrast are merged to form a channel three-dimensional image; the channel three-dimensional image and the corresponding mask are divided into a number of patches, and the corresponding patches are moved on the channel three-dimensional image through a sliding window and extracted; a hot encoding operation is performed according to the lesion label type corresponding to each pixel in the patch to obtain the channel area corresponding to the lesion label type, wherein channel 0 is the background area, channel 1 is the necrotic tumor core area, channel 2 is the peritumoral edema area, and channel 3 is the enhanced tumor area.

[0078] Specifically, when the redundant background with zero voxel value is cropped and removed based on the voxel value of the glioma scanning image, and the glioma scanning image is normalized using the image pixels, a labeling matrix with the same dimension as the original glioma scanning image can be initialized based on the three-dimensional image matrix of the glioma scanning image, and the voxel values ​​of the glioma scanning image can be traversed; the modal value of the voxel value in the lesion label type is analyzed, if the modal value is zero, it is defined as a background voxel, and the corresponding position of the background voxel in the labeling matrix is ​​marked as zero; if the modal value is non-zero, it is defined as a normal voxel, and the corresponding position of the normal voxel in the labeling matrix is ​​marked as one; the minimum bounding box of the non-zero voxels in the labeling matrix is ​​calculated to crop and remove the redundant background with zero voxel value, and the area of ​​the non-zero voxels is retained; the normalization range is defined based on the image pixels, and the maximum and minimum pixel intensity values ​​are selected by traversing the glioma scanning image, and the normalization processing is performed using the value results.

[0079] Among them, the image normalization function is:

[0080]

[0081] In the formula, u represents the mean value of the image, σ represents the standard deviation of the image, X represents the image matrix, Represents the normalized image matrix.

[0082] S2. Combine the processed glioma scanning image with the segmentation network architecture to build a sequence image automatic segmentation model, and use the sequence image automatic segmentation model to output the segmentation features of the glioma scanning image.

[0083] See also Figure 4 In this embodiment, the processed glioma scanning image is combined with the segmentation network architecture to construct a sequence image automatic segmentation model. When the sequence image automatic segmentation model is used to output the segmentation features of the glioma scanning image, the processed glioma scanning image can be divided into a training set, a validation set and a test set according to a preset ratio by a cross-validation method, and the STNet segmentation network is used as the architecture of the sequence image automatic segmentation model; the training set is input into the recognition segmentation network, the feature map of the training set is obtained by using the convolution layer, and the feature result is input into the parallel layer; in the parallel layer, the convolution block and the window self-attention block are used to repeatedly extract the features according to the preset extraction rules to obtain the underlying features; the output layer is used to convert the underlying features into probability values ​​to obtain the segmentation features corresponding to the lesion label type channel area, and the composite loss function is used to verify the output error optimization segmentation network; the optimized segmentation network is used as the sequence image automatic segmentation model, and the sequence image automatic segmentation model is used to output the segmentation features of the glioma scanning image.

[0084] It should be explained that Conv stands for convolutional operation, LN stands for layer normalization, W / SW-MSA stands for window multi-head self-attention mechanism and shifted window multi-head self-attention mechanism respectively, GAP stands for global average pooling, and MLP stands for multi-layer perceptron.

[0085] Specifically, when the output layer is used to convert the underlying features into probability values ​​to obtain the segmentation features corresponding to the lesion label type channel area, and the composite loss function is used to verify the output error optimization segmentation network, the output layer can be used to receive the underlying features, and the feature values ​​corresponding to the underlying features are converted into probability values, and the pixel points corresponding to the lesion label type channel area are defined based on the probability value conversion results; the encoder and decoder of the segmentation network are connected by using the jump connection technology, and after each upsampling operation, the feature map of the current decoding layer is spliced ​​with the feature map of the corresponding layer in the encoding stage in the channel dimension; the high-resolution features are combined with the low-resolution features to obtain the segmentation results of the glioma scanning image, and the composite loss function is defined by using the weighted cross entropy loss function and the dice loss function; the predicted segmentation label output by the segmentation network is calculated based on the composite loss function, and the predicted segmentation label is compared with the true segmentation label to obtain the error result, and the error result is used to back propagate and update the optimized segmentation network parameters.

[0086] In short, the specific application process of building a sequence image automatic segmentation model based on the segmentation network architecture is as follows:

[0087] Step 1: Use cross-validation to divide the data set into training set, validation set and test set, input the preprocessed glioma image into the STNet segmentation network, and use the shallow convolution layer to obtain the glioma MR image feature map;

[0088] Step 2: Send the features to the parallel layer, and use the convolution layer and Swin Transformer (ST) layer to further extract the features. The convolution layer is repeated M1 times in units consisting of convolution operation and global average pooling (GAP) layer, and the ST block is repeated M2 times in units consisting of shift window and multi-head self-attention mechanism (MSA);

[0089] Among them, the calculation function of the parallel layer is:

[0090] y i =conv M1 (xi-1 )+ST M2 (x i-1 );

[0091] In the formula, y i represents the output of the i-th level, x i-1 It represents the result of the i-1th level, M1 represents the number of convolution block repetitions, and M2 represents the number of ST block repetitions.

[0092] Step 3: After fusing the outputs of the convolutional layer and the ST layer through addition operations, the Softmax layer is used to convert the value of each channel into a probability value, and then the encoder and decoder of the network are connected using the skip connection technology;

[0093] Step 4: After each upsampling operation, the feature map of the current decoding layer is spliced ​​with the feature map of the corresponding layer in the encoding stage in the channel dimension. The splicing operation of the feature map is performed on each layer to combine the high-resolution features of the shallow layer with the low-resolution features of the deep layer to retain more comprehensive feature information;

[0094] Step 5. Use a composite loss function composed of a weighted cross entropy loss function and a dice loss function to calculate the error between the segmentation result of the sequence image automatic segmentation model and the corresponding true label, and back-propagate it to update the parameters of the sequence image automatic segmentation model to optimize the segmentation performance of the sequence image automatic segmentation model.

[0095] Among them, the expression of the weighted cross entropy loss function is:

[0096]

[0097] The dice loss function is:

[0098]

[0099] Then the composite loss function is:

[0100] L=L dice +L wce ;

[0101] In the formula, L represents the composite loss function, C represents the total number of classes, i represents the i-th class, and w i represents the weight of the i-th class, y i is the true segmentation label of the i-th category, p i is the predicted segmentation label of the i-th class, n represents the total number of pixels, P j represents the predicted probability of the jth pixel, g j represents the true probability of the jth pixel, ε represents a constant, which is a very small number used to prevent the denominator from being zero, and can be taken as 0.01 here.

[0102] S3. Map the segmentation features of the glioma scanning image to a geometric high-dimensional space, and obtain the segmentation result of the glioma scanning image based on the mapping result and a geometric iterative optimization algorithm.

[0103] See also Figure 5 In this embodiment, when mapping the segmentation features of the glioma scanning image to a geometric high-dimensional space and obtaining the segmentation results of the glioma scanning image based on the mapping results and the geometric iterative optimization algorithm, the segmentation features of the glioma scanning image can be converted into feature vectors using a convolutional layer, and the feature vectors are mapped to a geometric high-dimensional space as inputs to the geometric iterative optimization algorithm; the confidence of the feature vectors is iteratively updated alternately using the geometric iterative optimization algorithm, the feature vectors are updated based on the confidence results, and the optimal feature vectors are found as feature fusion results; the feature fusion results are reshaped using inverse convolution to obtain a feature map, and the feature map is input into the decoding stage of the segmentation network to obtain the fused calibrated segmentation results.

[0104] Specifically, the confidence of the feature vector is iteratively and alternately updated using the geometric iterative optimization algorithm, the feature vector is updated based on the confidence result, and when the optimal feature vector is found as the feature fusion result, the confidence of the high-dimensional segmentation vector can be pre-initialized after the feature vector is mapped to the geometric high-dimensional space, and the predicted output of the feature vector under the confidence is judged; the confidence of the feature vector and the high-dimensional segmentation vector is iteratively optimized using the loss function, and the feature vector with the best confidence is selected as the true feature vector; the true feature vector is repeatedly updated within the range of the number of iterations based on the optimal confidence, and the geometric iterative update of the true feature vector is terminated when the trend of the true feature vector is stable; based on the update result, the true feature vector whose confidence is consistent with the true segmentation is obtained, and the true feature vector is used as the feature fusion result.

[0105] Among them, when the loss function is used to iteratively optimize the confidence of the feature vector and the high-dimensional segmentation vector, and the feature vector corresponding to the optimal confidence is selected as the true feature vector, the difference value between the predicted output and the true segmentation label can be calculated based on the loss function, the difference value can be used to measure the distance between the predicted distribution and the actual distribution, and the feature vector gradient can be analyzed according to the distance result; based on the feature vector gradient, the confidence of the feature vector and the high-dimensional segmentation vector is repeatedly updated using a geometric iterative optimization algorithm, and the distance between the predicted distribution and the actual distribution is obtained based on the update result; the feature vector and the confidence of the high-dimensional segmentation vector corresponding to the minimum distance value are selected as the true feature vector and the optimal confidence, respectively.

[0106] Specifically, the optimal confidence function is:

[0107]

[0108] The update function of the true feature vector δ* is:

[0109]

[0110] In the formula, S represents the total number of segmentation model training times, δ represents the segmentation feature vector, ε represents the confidence of the segmentation feature vector, and ε S represents the optimal confidence function of the segmentation model during the Sth training, δ S represents the segmentation feature vector of the segmentation model during the Sth training, δ i-1 represents the segmentation feature vector corresponding to the i-1th class, δ i Represents the segmentation feature vector corresponding to the i-th class.

[0111] S4. Use the automatic segmentation model of sequential images to output the segmentation results of real-time glioma scan images, and match the segmentation results with the image labeling results to obtain glioma characteristics.

[0112] In order to facilitate understanding of the above technical solutions of the present invention, the working principle or operation mode of the present invention in the actual process is described in detail below.

[0113] Glioma MR images were collected in advance. In order to solve the problem of different intensity ranges and contrast levels of glioma MR images and avoid affecting the stability and convergence in later use, the voxels with zero values ​​in the image were firstly cropped to remove redundant background voxels and enhance the contrast. The image data was normalized by subtracting the mean value from the image pixels and dividing by the standard deviation. Then, the spatial resolution of the image was adjusted by the neighbor interpolation algorithm. All images were scaled to the same size. Four sequences of images with the same contrast were merged to form a three-dimensional image with four channels. The image and the corresponding mask were divided into multiple patches. The sliding window was moved on the image to extract the corresponding patches. Based on the fact that each patch has its position information in the original image and the mask, a hot encoding operation was performed according to the category label of each pixel in the patch.

[0114] Moreover, traditional segmentation algorithms may lose a large amount of image details and local information during the downsampling process of the encoder, affecting the accuracy of the segmentation results. At the same time, there will be gradient vanishing or explosion problems for deep network structures, making the network difficult to train and optimize, and unable to solve the data imbalance problem of glioma data. First, parallel layers are used to simultaneously extract richer local and global features of glioma MR images. Secondly, the jump connection technology is used to connect the encoder and decoder of the segmentation network. After each upsampling operation, the feature map of the current decoding layer is spliced ​​with the feature map of the corresponding layer in the encoding stage in the channel dimension. The feature map splicing operation is performed on each layer, and the high-resolution features of the shallow layer are combined with the low-resolution features of the deep layer to retain more comprehensive feature information. A composite loss function composed of a weighted cross entropy loss function and a dice loss function is used to optimize the segmentation performance to address the problem of imbalanced distribution of lesion areas.

[0115] After the optimization is completed, multiple sequences of glioma MR images of the same case are simultaneously input into the STNet segmentation network. The high-level features of different sequences extracted in the encoding stage are passed through a 1×1 convolutional layer to obtain feature vectors, which are then mapped to a high-dimensional space as the input of the geometric iterative optimization algorithm. The confidence of each high-dimensional segmentation vector is initialized, and the cross-entropy loss function is used to calculate the difference between the predicted output and the true label based on the predicted output of each feature vector at the current confidence level. The distance between the predicted probability distribution and the actual distribution is measured, and the gradient of the loss function relative to the feature vector is calculated. The feature vector is updated according to the gradient using the optimization algorithm to minimize the cross-entropy loss.

[0116] Update the vector and confidence alternately to obtain the optimal confidence. After determining the optimal confidence, update the true feature vector. Optimize it repeatedly in the number of iterations. When the true feature vector tends to be stable, terminate the geometric iteration update of the true vector and confidence, and obtain the true feature vector whose confidence and true probability tend to be consistent. Then optimize the result of the previous iteration to obtain the optimal true feature vector as the feature fusion result. Finally, reshape the optimal feature vector into a feature map through the inverse convolution operation, input it into the decoder of the deep convolutional neural segmentation network, and output the fused segmentation result.

[0117] In summary, with the help of the above technical scheme of the present invention, the present invention collects a number of glioma sequence scanning images, and has experts manually outline and annotate them with segmentation labels, and performs preprocessing operations to standardize them to form a multi-sequence glioma MR image data set, and uses a geometric iterative optimization algorithm to calibrate the glioma MR multi-sequence image segmentation to achieve the fusion of different sequence MR image feature information, which can solve the problem of ignoring the correlation of multi-sequence MR data feature information and provide auxiliary decision support for clinical diagnosis; the image segmentation model proposed in the present invention is based on the joint influence of multi-sequence glioma images, and through improving the iterative optimization fusion algorithm, the feature information extracted by the calibration model is obtained to obtain a more accurate glioma segmentation result. The results can further improve the accuracy of glioma MR multi-sequence image segmentation, provide a more reliable basis for clinical diagnosis and treatment, and use the parallel module to simultaneously extract richer local and global features of glioma MR images, providing an efficient and reliable solution for medical image segmentation; the automatic segmentation of glioma images using the segmentation network proposed in the present invention can extract richer and more comprehensive glioma MR image features, improve the accuracy of glioma MR image segmentation, and realize the calibration of the deep convolutional network model based on the geometric iterative optimization algorithm, which solves the problem of limited calibration of experimental results caused by the non-optimal fusion of multi-sequence MR image features by the traditional deep convolutional network model, thereby improving the performance of the model and the reliability of the experimental results.

[0118] Those skilled in the art will appreciate that the units, i.e., algorithm steps, of the various examples described in conjunction with this embodiment can be implemented in electronic hardware or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0119] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A multi-sequence image segmentation method based on geometric iterative optimization fusion, characterized in that: The multi-sequence image segmentation method includes the following steps: The collected glioma scan images were labeled by manual delineation, and the patch extraction technology was used to pre-process the glioma scan images; The processed glioma scan images are combined with the segmentation network architecture to construct a sequence image automatic segmentation model, and the sequence image automatic segmentation model is used to output the segmentation features of the glioma scan images; Map the segmentation features of the glioma scan image to a geometric high-dimensional space, and obtain the segmentation result of the glioma scan image based on the mapping result and the geometric iterative optimization algorithm; The automatic segmentation model of sequential images is used to output the segmentation results of real-time glioma scan images, and the segmentation results are matched with the image labeling results to obtain the glioma characteristics; The step of mapping the segmentation features of the glioma scan image to a geometric high-dimensional space and obtaining the segmentation result of the glioma scan image based on the mapping result and a geometric iterative optimization algorithm comprises the following steps: The convolutional layer is used to convert the segmentation features of the glioma scan image into a feature vector, and the feature vector is mapped to a geometric high-dimensional space as the input of the geometric iterative optimization algorithm; The confidence of the feature vector is updated iteratively and alternately using the geometric iterative optimization algorithm, the feature vector is updated based on the confidence result, and the optimal feature vector is found as the feature fusion result; The feature fusion result is reshaped by using inverse convolution to obtain a feature map, and the feature map is input into the decoding stage of the segmentation network to obtain the fused calibrated segmentation result.

2. The multi-sequence image segmentation method based on geometric iterative optimization fusion according to claim 1 is characterized in that: The method of labeling the collected glioma scan images by manual delineation and preprocessing the glioma scan images by using patch extraction technology includes the following steps: Acquire a corresponding set of sequential glioma scanning images based on a preset magnetic resonance imaging parameter range value, and verify the quality level of the glioma scanning images; Adjust the clarity and contrast of the glioma scan image according to the verification results, and perform image registration processing after the adjustment is completed; The annotation protocol was defined based on the glioma type information, and the manual annotator combined the annotation protocol with the image processing software to manually outline the boundary information of the glioma; Label the glioma lesion type based on the boundary information results, and combine all the labeling results to perform cross-validation to compare the accuracy of the labeling results; If the accuracy is greater than or equal to the preset reference value, the lesion label type is saved in the specified file format, and the standard parameters generated by the annotation process are recorded. If the accuracy is less than the preset reference value, the boundary information delineation operation is re-executed; The image pixels of the glioma scanning image are obtained, the glioma scanning image is normalized according to the image pixels, and the glioma scanning image is subjected to a hot encoding operation using a patch extraction technique.

3. The multi-sequence image segmentation method based on geometric iterative optimization fusion according to claim 2 is characterized in that: The method of obtaining image pixels of the glioma scanning image, normalizing the glioma scanning image according to the image pixels, and performing a thermal encoding operation on the glioma scanning image using a patch extraction technique comprises the following steps: Based on the voxel value of the glioma scan image, the redundant background with zero voxel value is cropped and removed, and the glioma scan image is normalized using the image pixels; The nearest neighbor interpolation algorithm was used to scale the glioma scan images to the same size, and the glioma scan images with the same contrast were merged to form a channel three-dimensional image; The channel 3D image and the corresponding mask are divided into several patches, and the corresponding patches are extracted by moving the sliding window on the channel 3D image; A hot encoding operation is performed according to the lesion label type corresponding to each pixel in the patch to obtain the channel area corresponding to the lesion label type.

4. The multi-sequence image segmentation method based on geometric iterative optimization fusion according to claim 3 is characterized in that: The method of cutting out redundant background with zero voxel value based on the voxel value of the glioma scan image and normalizing the glioma scan image using image pixels comprises the following steps: Initialize a labeling matrix with the same dimension as the original glioma scan image based on the three-dimensional image matrix of the glioma scan image, and traverse the voxel values ​​of the glioma scan image; Analyze the modal value of the voxel value in the lesion label type. If the modal value is zero, it is defined as a background voxel, and the corresponding position of the background voxel in the labeling matrix is ​​marked as zero; If the modality value is non-zero, it is defined as a normal voxel, and the corresponding position of the normal voxel in the labeling matrix is ​​marked as one; Calculate the minimum bounding box of non-zero voxels in the label matrix, crop to remove redundant background with zero voxel value, and retain the area of ​​non-zero voxels; The normalization range is defined based on the image pixels, and the maximum and minimum pixel intensity values ​​are selected through the glioma scanning image, and the normalization processing is performed using the value selection results.

5. The multi-sequence image segmentation method based on geometric iterative optimization fusion according to claim 1 is characterized in that: The process of combining the processed glioma scan image with the segmentation network architecture to construct a sequence image automatic segmentation model, and using the sequence image automatic segmentation model to output the segmentation features of the glioma scan image comprises the following steps: The processed glioma scan images were divided into training set, validation set and test set according to the preset ratio by cross-validation method, and the segmentation network was used as the architecture of the automatic segmentation model of the sequence images; Input the training set into the recognition and segmentation network, use the convolution layer to obtain the feature map of the training set, and input the feature result into the parallel layer; In the parallel layer, the convolution block and the window self-attention block are used to repeatedly extract the features according to the preset extraction rules to obtain the underlying features; The output layer is used to convert the underlying features into probability values ​​to obtain the segmentation features corresponding to the lesion label type channel area, and the composite loss function is used to verify the output error optimization segmentation network; The optimized segmentation network is used as the automatic segmentation model for sequential images, and the automatic segmentation model for sequential images is used to output the segmentation features of glioma scan images.

6. The multi-sequence image segmentation method based on geometric iterative optimization fusion according to claim 5 is characterized in that: The method of converting the underlying features into probability values ​​using the output layer to obtain the segmentation features corresponding to the lesion label type channel area, and verifying the output error optimization segmentation network using the composite loss function includes the following steps: The output layer receives the underlying features, converts the feature values ​​corresponding to the underlying features into probability values, and defines the pixel points corresponding to the lesion label type channel area based on the probability value conversion results; The encoder and decoder of the segmentation network are connected using the skip connection technology. After each upsampling operation, the feature map of the current decoding layer is concatenated with the feature map of the corresponding layer in the encoding stage in the channel dimension. The high-resolution features are combined with the low-resolution features to obtain the segmentation results of the glioma scan images, and the composite loss function is defined using the weighted cross entropy loss function and the dice loss function; The predicted segmentation label of the segmentation network output is calculated based on the composite loss function, and the predicted segmentation label is compared with the true segmentation label to obtain the error result, and the error result is used to back propagate and update the optimized segmentation network parameters.

7. The multi-sequence image segmentation method based on geometric iterative optimization fusion according to claim 6 is characterized in that: The expression of the composite loss function is: In the formula, L represents the composite loss function, C represents the total number of classes, i represents the i-th class, and w i represents the weight of the i-th class, y i is the true segmentation label of the i-th category, p i is the predicted segmentation label of the i-th class, n represents the total number of pixels, P j represents the predicted probability of the jth pixel, g j represents the true probability of the jth pixel, and ε represents a constant.

8. The multi-sequence image segmentation method based on geometric iterative optimization fusion according to claim 1 is characterized in that: The method of iteratively and alternately updating the confidence of the feature vector using the geometric iterative optimization algorithm, updating the feature vector based on the confidence result, and finding the optimal feature vector as the feature fusion result includes the following steps: After mapping the feature vector to a geometric high-dimensional space, the confidence of the high-dimensional segmentation vector is initialized in advance, and the predicted output of the feature vector under the confidence is determined; The confidence of the feature vector and the high-dimensional segmentation vector is iteratively optimized using the loss function, and the feature vector with the best confidence is selected as the true feature vector; Repeatedly updating the true feature vector within the range of iteration times based on the optimal confidence, and terminating the geometric iteration update of the true feature vector when the trend of the true feature vector is stable; Based on the updated result, a true feature vector whose confidence is consistent with the true segmentation is obtained, and the true feature vector is used as the feature fusion result.

9. The multi-sequence image segmentation method based on geometric iterative optimization fusion according to claim 8, characterized in that: The method of iteratively optimizing the confidence of the feature vector and the high-dimensional segmentation vector using the loss function and selecting the feature vector corresponding to the best confidence as the true feature vector includes the following steps: The difference between the predicted output and the true segmentation label is calculated based on the loss function, the difference is used to measure the distance between the predicted distribution and the actual distribution, and the feature vector gradient is analyzed based on the distance result; Based on the gradient of the feature vector, the confidence of the feature vector and the high-dimensional segmentation vector is repeatedly updated using a geometric iterative optimization algorithm, and the distance between the predicted distribution and the actual distribution is obtained based on the update result; The feature vector corresponding to the minimum distance value and the confidence of the high-dimensional segmentation vector are selected as the true feature vector and the optimal confidence respectively.

Images