WMH image segmentation system with mixed scale attention

Abstract

The invention relates to a WMH image segmentation system with mixed scale attention, which belongs to the technical field of image segmentation and comprises an acquisition unit, an enhancement unit, a segmentation unit, an evaluation unit and a three-dimensional reconstruction unit. Firstly, file format conversion is carried out, and a WMH image data set is obtained; secondly, processing WMH image data by means of data enhancement to obtain a high-quality training image data set; and inputting the WMH image data set into the MixAtten-Unet deep neural network model, and carrying out image segmentation and three-dimensional reconstruction processes to obtain a three-dimensional model of the WMH image data of the patient. A mixed scale idea is introduced into the model, and a mixed convolutional layer and an image mixed scale attention mechanism are introduced, so that the model can consider multi-scale feature data during feature processing, and more accurate white matter high-signal segmentation is realized. In addition, three-dimensional reconstruction of the segmented image is realized through a Marking Cubes algorithm, and a new thought and method are provided for quantitative analysis and reconstruction of WMH.

Description

Technical Field

[0001] The present invention relates to the technical field of image segmentation, and in particular to a mixed-scale attention WMH image segmentation system. Background Art

[0002] White matter hyperintensities (WMH) refer to high-signal lesions in the white matter area on magnetic resonance imaging (MRI) images. Such lesions are common in the elderly and are closely related to a variety of neurological diseases such as stroke and dementia. At present, the detection of WMH mainly relies on doctors to manually annotate MRI images. Although this method can provide accurate diagnostic results to a certain extent, its limitations are also obvious. First, manual annotation relies on the doctor's professional knowledge and experience, which is highly subjective, and there may be significant differences in the judgments of different doctors. Secondly, the manual annotation process is time-consuming and laborious, and the efficiency is obviously insufficient when a large number of images need to be analyzed. In addition, factors such as doctor fatigue and distraction can also affect the accuracy of the test, which may lead to missed or misdiagnosed cases.

[0003] With the rapid development of computer technology and artificial intelligence (AI), computer-aided detection is increasingly used in medical image analysis. In particular, deep learning technology, especially neural networks, has shown great potential and advantages in image recognition and segmentation tasks. Neural networks can automatically learn features in images and perform efficient feature extraction and classification. They have been successfully applied in the detection of various diseases such as lung cancer and breast cancer. The automatic image segmentation method based on deep learning has the advantages of fast segmentation speed and high segmentation accuracy, and has a wider range of applicability. Many existing methods cannot directly display the reconstruction effect and cannot well assist doctors in quantitative analysis. Summary of the invention

[0004] The present invention is proposed to alleviate or solve at least one aspect or at least one point of the above problems.

[0005] A mixed scale attention WMH image segmentation system of the present invention comprises:

[0006] An acquisition unit is used to acquire a WMH image dataset; select an MRI with white matter hyperintensities, and convert the MRI data file format from a DICOM format to an image format, so as to acquire a WMH image dataset as a training set for a MixAtten-Unet deep neural network model;

[0007] The enhancement unit processes the WMH image data by means of data enhancement to obtain a high-quality training image data set. The means of data enhancement refers to using histogram equalization technology to perform data processing;

[0008] The segmentation unit uses the MixAtten-Unet deep neural network model to segment the input WMH image dataset;

[0009] An evaluation unit, which evaluates the WMH image segmentation method trained by the MixAtten-Unet deep neural network model;

[0010] The 3D reconstruction unit, after the model accurately segments the WMH image, uses the segmented image to perform 3D reconstruction.

[0011] Preferably, the MixAtten-Unet deep neural network model includes an input module, a feature extraction module, a skip connection module, an image restoration module and a prediction module;

[0012] The input module is used to convert patient image data into vector input for the MixAtten-Unet deep neural network model;

[0013] The feature extraction module is used to gradually reduce the spatial resolution of the image and increase the depth of the image feature channel to extract features from the image data;

[0014] The skip connection module connects the high-resolution feature map in the feature extraction module with the corresponding layer in the image restoration module, so that the model can retain more detailed information in the feature extraction module;

[0015] The image restoration module is used to gradually restore the spatial resolution of the image after feature encoding, to distinguish and extract the feature area of ​​the model, and to increase the information of image learning;

[0016] The prediction module changes the decoded output to have the same spatial resolution as the input image, and finally predicts the segmented image to obtain the WMH area in the data.

[0017] Preferably, the feature extraction module includes five feature encoding blocks:

[0018] The first feature encoding block includes two single convolutional layers, one hybrid convolutional layer, and one scale-adaptive attention layer; the input of the first feature encoding block passes through the single convolutional layer, the hybrid convolutional layer, and the scale-adaptive attention layer in sequence;

[0019] The second to fifth feature encoding blocks add a mixed-scale downsampling layer based on the first feature encoding block.

[0020] The output of the input module is connected to the first feature encoding block;

[0021] The input of the first single convolutional layer in the feature encoding block is added to the output of the scale-adaptive attention layer to obtain the output of the feature encoding block.

[0022] Preferably, the image restoration module includes four feature decoding blocks:

[0023] The feature decoding block consists of an upsampling layer, three single convolutional layers, and a scale-adaptive attention layer in sequence;

[0024] After the input of the feature decoding block undergoes feature size transformation through the upsampling layer, it is jump-connected with the output feature map of the corresponding feature encoding block to obtain the input of the first single convolutional layer of the feature decoding block. The input of the first single convolutional layer of the feature decoding block is added to the output of the scale-adaptive attention layer of the feature decoding block to obtain the output of the feature decoding block.

[0025] Preferably, the skip connection module includes four skip connection blocks:

[0026] The first skip connection block adds the output of the first feature encoding block to the upsampling layer output of the fourth feature decoding block; the second skip connection block adds the output of the second feature encoding block to the upsampling layer output of the third feature encoding block; the third skip connection block adds the output of the third feature encoding block to the upsampling layer output of the second feature decoding block; the fourth skip connection block adds the output of the fourth feature encoding block to the upsampling layer output of the first feature decoding block.

[0027] Preferably, the scale-adaptive attention layer includes a channel self-attention module and a spatial self-attention module:

[0028] The channel self-attention module processes the input feature map through a 3×3×3 convolution and a dilated convolution respectively, and adds the features of the two processed feature maps, sequentially passes through the average pooling layer and the fully connected layer to obtain the importance matrix, which is multiplied with the initial original to obtain the output of the channel self-attention module;

[0029]

[0030]

[0031]

[0032]

[0033] in represents the feature vector after 3x3x3 convolution, W 3*3*3 represents the 3x3x3 convolution kernel weight matrix, represents the input vector, represents the feature vector after dilated convolution, W Dilated represents the dilated convolution weight matrix, represents the input vector, represents the attention coefficient, GAP() represents the global average pooling operation, FC() represents the fully connected layer, Represents the output vector.

[0034] Preferably, the spatial self-attention module adds the initial feature map to the output of the channel self-attention module through a 1×1×1 convolution to obtain the original input, and sequentially passes the feature map through a 1×1×1 single convolution layer and a resampling layer, and then passes through a sigmoid function to obtain the attention coefficient, which is multiplied by the initial feature map and the output of the channel self-attention module and then added to obtain the final output:

[0035]

[0036]

[0037]

[0038] in Represents the feature vector after 1x1x1 convolution, W 1*1*1 Represents a 1x1x1 convolution kernel weight matrix, represents the vector input of the spatial self-attention module, sigmoid() represents the sigmoid activation function, represents the output of the channel self-attention module, represents the vector output of the spatial self-attention module, Represents the feature vector after 1x1x1 convolution.

[0039] Preferably, the evaluation unit selects Dice coefficient and MPA as evaluation indicators.

[0040] Compared with the prior art, the present invention proposes a mixed-scale attention WMH image segmentation model, called MixAttention-Unet, for the problem of WMH image segmentation and three-dimensional reconstruction. The model introduces the idea of ​​mixed scales. By introducing mixed convolutional layers and image mixed-scale attention mechanisms, the model can take into account multi-scale feature data during feature processing, thus achieving more accurate white matter hyperintensity segmentation. In addition, the Marching Cubes algorithm is used to achieve three-dimensional reconstruction of the segmented image, providing new ideas and methods for the quantitative analysis and reconstruction of WMH. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1The figure is a flow chart of the image segmentation method of the present invention.

[0042] Figure 2 Schematic diagram of the structural framework of the WMH image segmentation model with mixed scale attention in an exemplary embodiment of the present invention.

[0043] Figure 3 Schematic diagram of a hybrid convolutional layer in an exemplary embodiment of the present invention.

[0044] Figure 4 Schematic diagram of a hybrid downsampling layer in an exemplary embodiment of the present invention.

[0045] Figure 5 Schematic diagram of a scale-adaptive attention layer in an exemplary embodiment of the present invention.

[0046] Figure 6 FIG. 4 is a schematic diagram of an image segmentation system according to an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0047] The following description of the embodiments of the present invention with reference to the accompanying drawings is intended to explain the overall inventive concept of the present invention, and should not be construed as a limitation of the present invention. In the present invention, the same reference numerals represent the same or similar components.

[0048] The features described herein can be implemented in different forms and should not be construed as being limited to the examples described herein. Rather, the examples described herein have been provided to illustrate only some of the many possible ways to implement the methods, devices, and / or systems described herein, which will be clear after understanding the disclosure of the present invention.

[0049] Although terms such as "first", "second", and "third" may be used herein to describe various members, components, regions, layers, or portions, these members, components, regions, layers, or portions should not be limited by these terms. Instead, these terms are only used to distinguish one member, component, region, layer, or portion from another member, component, region, layer, or portion.

[0050] In the specification, when an element (such as a layer, a region, or a substrate) is described as being “on”, “connected to”, or “coupled to” another element, the element may be directly “on”, “connected to”, or “coupled to” another element, or one or more other elements may be present therebetween. Conversely, when an element is described as being “directly on”, “directly connected to”, or “directly coupled to” another element, there may be no other elements present therebetween.

[0051] The terms used herein are only used to describe various examples and are not intended to limit the disclosure. Unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. The terms "comprise", "include" and "have" indicate the presence of the described features, quantities, operations, components, elements and / or combinations thereof, but do not exclude the presence or addition of one or more other features, quantities, operations, components, elements and / or combinations thereof.

[0052] In order to enable those skilled in the art to use the contents of the present invention, the following exemplary embodiments may be provided in combination with specific application scenarios, specific systems, parameters of devices and components, and specific connection methods. However, for those skilled in the art, these embodiments are only examples, and the general principles defined herein may be applied to other embodiments and application scenarios without departing from the spirit and scope of the present invention.

[0053] According to an exemplary embodiment of the present invention: Figure 1 , 6 As shown, a mixed scale attention WMH image segmentation system of the present invention includes:

[0054] An acquisition unit is used to acquire a WMH image dataset; select an MRI with white matter hyperintensities, and convert the MRI data file format from a DICOM format to an image format, so as to acquire a WMH image dataset as a training set for a MixAtten-Unet deep neural network model;

[0055] The enhancement unit processes the WMH image data by means of data enhancement to obtain a high-quality training image data set. The means of data enhancement refers to using histogram equalization technology to perform data processing;

[0056] The segmentation unit uses the MixAtten-Unet deep neural network model to segment the input WMH image dataset;

[0057] An evaluation unit, which evaluates the WMH image segmentation method trained by the MixAtten-Unet deep neural network model;

[0058] The 3D reconstruction unit, after the model accurately segments the WMH image, uses the segmented image to perform 3D reconstruction.

[0059] like Figure 1-3 As shown, a mixed scale attention WMH image segmentation method of the present invention comprises the following steps:

[0060] S1. Obtain a WMH image dataset; select MRI with high white matter signals in the WMH image dataset, and convert the data file format from DICOM format to an image format, such as PNG format, to obtain a training set for MixAtten-Unet;

[0061] Specifically, the original WMH image data of the hospital is only a file in DICOM format. DICOM files contain not only image data, but also rich metadata, such as patient information, scanning parameters, and detailed information of image acquisition. This format is widely used in a variety of medical imaging devices such as CT, MRI, and ultrasound, ensuring interoperability between different devices and software. Because DICOM files are usually large and complex, they require specialized software tools to view and process, and cannot be used as the original input of the image segmentation model.

[0062] The preprocessing method for WMH images is as follows:

[0063] (1) To convert the original DICOM file into a PNG file, you only need to read the image file in the DICOM, normalize it, then expand its pixel value to the range of 0 to 255, then adjust the image size, convert it to RGB format, and finally save it as a PNG file.

[0064] (2) Since the mixed-scale attention WMH image segmentation method requires formal labels to update the model during the model training process, Labelme is used to annotate the label part in the image as the label value of the model. The output of Labelme is in json format, which needs to be converted to png format of the same size as the original image as the label corresponding to the original data.

[0065] (3) In addition, in order to ensure the performance of the model, for images that do not have WMH signals, a blank image is added as its label to enrich the diversity of the dataset.

[0066] For this dataset, we finally constructed the input set of the interactive segmentation method, and the data was divided into a training set and a test set in a ratio of 8:2. The training set is used for training the image segmentation of white matter hyperintensities, and the test set is used for testing the image segmentation of white matter hyperintensities.

[0067] S2, processing WMH images by means of data enhancement to obtain a high-quality training image set;

[0068] The method of data enhancement refers to the use of histogram equalization technology to process data. Histogram equalization technology is a commonly used image processing technology. It redistributes the grayscale values ​​of the image to make its histogram more uniform throughout the grayscale range, thereby enhancing the contrast of the image.

[0069] Specifically, this method calculates the cumulative distribution function (CDF) of the input image and then uses this function to map the grayscale values ​​of the original image to the new grayscale values. This process can enhance the contrast of the image, especially for images with grayscale values ​​concentrated in a specific range. It helps to solve the problem of limited annotated data in the field of medical imaging and reduce overfitting during network training by expanding the amount of data.

[0070] S3. Based on the mixed scale module and the attention module, a MixAtten-Unet deep neural network model is established; the overall architecture is as follows Figure 2 As shown:

[0071] The MixAtten-Unet model includes an input module, a feature extraction module, an image restoration module, a skip connection module, and a prediction module;

[0072] The input module is used to convert patient image data into vector input of the model;

[0073] The feature extraction module is used to gradually reduce the spatial resolution of the image and increase the depth of the image feature channel to extract features from the data image;

[0074] The skip connection connects the high-resolution feature map in the feature extraction module with the corresponding layer in the image restoration module, and the model is used to retain more detailed information in the feature extraction module;

[0075] The image restoration module is used to gradually restore the spatial resolution of the encoded image and to distinguish and extract the characteristic areas of the model;

[0076] The prediction module changes the decoded output to have the same spatial resolution as the input image, and finally obtains the segmented image and the WMH area in the data.

[0077] MixAtten-Unet adds mixed convolutional layers, mixed downsampling layers and scale-adaptive attention layers to Unet.

[0078] The hybrid convolutional layer has three different branches for feature extraction, such as Figure 3 As shown:

[0079] The first branch uses 1×1×1 convolution with a step size of 1 and padding of 0, which is mainly used to maintain the local information of the feature;

[0080] The second branch uses 3×3×3 convolution with a stride of 1 and padding of 1 to increase the receptive field of the convolution, extract large-scale image features, and make the output size the same as the input size;

[0081] The third branch uses 3×3×3 dilated convolution with a dilation rate of 2 and a padding of 2 to extract features.

[0082]

[0083] Among them, F out represents the output vector, W 1*1*1 represents the weight matrix of the first branch, W 3*3*3 represents the weight matrix of the second branch, F in represents the input vector, W Dilated Represents the weight matrix of the third branch.

[0084] Finally, the outputs of the three branches are added element by element to obtain an output with unchanged size. The introduction of the hybrid convolutional layer can take into account information of different scales in the feature extraction process, improving the robustness and expression ability of the network.

[0085] The hybrid downsampling layer is similar to the hybrid convolutional layer, and features are extracted by two different branches, such as Figure 4 As shown:

[0086] The first branch uses 2×2×2 maximum pooling, which is mainly used to maintain the local information of the feature;

[0087] The second branch uses 2×2×2 average pooling to increase the receptive field and extract large-scale image features.

[0088]

[0089] Among them, F out represents the output vector, W Max Pooling represents the maximum pooling weight matrix, W Average Pooling represents the average pooling weight matrix, F in Represents the input vector.

[0090] The scale-adaptive attention layer is mainly divided into two modules, such as Figure 5 As shown:

[0091] The channel self-attention module processes the input feature map through a 3×3×3 convolution and a dilated convolution respectively, and adds the features of the two processed feature maps, sequentially passes through the average pooling layer and the fully connected layer to obtain the importance matrix, which is multiplied with the initial original to obtain the output of the channel self-attention module;

[0092]

[0093]

[0094]

[0095]

[0096] in represents the feature vector after 3x3x3 convolution, W 3*3*3 represents the 3x3x3 convolution kernel weight matrix, represents the input vector, represents the feature vector after dilated convolution,

[0097] W Dilated represents the dilated convolution weight matrix, represents the input vector, represents the attention coefficient, GAP() represents the global average pooling operation, FC() represents the fully connected layer, Represents the output vector.

[0098] The spatial self-attention module adds the initial feature map through a 1×1×1 convolution and the output of the channel self-attention module to obtain the original input, and then passes the feature map through a 1×1×1 single convolution layer, a resampling layer, and a sigmoid function to obtain the attention coefficient, which is multiplied by the initial feature map and the output of the channel self-attention module and then added to obtain the final output;

[0099]

[0100]

[0101]

[0102] in Represents the feature vector after 1x1x1 convolution, W 1*1*1 Represents a 1x1x1 convolution kernel weight matrix, represents the vector input of the spatial self-attention module, sigmoid() represents the sigmoid activation function, represents the output of the channel self-attention module, represents the vector output of the spatial self-attention module, Represents the feature vector after 1x1x1 convolution.

[0103] The feature extraction module includes five feature encoding blocks:

[0104] The first feature encoding block consists of two single convolutional layers, a hybrid convolutional layer, and a scale-adaptive attention layer; the input of the feature encoding block passes through a single convolutional layer, a hybrid convolutional layer, and a scale-adaptive attention layer in sequence;

[0105] The remaining feature encoding blocks add a mixed-scale downsampling layer based on the first feature encoding block.

[0106] A residual connection is set between each feature coding block in the feature extraction module, and the input of the first single convolutional layer in the first feature coding block is added to the output of the scale adaptive attention layer in the feature coding block as the input of the second feature coding block;

[0107] The input of the first single convolutional layer in the second feature encoding block is added to the output of the scale adaptive attention layer in the feature encoding block as the input of the third feature encoding block;

[0108] The input of the first single convolutional layer in the third feature coding block is added to the output of the scale adaptive attention layer in the feature coding block as the input of the fourth feature coding block;

[0109] The input of the first single convolutional layer in the fourth feature coding block is added to the output of the scale adaptive attention layer in the feature coding block as the input of the fifth feature coding block;

[0110] The image restoration module consists of four feature decoding blocks:

[0111] Each feature decoding block includes an upsampling layer, a single convolutional layer, and a scale-adaptive attention layer in sequence;

[0112] Residual connections are also set between the feature encoding blocks in the image restoration module. The input of the first feature decoding block is subjected to feature size transformation through the upsampling layer, and then jump-connected with the output feature map of the fourth feature encoding block to obtain the input of the first single convolutional layer of the first feature decoding block. The input of the first single convolutional layer of the first feature decoding block is added to the output of the scale-adaptive attention layer of the first feature decoding block to obtain the output of the first feature decoding block.

[0113] Similarly, the input of the first single convolutional layer of the second and third feature decoding blocks is added to the output of their scale-adaptive attention layers to obtain the output of the feature decoding block;

[0114] The input of the fourth feature decoding block is transformed in feature size through the upsampling layer, and then jump-connected with the output feature map of the first feature encoding block to obtain the input of the first single convolutional layer of the fourth feature decoding block. The input of the first single convolutional layer of the fourth feature decoding block is added to the output of the scale-adaptive attention layer of the fourth feature decoding block to obtain the output of the fourth feature decoding block, that is, the final output: white matter cut map.

[0115] The skip connection module consists of four skip connection blocks:

[0116] The first skip connection block adds the output of the first feature encoding block to the upsampling layer output of the fourth feature decoding block; the second skip connection block adds the output of the second feature encoding block to the upsampling layer output of the third feature encoding block; the third skip connection block adds the output of the third feature encoding block to the upsampling layer output of the second feature decoding block; the fourth skip connection block adds the output of the fourth feature encoding block to the upsampling layer output of the first feature decoding block.

[0117] S4, inputting the WMH image dataset into the MixAtten-Unet deep neural network model;

[0118] Specifically, the method for extracting the features of the WMH image is as follows: the input module of the model first transforms the input image into the input format of the model through techniques such as histogram equalization; the feature extraction module includes five feature encoding blocks from top to bottom. As can be seen from the above, the feature encoding block includes two 3×3×3 single convolutional layers, a hybrid convolutional layer, and a scale-adaptive attention layer; the output of the input module first enters the 3×3×3 single convolutional layer, 3×3×3 single convolutional layer, hybrid convolutional layer, and scale-adaptive attention layer of the first feature encoding block in sequence; and the input of the first single convolutional layer is residually connected with the output of the scale-adaptive attention layer; finally, the number of channels of the WMH image obtained by the first feature encoding block is 64. After the output of the first feature encoding block passes through the hybrid scale downsampling layer of the second feature encoding block, the output is residually connected with the output of the scale-adaptive attention layer of the second feature encoding block. The number of channels of the second, third, fourth, and fifth feature encoding blocks are 128, 256, 512, and 1024.

[0119] The image restoration module of the model includes four feature decoding blocks, which in turn include two single convolutional layers and a scale-adaptive attention layer; the input of the single convolutional layer of each feature decoding block is the sum of the output of the previous feature decoding block and the feature map of the corresponding layer of the feature encoding part through a jump connection.

[0120] The number of channels of the output feature map of the fifth feature encoding block is 1024. First, the spatial dimension is reduced by half through the upsampling layer of the first feature decoding block, and then it is jump-connected with the output feature map of the fourth feature encoding block (the number of channels is 512). The number of channels of the connected feature map is 512. Then it successively enters the 3×3×3 single convolution layer, 3×3×3 single convolution layer, hybrid convolution layer and scale-adaptive attention layer to obtain an output feature map with a channel number of 512.

[0121] The number of channels of the output feature map of the first feature decoding block is 512. First, the spatial dimension is reduced by half through the upsampling layer of the second feature decoding block, and then it is jump-connected with the output feature maps (with 256 channels) of the third and fourth feature encoding blocks. The number of channels of the connected feature map is 256. Then it enters the 3×3×3 single convolutional layer, 3×3×3 single convolutional layer and scale-adaptive attention layer in turn to obtain an output feature map with 256 channels.

[0122] And so on, finally, the decoded output passes through a 3×3×3 single convolutional layer to transform the final number of feature map channels into the number of target categories, which is the final segmentation result.

[0123] S5. Evaluating the WMH image segmentation method trained by the MixAtten-Unet deep neural network model;

[0124] Two general indicators are selected to evaluate the model: Dice coefficient is used as an evaluation indicator of the spatial correspondence of each voxel between two segmentations. The Dice coefficient can measure the similarity between the WMH output by the segmentation network and the real sample. The value of the Dice coefficient is between 0 and 1. The larger the value, the closer the segmentation result is to the real value. The Dice coefficient formula is as follows:

[0125]

[0126] Among them, the prediction result of the segmentation network is |A|, the true result is |B|, and the intersection of the two is expressed as

[0127] |A∩B|.

[0128] In addition, in order to evaluate the accuracy of segmented pixels, the mean pixel accuracy (MPA) was used to analyze the experimental results of the two types of pixels in the WMH segmentation process. The MPA formula is as follows:

[0129]

[0130] The number of categories is n d Indicates that the number of correctly classified pixels is P ii Indicates that the number of misclassified pixels is P ij The value of MPA is also between 0 and 1. The larger the value, the higher the pixel accuracy.

[0131] S6. After the model accurately segments the WMH image, the segmented image is used for 3D reconstruction.

[0132] After accurately segmenting the white matter high signal, it is necessary to extract the isosurface based on the Marching Cubes algorithm from the segmented image, process the cubes (voxels) in the data field one by one, separate the cubes that intersect with the isosurface, and use interpolation to calculate the intersection of the isosurface and the cube edge.

[0133] A voxel is a grid of eight pixels arranged in order, with each voxel (except for the border) shared by eight voxels. In a volume element, there are three vertex values: those above or equal to the value are inside the surface, and those below the value are outside the surface. Figure 2 As shown. Figure 2 As shown in the figure, lk and lk+1 represent two different isosurfaces; (i, j, k) represents the vertices of a voxel. A vertex of a voxel has two possible states, so a voxel (8 vertices) has a total of 28 or 256 states. The gray value of a point in the voxel can be calculated using a trilinear interpolation equation.

[0134] f(x,y,z)=a0+a1x+a2y+a3z+a4xy+a5yz+a6zx+a7xyz

[0135] a0 to a7 represent the grayscale values ​​of the eight vertices of the voxel respectively; x, y, z represent the coordinate points within the voxel; f(x, y, z) represents the grayscale value of the point (x, y, z) within the voxel.

[0136] The 256 combinations can be reduced to 128 combinations by symmetry flipping (swapping the values ​​of vertices 0 and 1 in the boundary voxels). The 256 combinations can be reduced to 15 cases by rotational symmetry (the positions of vertices 0 and 1 in the boundary voxels are the same after rotation). The three-dimensional reconstruction of a series of segmented images is performed using the moving tetrahedron method. The moving tetrahedron algorithm divides the cubic elements in the moving cube algorithm into tetrahedrons. There are many ways to split, usually into 5 tetrahedrons, and then construct isosurfaces in the tetrahedrons. There are a total of 24 and 16 combinations. Through flipping and rotational symmetry, only one vertex in the boundary voxel is larger than the isosurface, so a triangular face is generated. If two vertices are larger than the isosurface, a quadrilateral face is generated. This paper uses the moving tetrahedron method to perform three-dimensional reconstruction of brain white matter and can achieve better modeling accuracy.

[0137] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes may be made to the embodiments and combinations of elements may be made without departing from the principles and spirit of the invention, the scope of the invention being defined by the appended claims and their equivalents.

Claims

1. A mixed-scale attention WMH image segmentation system, characterized in that: include: An acquisition unit is used to acquire a WMH image dataset; select an MRI with white matter hyperintensities, and convert the MRI data file format from a DICOM format to an image format, so as to acquire a WMH image dataset as a training set for a MixAtten-Unet deep neural network model; The enhancement unit processes the WMH image data by means of data enhancement to obtain a high-quality training image data set. The means of data enhancement refers to using histogram equalization technology to perform data processing; The segmentation unit uses the MixAtten-Unet deep neural network model to segment the input WMH image dataset; An evaluation unit, which evaluates the WMH image segmentation method trained by the MixAtten-Unet deep neural network model; The 3D reconstruction unit, after the model accurately segments the WMH image, uses the segmented image to perform 3D reconstruction.

2. The WMH image segmentation system according to claim 1, characterized in that: The MixAtten-Unet deep neural network model includes an input module, a feature extraction module, a skip connection module, an image restoration module and a prediction module; The input module is used to convert patient image data into vector input for the MixAtten-Unet deep neural network model; The feature extraction module is used to gradually reduce the spatial resolution of the image and increase the depth of the image feature channel to extract features from the image data; The skip connection module connects the high-resolution feature map in the feature extraction module with the corresponding layer in the image restoration module, so that the model can retain more detailed information in the feature extraction module; The image restoration module is used to gradually restore the spatial resolution of the image after feature encoding, to distinguish and extract the feature area of ​​the model, and to increase the information of image learning; The prediction module changes the decoded output to have the same spatial resolution as the input image, and finally predicts the segmented image to obtain the WMH area in the data.

3. The WMH image segmentation system according to claim 2, characterized in that: The feature extraction module includes five feature encoding blocks: The first feature encoding block includes two single convolutional layers, a hybrid convolutional layer, and a scale-adaptive attention layer; The input of the first feature encoding block passes through a single convolutional layer, a hybrid convolutional layer, and a scale-adaptive attention layer in sequence; The second to fifth feature encoding blocks add a mixed-scale downsampling layer based on the first feature encoding block. The output of the input module is connected to the first feature encoding block; The input of the first single convolutional layer in the feature encoding block is added to the output of the scale-adaptive attention layer to obtain the output of the feature encoding block.

4. The WMH image segmentation system according to claim 2, characterized in that: The image restoration module includes four feature decoding blocks: The feature decoding block consists of an upsampling layer, three single convolutional layers, and a scale-adaptive attention layer in sequence; After the input of the feature decoding block undergoes feature size transformation through the upsampling layer, it is jump-connected with the output feature map of the corresponding feature encoding block to obtain the input of the first single convolutional layer of the feature decoding block. The input of the first single convolutional layer of the feature decoding block is added to the output of the scale-adaptive attention layer of the feature decoding block to obtain the output of the feature decoding block.

5. The WMH image segmentation system according to claim 2, characterized in that: The skip connection module includes four skip connection blocks: The first skip connection block adds the output of the first feature encoding block to the upsampling layer output of the fourth feature decoding block; the second skip connection block adds the output of the second feature encoding block to the upsampling layer output of the third feature encoding block; the third skip connection block adds the output of the third feature encoding block to the upsampling layer output of the second feature decoding block; the fourth skip connection block adds the output of the fourth feature encoding block to the upsampling layer output of the first feature decoding block.

6. The WMH image segmentation system according to claim 3, characterized in that: The scale-adaptive attention layer includes a channel self-attention module and a spatial self-attention module: The channel self-attention module processes the input feature map through a 3×3×3 convolution and a dilated convolution respectively, and adds the features of the two processed feature maps, sequentially passes through the average pooling layer and the fully connected layer to obtain the importance matrix, which is multiplied with the initial original to obtain the output of the channel self-attention module; in represents the feature vector after 3x3x3 convolution, W 3*3*3 represents the 3x3x3 convolution kernel weight matrix, represents the input vector, represents the feature vector after dilated convolution, W Dilated represents the dilated convolution weight matrix, represents the input vector, represents the attention coefficient, GAP() represents the global average pooling operation, FC() represents the fully connected layer, Represents the output vector.

7. The WMH image segmentation system according to claim 6, characterized in that: The spatial self-attention module obtains the original input by adding the initial feature map through a 1×1×1 convolution and the output of the channel self-attention module. The feature map is sequentially passed through a 1×1×1 single convolution layer and a resampling layer, and then passed through a sigmoid function to obtain the attention coefficient, which is then multiplied with the initial feature map and the output of the channel self-attention module and then added to obtain the final output: in Represents the feature vector after 1x1x1 convolution, W 1*1*1 Represents a 1x1x1 convolution kernel weight matrix, represents the vector input of the spatial self-attention module, sigmoid() represents the sigmoid activation function, represents the output of the channel self-attention module, represents the vector output of the spatial self-attention module, Represents the feature vector after 1x1x1 convolution.

8. The WMH image segmentation system according to claim 2, characterized in that: The evaluation unit selected Dice coefficient and MPA as evaluation indicators.

Images