Image segmentation method and system based on wavelet scattering learning network

Abstract

The present invention discloses an image segmentation method and system based on a wavelet scattering learning network, 1) constructing a segmentation network model integrating a wavelet scattering network and a full convolutional network, the segmentation network model is based on an encoder-decoder structure of a full convolutional network; adding a double-tree complex wavelet scattering transform to each layer of the encoder to effectively separate the high-frequency features and low-frequency features of the feature map, and then using a convolution operator to select the low-frequency main features and high-frequency boundary features of the feature map respectively, and finally fusing them; 2) pre-processing the labeled image data set and inputting it into the segmentation network model to train it; 3) pre-processing the actual image data to be segmented and inputting it into the trained network model, so as to achieve image segmentation. The present invention can improve the feature extraction capability of the image boundary, can significantly improve the image segmentation accuracy, and the segmentation result shows a relatively clear and complete outline in each area of ​​the image.

Description

Technical Field

[0001] The present invention relates to an improvement in image segmentation technology, and in particular to an image segmentation method and system based on a wavelet scattering learning network. The present invention uses a wavelet scattering algorithm to improve a full convolutional network model, thereby realizing automatic image segmentation and three-dimensional reconstruction and visualization of the image, and belongs to the field of computer vision technology. Background Art

[0002] Image segmentation technology is widely used in autonomous driving, satellite image systems, robots, medical imaging and other fields. In the field of unmanned driving, high-quality semantic segmentation of road scenes can provide protection for the safe driving of autonomous vehicles. The segmentation of roads, forests, farmland organisms and other categories in satellite remote sensing images can be applied to the monitoring and management of resources and the environment. In medical images, the segmentation of lesions and tissues and organs can be achieved to assist medical workers in estimating the location and volume of lesions and tissues and organs.

[0003] As a prevalent disease with the highest mortality rate, the diagnosis and treatment of glioma has attracted the attention of medical workers. In the actual clinical diagnosis and treatment process, magnetic resonance imaging (MRI) technology is widely used for imaging diagnosis of the brain, spinal cord, heart and blood vessels, joint bones, soft tissues and pelvic cavity. Clinicians use multiple sequences of MRI imaging to analyze the pathological conditions of glioma, such as conventional brain images of Flair, T1, T1c, T2 (see Figure 1 ), and the task of brain tumor diagnosis and identification is mainly to focus on gliomas in the brain. Glioma is the most common brain tumor in adults. Due to the particularity of its anatomical structure, it can be visually examined and analyzed through MRI images. Due to the differences in clinical diagnosis and treatment between different individuals and the lack of medical resources for doctors, computer-assisted diagnosis and treatment has become an indispensable tool.

[0004] In the process of computer-assisted diagnosis and treatment of gliomas, an accurate and reliable glioma image segmentation step plays a key role. It can assist doctors in clinical diagnosis and treatment by quickly locating the brain lesion tumor area and distinguishing multiple types of tumors, and can be applied to medical teaching and medical technology fields such as virtual surgery. Since manual segmentation of glioma images is a very tedious and complicated task, and requires experienced clinical doctors to complete, this will not only greatly increase the doctors' extra working time, but also seriously affect the efficiency of doctors' clinical diagnosis and treatment work, and is also a waste of extremely valuable medical resources. For such repetitive and tedious tasks, it is more suitable for computers to handle them automatically.

[0005] At present, brain tumor segmentation methods are mainly divided into two categories. One is the unsupervised learning method. Kaur et al. proposed a hybrid multi-level threshold algorithm that combines intuitive fuzzy sets and tsallis entropy to automatically extract tumor tissue areas from fuzzy boundaries and poor contrast MR images. The algorithm lacks the extraction of semantic information and the overall segmentation accuracy is low. Singh et al. proposed an automatic sparse constrained level set method to achieve brain tumor segmentation in MR images. This method studies brain tumor images, finds the common features of brain tumor shapes, and constructs a sparse representation model. However, the algorithm only segments the overall area of ​​the brain tumor, without segmenting the edema and necrotic areas in the overall area. Sukanya et al. proposed a full-image segmentation method using an improved region growing method. For different images, the setting of the initial growth seed position requires manual intervention, which affects the execution efficiency of the algorithm. In general, the advantage of unsupervised learning segmentation algorithms is that they can segment images without labeled samples, but the accuracy of image segmentation of this type of algorithm is low, and the parameters need to be manually adjusted based on experience.

[0006] The other is supervised learning methods. With the development of big data, MRI image data samples of brain tumors are gradually enriched. Supervised learning deep learning segmentation methods have become the mainstream of segmentation algorithms. In deep learning, convolutional neural networks (CNNs) introduce semantic information when segmenting targets, injecting new vitality into semantic segmentation research. The fully convolutional network model based on the CNN architecture has achieved excellent performance in automatic medical image segmentation, further promoting the application of deep learning in image segmentation. The SegNet network adopts an encoder-decoder structure, which passes the pixel index value of the maximum pooling operation in the encoding process to the decoder, retains the detailed information of the pixel, and improves the accuracy of semantic segmentation. The attention mechanism Unet network (Attention Unet, abbreviated as At-Unet) adds attention gating units to the U-shaped fully convolutional network model, providing pixel-level attention for the feature map. The network tends to focus on feature points with more information, and also improves the feature extraction ability of the model. The Deeplab network obtains multi-scale context information by cascading dilated convolutions with different dilation rates, and then introduces conditional random fields to enhance the relevance of contextual semantic information, thereby improving segmentation accuracy. Although the above-mentioned network models have improved the image segmentation accuracy of some data sets, due to the ambiguity and infiltration of medical images such as tumors and adjacent organs, and the changes of lesions in different periods, the above-mentioned network models are still unable to accurately extract the boundary features of the image when processing medical imaging data, resulting in discontinuity in the segmentation boundaries. Summary of the invention

[0007] In view of the above shortcomings of the prior art, the present invention proposes an image segmentation method and system based on a wavelet scattering learning network, which can improve the feature extraction capability of image boundaries and significantly improve the image segmentation accuracy. The segmentation result shows relatively clear and complete contours in each area of ​​the image, and has high robustness in segmentation accuracy.

[0008] The technical solution of the present invention is achieved in this way:

[0009] The image segmentation method based on wavelet scattering learning network has the following steps:

[0010] 1) Construct a learning wavelet scattering segmentation network model that integrates a wavelet scattering network and a fully convolutional network. The learning wavelet scattering segmentation network model is based on the encoder-decoder structure of the fully convolutional network. For each layer of the encoder, a dual-tree complex wavelet scattering transform is added to effectively separate the high-frequency features and low-frequency features of the feature map. Then, the convolution operator is used to select the low-frequency main features and high-frequency boundary features of the feature map respectively, and finally the features are fused.

[0011] 2) Preprocessing the labeled image data set and inputting it into the learning wavelet scattering segmentation network model constructed in step 1) to train the network model, training the network model weights by the gradient descent algorithm, and saving the optimal weights in the validation set;

[0012] 3) Preprocessing the actual image data to be segmented is input into the trained network model in step 2), and the image segmentation can be achieved through the trained network model.

[0013] The preprocessing in step 2) and step 3) is specifically to scale each type of image data to a matrix of size [240, 240] by a bilinear interpolation algorithm, and perform Z-score normalization on non-zero regions.

[0014] The Z-score normalization is performed according to the following formula (1):

[0015]

[0016] Where x is the input modal data, μ and σ are the mean and variance of x respectively; the values ​​of μ and σ are from 1% to 99% quantiles to prevent extreme points from affecting the overall image.

[0017] Step 2) When training the network model, set the learning rate to 1e-4, select Adam as the optimizer, select the weighted combination of binary cross entropy and Dice-Loss as the loss function, and perform weighted training with a balance coefficient of 0.5, as shown in formula (2)

[0018] loss BraTS =loss Dice+0.5·loss BCE (2)

[0019] in:

[0020]

[0021] In formulas (3)-(4), y i ∈{0,1} is the binary encoding value of the segmentation standard, is the predicted value; after training, the model with the best Dice index in the validation set is retained.

[0022] In the learning wavelet scattering segmentation network model of step 1), the low-frequency features of the feature map obtained by the dual-tree complex wavelet scattering transform are processed by 2×2 average pooling low-pass filtering, that is, S=Avgpool(yl,2), and the amplitudes of the real and imaginary parts of the high-frequency features of the obtained feature map are calculated, that is, Among them, yl, yh, Re(·), Im(·) represent low-frequency information, high-frequency information, real part and imaginary part operators respectively. Then S and U are concatenated and output to realize the fusion of high and low frequency information.

[0023] The present invention also provides an image segmentation system based on a wavelet scattering learning network, comprising:

[0024] Data acquisition module: used to collect image data to be segmented from the image acquisition device and transmit it to the data preprocessing module;

[0025] Data preprocessing module: preprocess the image data transmitted by the data acquisition module. The preprocessing includes scaling the image to a matrix of size [240,240] through a bilinear interpolation algorithm and performing Z-score normalization on non-zero areas.

[0026] Image segmentation module: The image segmentation module has a built-in learning wavelet scattering segmentation network model that integrates a wavelet scattering network and a fully convolutional network. The learning wavelet scattering segmentation network model is based on the encoder-decoder structure of the fully convolutional network. A dual-tree complex wavelet scattering transform is added to each layer of the encoder to effectively separate the high-frequency features and low-frequency features of the feature map. Then, the convolution operator is used to select the low-frequency main features and high-frequency boundary features of the feature map respectively, and finally the features are fused. The image data preprocessed by the data preprocessing module is input into the image segmentation module, and the output of the image segmentation module realizes the segmentation of the image.

[0027] The image acquisition device is a vehicle-mounted camera device; the image data is a surrounding environment image acquired by the vehicle-mounted camera device.

[0028] The image acquisition device may also be a medical imaging device, and the image data is an MRI glioma medical image acquired by the medical imaging device, and MRI images of four modes, T1, T2, T1-ce, and flair, are acquired according to different relaxation times; the output of the image segmentation module is a prediction probability map;

[0029] It also includes a 2D visualization module, which is used to receive the prediction probability map output by the image segmentation module, classify the pixels of the prediction probability map point by point through threshold processing, and obtain a 2D visualization image;

[0030] 3D visualization module: receives 2D visualization images from the 2D visualization module and converts them into 3D voxels to achieve 3D visualization effects.

[0031] When the 3D visualization module converts the stacked 2D visualization images into 3D voxels, it converts the 3D voxels into nii or nii.gz files through the GetImageFromArray function in the SimpleITK package, and displays the 3D visualization effect through the ITK-SNAP tool.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] Compared with traditional methods, based on thresholding, level set, region growing and other algorithms, the present invention can extract semantic information of segmented images, and learn segmentation features in images through end-to-end data-driven learning, with higher accuracy and robustness. Compared with segmentation networks such as FCN, SegNet, and AttentionUNet, the present invention can extract frequency domain features, and will be more stable and sensitive to slight deformations of images, and will be more accurate in extracting small features. The segmentation results of the model show relatively clear and complete contours in all areas of the image, and the volume of mis-segmented patches is smaller. Therefore, this method has high robustness in segmentation accuracy. The experimental results are shown in Table 1.

[0034] Table 1: Comparison of the segmentation algorithm of the present invention with other algorithms on the BraTS glioma dataset

[0035]

[0036] BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 Sample MRI images of glioma patients using four modalities.

[0038] Figure 2 This is a system structure diagram of the MRI glioma segmentation model provided by the present invention.

[0039] Figure 3 This is a comparison chart of the preprocessing effect of the present invention on multimodal MRI signals, wherein the upper part is the original image and the lower part is the corresponding preprocessed image.

[0040] Figure 4 The present invention learns the wavelet scattering segmentation network structure diagram.

[0041] Figure 5 This is a comparison diagram of the 2D slice segmentation effects of the present invention and other segmentation methods.

[0042] Figure 6 This is a 3D glioma visualization effect diagram of the present invention and other segmentation methods. DETAILED DESCRIPTION

[0043] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] There are many types of input image boundary contour features, which makes it difficult to extract and optimize them in deep neural network learning. In order to solve this problem, the present invention provides the following problem-solving concept: a boundary feature extraction module is proposed, through which boundary features can be well extracted; specifically, it can be described as first using dual-tree complex wavelet scattering transform to separate high-frequency and low-frequency features of the feature map, and then using the convolution operator to extract low-frequency main features and high-frequency boundary features. Furthermore, the dual-tree complex wavelet scattering transform is built into the full convolutional network model, and a new learning wavelet scattering network (Learning scatting waveletnetworks, referred to as LSW-Net) semantic segmentation model is designed through end-to-end data-driven learning scattering transform features. The network uses the Unet network as the backbone network, and introduces the dual-tree complex wavelet scattering transform (DTCWT-Scat) for boundary feature extraction during downsampling to enhance the model's ability to extract image boundary contour information.

[0045] Based on the above inventive concept, the image segmentation method based on wavelet scattering learning network of the present invention comprises the following steps:

[0046] 1) Construct a learning wavelet scattering segmentation network model that integrates a wavelet scattering network and a fully convolutional network. The learning wavelet scattering segmentation network model is based on the encoder-decoder structure of the fully convolutional network. For each layer of the encoder, a dual-tree complex wavelet scattering transform is added to effectively separate the high-frequency features and low-frequency features of the feature map. Then, the convolution operator is used to select the low-frequency main features and high-frequency boundary features of the feature map respectively, and finally the features are fused.

[0047] 2) Preprocessing the labeled image data set and inputting it into the learning wavelet scattering segmentation network model constructed in step 1) to train the network model, training the network model weights by the gradient descent algorithm, and saving the optimal weights in the validation set;

[0048] 3) Preprocessing the actual image data to be segmented is input into the trained network model in step 2), and the image segmentation can be achieved through the trained network model.

[0049] The preprocessing refers to scaling each type of image data to a matrix of size [240, 240] by a bilinear interpolation algorithm, and performing Z-score standardization on non-zero regions.

[0050] The Z-score normalization is performed according to the following formula (1):

[0051]

[0052] Where x is the input modal data, μ and σ are the mean and variance of x respectively; the values ​​of μ and σ are from 1% to 99% quantiles to prevent extreme points from affecting the overall image.

[0053] Step 2) When training the network model, set the learning rate to 1e-4, select Adam as the optimizer, select the weighted combination of binary cross entropy and Dice-Loss as the loss function, and perform weighted training with a balance coefficient of 0.5, as shown in formula (2)

[0054] loss BraTS =loss Dice +0.5·loss BCE (2)

[0055] in:

[0056]

[0057] In formulas (3)-(4), y i ∈{0,1} is the binary encoding value of the segmentation standard, is the predicted value; after training, the model with the best Dice index in the validation set is retained.

[0058] Based on the above segmentation method, the present invention proposes an image segmentation system based on wavelet scattering learning network. Figure 2 ,include

[0059] Data acquisition module: It is used to collect image data to be segmented from the image acquisition device and transmit it to the data preprocessing module.

[0060] Data preprocessing module: Preprocess the image data transmitted by the data acquisition module. The preprocessing includes scaling the image to a matrix of size [240,240] through a bilinear interpolation algorithm and performing Z-score normalization on non-zero areas.

[0061] Image segmentation module. In order to improve the image segmentation accuracy of the full convolutional network and enhance the feature extraction capability of the image boundary, the present invention has a built-in learning wavelet scattering segmentation network model that integrates a wavelet scattering network with a full convolutional network in the image segmentation module. The learning wavelet scattering segmentation network model is based on the encoder-decoder structure of the full convolutional network. The process can be described as follows: a double-tree complex wavelet scattering transform is added to each layer of the encoder to effectively separate the high-frequency features from the low-frequency features of the feature map, and then a convolution operator is used to perform feature selection on the low-frequency main features and high-frequency boundary features of the feature map, and finally the features are fused; the image data preprocessed by the data preprocessing module is input into the image segmentation module, and the output of the image segmentation module realizes the segmentation of the image.

[0062] The image acquisition device is a vehicle-mounted camera device; the image data is the surrounding environment image acquired by the vehicle-mounted camera device, including traffic light images, road images, pedestrian images on the road, etc. Through the present invention, the traffic lights in the image, the road contour lines in the image, the marking lines on the road, and the pedestrians in the image can be effectively and accurately segmented and extracted, thereby providing accurate surrounding environment information for intelligent driving, which is conducive to improving the safety level of intelligent driving.

[0063] Specifically, the image acquisition device is a medical imaging device, and the image data is an MRI glioma medical image acquired by the medical imaging device. According to different relaxation times, MRI images of four modes, namely T1, T2, T1-ce, and flair, are acquired; the output of the image segmentation module is a predicted probability map.

[0064] Based on this, the present invention also includes a 2D visualization module and a 3D visualization module. The 2D visualization module is used to receive the predicted probability map output by the image segmentation module, and classify the pixels of the predicted probability map point by point through threshold processing, such as taking 0.5 as the threshold, to obtain a 2D visualization image. Taking glioma as an example, the non-tumor area is displayed in black, the edema area is displayed in green, the enhanced tumor area is displayed in yellow, and the necrotic area is displayed in red.

[0065] 3D visualization module: receives 2D visualization images from the 2D visualization module and converts them into 3D voxels to achieve 3D visualization. When the 3D visualization module converts the stacked 2D visualization images into 3D voxels, it converts the 3D voxels into nii or nii.gz files through the GetImageFromArray function in the SimpleITK package, and displays the 3D visualization effect through the ITK-SNAP tool.

[0066] Taking glioma MRI image segmentation as an example, the specific segmentation steps are as follows:

[0067] Step 1: Send the brain tumor data collected by the MRI imaging device to the local area, read the original data, and input it into the preprocessing module.

[0068] Step 2: For each modality’s data processing module, perform Z-score normalization on the original input data, see formula (1)

[0069]

[0070] Where x is the input modal data, μ and σ are the mean and variance of x respectively. The values ​​of μ and σ are taken from 1% to 99% quantiles to prevent the influence of extreme points on the overall image. The visualization of preprocessed data can be seen in Figure 3 , the four modal data are standardized and dimensionally spliced ​​and input into the next module.

[0071] Step 3: After preprocessing, the labeled training data set is input into the learning wavelet scattering segmentation network, the network model weights are trained by the gradient descent algorithm, and the optimal weights in the validation set are saved. Taking the multimodal glioma data as an example, the labeled data can be obtained from the MICCAI-BraTS official website. During specific training, the learning rate is set to 1e-4, the optimizer is Adam, the loss function is the weighted combination of binary cross entropy and Dice-Loss, and weighted training is performed with a balance coefficient of 0.5, see formula (2)

[0072] loss BraTS =loss Dice +0.5·loss BCE (2)

[0073] in:

[0074]

[0075] y in formula (3-4) i ∈{0,1} is the binary encoding value of the segmentation standard, is the predicted value. Train for 200 epochs. And retain the model with the best Dice index in the validation set. The collected unlabeled MRI multimodal dataset is preprocessed and input into the trained model to obtain the predicted probability map and output it to the 2D slice visualization module.

[0076] Step 4: Visualize the 2D slice results, process the predicted probability map through thresholding, and use the opencv library to store it as an image. The RGB value of the necrotic area is [255, 0, 0], the RGB value of the edema area is [255, 255, 0], and the RGB value of the enhanced tumor area is [0, 255, 0]. See the display effect. Figure 5 .

[0077] Step 5: Stack the 2D visualization images of all sample slices in the same case, construct a 3D vector, and then realize the 3D visualization of 2D glioma. The reconstruction tools mainly include opencv, SimpleITK, ITK-SNAP and other algorithm interfaces. The reconstruction effect is shown in Figure 6 .

[0078] In the present invention, the image segmentation algorithm is mainly based on the full convolutional network and the wavelet scattering algorithm. The algorithm framework includes a convolutional feature extraction module, that is, a batch normalization processing layer is connected after each convolution layer

[21] . The purpose is to accelerate the convergence speed of the entire network by normalizing the input of the current layer, and at the same time reduce the correlation between layers. Specifically, the input image is first convolved twice with a 3*3 size convolution kernel and normalized, and then the ReLU function is used as the activation function to achieve nonlinear transformation. This process can be simplified as follows: in They represent the input image, the number of convolution kernels, the size of the convolution kernel, and the module is executed twice. The DTCWT-Scat block is added to the encoder in the learning wavelet scattering segmentation network framework. Specifically, the feature map is first subjected to dual-tree complex wavelet transform, and then the low-frequency information is subjected to 2×2 average pooling low-pass filtering, that is, S=Avgpool(yl,2), and the amplitude (magnitude, abbreviated as mag) of the real and imaginary parts of the high-frequency information is also calculated, that is, Then S and U are concatenated and output, where yl, yh, Re(·), Im(·) represent low-frequency information, high-frequency information, real part, and imaginary part operators, respectively. Figure 4 The present invention learns the wavelet scattering segmentation network structure diagram.

[0079] The DTCWT-Scat feature extraction module has two significant advantages. First, it performs a dual-tree complex wavelet transform on the input image. This transform supports the back propagation of errors and can update parameters, so that the parameters of the previous convolutional layer can be learned, thereby extracting frequency domain features. Second, the wavelet function has local waveform characteristics and is stable to local deformations. Therefore, the LSW-Net model proposed in the present invention will be more stable and sensitive to tiny deformations of medical images such as tumors, and will be more accurate in extracting small features, so it is particularly suitable for medical image segmentation.

[0080] The present invention verifies the segmentation results of 72 glioma patients and compares them with three segmentation methods: FCN, SegNet, and AttentionUNet. The manually segmented glioma images are used as the standard (Ground Truth, GT). The 2D slice comparison effects of some sample gliomas are shown in Figure 2. Figure 5 As shown, red, yellow, and green represent necrotic areas, enhanced tumor areas, and edema areas, respectively. Compared with the true labels, the segmentation effect of the present invention is more accurate than that of the three models, and is more suitable for processing the segmentation problem of glioma datasets.

[0081] Comparison of 3D reconstruction effects of MRI gliomas of two samples Figure 6 It can be found that compared with the 3D reconstruction results of FCN, SegNet, and At-Unet, the outlier mis-segmented plaques in the MRI glioma segmentation 3D reconstruction of the present invention are smaller in volume, closer to the true label, and the segmentation results of brain tumors are more accurate.

[0082] The image segmentation technology proposed in the present invention is only a common application example of the above-mentioned glioma image segmentation. In addition, it can also be widely used in the fields of autonomous driving, satellite image systems, robots, etc. In the field of unmanned driving, high-quality semantic segmentation of road scenes can provide guarantee for the safe driving of autonomous vehicles. The segmentation of categories such as roads, forests, and farmland organisms in satellite remote sensing images can be applied to the monitoring and management of resources and the environment.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the applicant has described the present invention in detail with reference to the preferred embodiments, ordinary technicians in the field should understand that modifications or equivalent substitutions of the technical solution of the present invention without departing from the purpose and scope of the technical solution should be included in the scope of the claims of the present invention.

Claims

1. Image segmentation method based on wavelet scattering learning network, characterized by: Here are the steps: 1) Construct a learning wavelet scattering segmentation network model that integrates a wavelet scattering network and a fully convolutional network. The learning wavelet scattering segmentation network model is based on the encoder-decoder structure of the fully convolutional network. For each layer of the encoder, a dual-tree complex wavelet scattering transform is added to effectively separate the high-frequency features and low-frequency features of the feature map. Then, the convolution operator is used to select the low-frequency main features and high-frequency boundary features of the feature map respectively, and finally the features are fused. 2) Preprocessing the labeled image data set and inputting it into the learning wavelet scattering segmentation network model constructed in step 1) to train the network model, training the network model weights by the gradient descent algorithm, and saving the optimal weights in the validation set; 3) Preprocessing the actual image data to be segmented and inputting it into the network model trained in step 2), and the image segmentation can be achieved through the trained network model; In the learning wavelet scattering segmentation network model of step 1), the low-frequency features of the feature map obtained by the dual-tree complex wavelet scattering transform are processed by 2×2 average pooling low-pass filtering, that is, S=Avgpool(yl,2), and the amplitudes of the real and imaginary parts of the high-frequency features of the obtained feature map are calculated, that is, Among them, yl, yh, Re(·), Im(·) represent low-frequency information, high-frequency information, real part and imaginary part operators respectively. Then S and U are concatenated and output to realize the fusion of high and low frequency information.

2. The image segmentation method based on wavelet scattering learning network according to claim 1 is characterized in that: The preprocessing in step 2) and step 3) is specifically to scale each type of image data to a matrix of size [240, 240] by a bilinear interpolation algorithm, and perform Z-score normalization on non-zero regions.

3. The image segmentation method based on wavelet scattering learning network according to claim 2 is characterized in that: The Z-score normalization is performed according to the following formula (1): Where x is the input modal data, μ and σ are the mean and variance of x respectively; the values ​​of μ and σ are from 1% to 99% quantiles to prevent extreme points from affecting the overall image.

4. The image segmentation method based on wavelet scattering learning network according to claim 1 is characterized in that: Step 2) When training the network model, set the learning rate to 1e-4, select Adam as the optimizer, select the weighted combination of binary cross entropy and Dice-Loss as the loss function, and perform weighted training with a balance coefficient of 0.5, as shown in formula (2) loss BraTS =loss Dice +0.5·loss BCE (2) in: In formulas (3)-(4), y i ∈{0,1} is the binary encoding value of the segmentation standard, is the predicted value; after training, the model with the best Dice index in the validation set is retained.

5. Image segmentation system based on wavelet scattering learning network, characterized by: include Data acquisition module: used to collect image data to be segmented from the image acquisition device and transmit it to the data preprocessing module; Data preprocessing module: preprocess the image data transmitted by the data acquisition module. The preprocessing includes scaling the image to a matrix of size [240,240] through a bilinear interpolation algorithm and performing Z-score normalization on non-zero areas. Image segmentation module: The image segmentation module has a built-in learning wavelet scattering segmentation network model that integrates a wavelet scattering network and a full convolutional network. The learning wavelet scattering segmentation network model is based on the encoder-decoder structure of the full convolutional network. A dual-tree complex wavelet scattering transform is added to each encoder layer to effectively separate the high-frequency features and low-frequency features of the feature map. Then, the convolution operator is used to select the low-frequency main features and high-frequency boundary features of the feature map respectively, and finally the features are fused. The image data preprocessed by the data preprocessing module is input into the image segmentation module, and the output of the image segmentation module realizes the image segmentation. In the learning wavelet scattering segmentation network model, the low-frequency features of the feature map obtained by the dual-tree complex wavelet scattering transform are processed by 2×2 average pooling low-pass filtering, that is, S=Avgpool(yl,2), and the amplitude of the real and imaginary parts of the high-frequency features of the feature map is calculated, that is, Among them, yl, yh, Re(·), Im(·) represent low-frequency information, high-frequency information, real part and imaginary part operators respectively. Then S and U are concatenated and output to realize the fusion of high and low frequency information.

6. The image segmentation system based on wavelet scattering learning network according to claim 5, characterized in that: The image acquisition device is a vehicle-mounted camera device; the image data is a surrounding environment image acquired by the vehicle-mounted camera device.

7. The image segmentation system based on wavelet scattering learning network according to claim 5 is characterized in that: The image acquisition device is a medical imaging device, and the image data is an MRI glioma medical image acquired by the medical imaging device. According to different relaxation times, MRI images of four modes, T1, T2, T1-ce, and flair, are acquired; the output of the image segmentation module is a prediction probability map; It also includes a 2D visualization module, which is used to receive the prediction probability map output by the image segmentation module, classify the pixels of the prediction probability map point by point through threshold processing, and obtain a 2D visualization image; 3D visualization module: receives 2D visualization images from the 2D visualization module and converts them into 3D voxels by stacking to achieve 3D visualization effects.

8. The image segmentation system based on wavelet scattering learning network according to claim 7 is characterized in that: When the 3D visualization module converts the stacked 2D visualization images into 3D voxels, it converts the 3D voxels into nii or nii.gz files through the GetImageFromArray function in the SimpleITK package, and displays the 3D visualization effect through the ITK-SNAP tool.

Images