A multi-scale semi-supervised object segmentation method based on noise index guidance

Abstract

The application discloses a multi-scale semi-supervised target segmentation method based on noise index guidance, belongs to the field of computer vision, and aims at the problems that the robustness of an existing single traditional segmentation method is not strong, the feature capturing capacity is limited, a deep learning method U-Net pays insufficient attention to context information, and a full supervision method is complex in pixel-level labeling. The application relates to information fusion at two levels: since the feature capturing capacity of a single traditional segmentation method is limited, a plurality of traditional methods can be fused to obtain more accurate pseudo labels; since the deep learning network U-Net pays insufficient attention to context information, the down-sampling feature maps of different scales can be fused in the skip connection of each stage, so that the fused feature maps will carry multi-scale background information and save fine-grained target position information, and then the network can pay attention to more important regions.

Description

Technical Field

[0001] This invention relates to a multi-scale semi-supervised target segmentation method based on noise index guidance, belonging to the field of computer vision. Background Technology

[0002] Image object segmentation has always been one of the most important applications and research directions in the field of computer vision. Locating and accurately segmenting the Region of Interest (ROI) is a classic challenge in image object localization and segmentation. While significant breakthroughs have been made in the field of natural image object segmentation, segmentation results are often poor for certain domain-specific images (such as blurred pollen images or satellite images) that suffer from blurriness, low contrast, or artifacts. Furthermore, object segmentation in these domain-specific images often has more stringent, precise, and efficient segmentation requirements. Therefore, research on low-quality image segmentation based on specific domains has significant practical implications.

[0003] Currently, mainstream image object segmentation methods are broadly classified into two categories: traditional segmentation and deep learning. Deep learning methods are further categorized based on the difference in supervised information, including fully supervised, semi-supervised, weakly supervised, and unsupervised methods. Traditional segmentation methods primarily utilize shallow information such as grayscale, color, and texture of the image for segmentation. They require no training and are simple to operate, such as Otsu's method. [1] The proposed threshold-based image target segmentation method, Tremeau [2] While traditional segmentation methods, such as those proposed by Ronneberger et al., lack high-level semantic information and are not robust, improving the accuracy of these methods remains a significant challenge. In contrast, deep learning methods, such as those developed by Ronneberger et al., exhibit greater robustness. [3] The proposed U-Net network performs well on small datasets, but it requires pixel-level manual annotation of all original images, which is always a very time-consuming and labor-intensive task. Therefore, semi-supervised methods that can successfully segment large amounts of unlabeled data with a small amount of labeled data have become a hot research topic. Furthermore, because the segmentation targets in images vary in size, the U-Net network's insufficient attention to context leads to suboptimal segmentation results for targets of varying sizes. Therefore, considering contextual information and minimizing information loss to achieve more accurate segmentation is a direction for future improvement.

[0004] To address the issues of poor robustness of traditional segmentation methods, insufficient attention to contextual information in deep learning methods like U-Net, and complex pixel-level annotation in fully supervised methods, this invention focuses on research into multi-scale target segmentation guided by noise exponent under semi-supervised conditions. This invention relates to information fusion at two levels: ① Since single traditional segmentation methods have limited feature capture capabilities, multiple traditional methods can be fused to obtain more accurate pseudo-labels; ② Because the U-Net deep learning network lacks attention to contextual information, downsampled feature maps of different scales can be fused at skip connections in each stage. This fused feature map will carry multi-scale background information and preserve fine-grained target location information, thereby enabling the network to focus on more important regions. Summary of the Invention

[0005] To address the shortcomings of existing single traditional segmentation methods, such as weak robustness and limited feature capture capabilities, as well as the insufficient attention to contextual information in deep learning methods like U-Net and the complex pixel-level annotation in fully supervised methods, this invention designs a multi-scale semi-supervised object segmentation method guided by noise index. Specifically, we determine the noise index (NI) of each traditional segmentation method using labeled data, then perform coarse and fine segmentation on all data separately, and finally fuse the segmentation results of different traditional methods on the same original image based on the noise index NI. The fused result is then used as pseudo-labels to train an improved network, U-Net, which adds a downsampling path to preserve more contextual information.

[0006] The noise index-guided multi-scale semi-supervised target segmentation method described in this invention consists of five stages: The first stage is image preprocessing, which mainly denoises and unifies the resolution of images in the dataset. The second stage is obtaining the noise index (NI), which mainly involves segmenting the original image with pixel-level annotations using various traditional methods, and obtaining the noise index (NI) for each traditional method by comparing the segmentation results with the annotations. The third stage is coarse segmentation and refinement using traditional methods, that is, coarse segmentation of all data using the traditional methods from the second stage, and then refinement using the K-Nearest Neighbor (KNN) algorithm. The fourth stage is the fusion of traditional methods, which combines the noise index (NI) obtained in the second stage and performs pixel-by-pixel fusion to obtain the final segmentation result. The fifth stage is target region prediction, which uses the segmentation results from the previous stage as pseudo-labels to train the segmentation network. This segmentation network adds a downsampling path to the U-Net, where the first 3×3 convolution is replaced with a 5×5 convolution, and the feature maps of the two downsampling paths are concatenated during skip connections, so that the network retains more contextual information and segments the target more accurately.

[0007] The specific solution of the present invention is attached. Figure 2 As shown.

[0008] Step 1: Image Preprocessing

[0009] The purpose of image preprocessing is to: ① unify the size of all original images; ② denoise all images using methods such as histogram equalization and filtering. This invention targets image data that is high-noise, of poor quality, and with varying target sizes, such as blurred pollen images and satellite image datasets. Traditional segmentation methods focus on shallow information such as grayscale, color, and texture. Without denoising, this severely impacts segmentation results; therefore, denoising of all input images is essential. In this invention, we enhance image contrast through histogram equalization and remove noise by smoothing the image using anisotropic diffusion filtering. Then, to facilitate subsequent processing, the denoised images are resized.

[0010] Step 2: Obtain the Noise Figure (NI)

[0011] This step takes the preprocessed, labeled image as input, segments the labeled image using five traditional segmentation methods to obtain the initial noise index, and then normalizes it to obtain the final noise index.

[0012] Step 2.1 Obtain the initial noise index NI′: The input image is segmented using five mainstream segmentation methods: threshold-based OTSU segmentation, variable model-based active contour segmentation, region-based watershed segmentation, region-based seed growth segmentation, and clustering-based K-means segmentation. The results are then compared with the true labels, focusing on the non-overlapping regions between the segmentation results and the labels, to obtain the initial noise index NI′ of these five traditional methods.

[0013] Step 2.2 Obtain the final noise index NI: Normalize the initial noise index NI′ obtained in 2.1 to obtain the final noise index NI of these five traditional methods.

[0014] Step 3: Traditional methods for coarse segmentation and refinement

[0015] This step involves performing coarse segmentation on all preprocessed images using the five traditional segmentation methods mentioned above, and then refining the coarse segmentation results.

[0016] Step 3.1: Traditional coarse segmentation: All images are segmented using five mainstream segmentation methods: threshold-based OTSU segmentation, variable model-based active contour segmentation, region-based watershed segmentation, region-based seed growth segmentation, and clustering-based K-means segmentation, to obtain coarse segmentation results.

[0017] Step 3.2: Refine the coarse segmentation result: Use a KNN-based method to refine the coarse segmentation result pixel by pixel. The specific rules are: ① The label of the center pixel is the same as the labels of all its neighboring pixels: the label of the center pixel remains unchanged; ② The label of the center pixel is not completely the same as the labels of its neighboring pixels: assign the labels of the majority of the neighborhood to the center pixel.

[0018] Step 4: Integration of Traditional Methods

[0019] This step obtains the segmentation results refined by the five traditional methods in step 3.2, and combines them with the weights of the five methods calculated from the final noise index NI in step 2. The pixel values ​​are calculated point by point to finally obtain the fused segmentation results. Specifically, this step is divided into three stages.

[0020] Step 4.1: Obtaining the fusion weights: Convert the noise index NI of the five traditional methods into fusion weights, and then normalize the fusion weights.

[0021] Step 4.2: Segmentation result fusion: Using pixels as the unit, multiply the segmentation result of each pixel under the five methods (1 for segmentation target, 0 for background) by its corresponding fusion weight to obtain the label probability of that point.

[0022] Step 4.3: Probability Label Transformation: If the label probability of a point is greater than 0.5, it indicates that most models tend to view this point as the segmentation target, so the label of this point is set to 1; if the label probability of a point is less than or equal to 0.5, it indicates that most models tend to view this point as the background, so the label of this point is set to 0. Using this method, the final fused segmentation result can be obtained.

[0023] Step 5: Target Area Prediction

[0024] This step first uses the final segmentation result from step 4.3 as a pseudo-label, then trains a localization and segmentation network for image ROI based on the pseudo-label. This network is an improvement on the traditional U-Net. Finally, the improved U-Net network is used to predict the ROI.

[0025] The network used in this step is an improved U-Net network, which adds a downsampling path to the original U-Net network. This added downsampling path transforms the first 3×3 convolution of each layer in the original downsampling path into a 5×5 convolution. Then, the generated feature map is concatenated with the feature map generated by the downsampling in the skip connections and central layers. With the same number of convolutions, the 5×5 convolution has a larger receptive field and loses less information than the 3×3 convolution. Therefore, the fused feature map can carry multi-scale background information and preserve fine-grained target location information, thereby guiding the network to preserve more contextual information and focus on more important regions.

[0026] Compared with existing technologies, the advantages of this invention are as follows:

[0027] I. This invention employs a multi-scale semi-supervised target segmentation method guided by noise index, which effectively avoids pixel-level annotation of all image data in full supervision, thus reducing the workload of annotation data.

[0028] Second, this invention adopts a multi-scale semi-supervised target segmentation method based on noise index guidance. The noise index NI is introduced into the segmentation stage of the traditional method. The traditional segmentation results are fused based on the noise index, which effectively solves the problems of limited feature capture capability and weak robustness of single methods.

[0029] Third, this invention employs a multi-scale semi-supervised target segmentation method guided by noise index. It improves the U-Net network during the deep learning stage by adding a downsampling path to the U-Net network, thereby obtaining feature maps at different scales.

[0030] By fusing these feature maps, the network is guided to retain more contextual information.

[0031] Experiments have shown that the DeepLabV3+ network based on ResNet, with its built-in ASPP (Atrous Spatial Pyramid Pooling) module and multi-scale feature map fusion, can achieve 84.4% Dice (Dice Similarity Coffiliation) in fuzzy pollen data segmentation, a 4.94% improvement over U-Net without multi-scale feature fusion. Therefore, incorporating multi-scale features into the U-Net network allows for a larger receptive field, significantly contributing to the preservation of more contextual information in the target image. Attached Figure Description

[0032] Figure 1 This is a diagram of the improved U-Net structure of the present invention.

[0033] Figure 2 This is an overall flowchart of the method proposed in this invention. Detailed Implementation

[0034] The following detailed description of embodiments of the present invention is provided in conjunction with the accompanying drawings:

[0035] The noise index-guided multi-scale semi-supervised target segmentation method described in this invention consists of five stages: The first stage is image preprocessing, primarily involving denoising the dataset images and unifying their resolution. The second stage involves obtaining the noise index (NI), which is achieved by comparing the segmentation results of traditional methods with the ground truth labels to obtain the NI for each traditional method. The third stage involves coarse segmentation and refinement using traditional methods, including coarse segmentation and refinement using the KNN algorithm. The fourth stage is traditional method fusion, combining the refinement results from the third stage and the NI from the second stage to obtain the final image segmentation result. The fifth stage is target region prediction, using the segmentation results from the fourth stage as pseudo-labels to train an improved segmentation network. This network adds multi-scale features to the original U-Net, and finally, the target region is predicted using this network.

[0036] Specifically, the method includes the following steps:

[0037] Step 1: Image Preprocessing

[0038] The purpose of image preprocessing is to standardize the size of all images in the dataset and reduce noise. The dataset referenced in this invention mainly consists of images with high noise, poor quality, and varying target sizes. We use a fuzzy pollen image dataset as an example, containing 2000 images covering 14 pollen species, including those from the Asteraceae, Moraceae, Chenopodiaceae, Pinaceae, and Poaceae families. The pollen shapes and sizes vary significantly among different species. We employ histogram equalization and anisotropic diffusion filtering to preprocess the images. We set the thermal conductivity R of the anisotropic diffusion filter to 15, the number of iterations N to 10, and the smoothing coefficient Lambla to 0.15. Finally, the final preprocessed image size is standardized to 512×512.

[0039] Step 2: Obtain the Noise Figure (NI)

[0040] This step involves segmenting the labeled image using five different traditional methods to obtain the noise index NI for each method. Specifically, obtaining NI consists of two parts: first, by comparing the segmentation results with the original annotations, focusing on non-overlapping regions, an initial noise index NI′ is obtained; second, the initial noise indices of the five traditional methods are normalized to obtain the final noise index NI.

[0041] Step 2.1 Obtain the initial noise index NI′: For a given 512*512 size, denoised original image with real labels, we use five traditional methods to segment it: threshold-based OTSU segmentation, variable model-based active contour segmentation, region-based watershed segmentation, region-based seed growth segmentation, and cluster-based K-means segmentation. We then compare the segmentation with the real labels and calculate the proportion of the intersection-union difference in the original labels to obtain the initial noise index NI′ corresponding to the segmentation method. The specific formulas are shown in (1) and (2).

[0042] D′ k =|{y=1}∪{y′=1}| k -|{y=1}∩{y′=1}| k (1)

[0043] NI′ k =D′ k / {y=1} k (2)

[0044] Where y is the ground truth label of the image (1 for target, 0 for background), y′ is the image target segmentation result (1 for target, 0 for background), k is the k-th traditional method, and D′ k For intersection, union, and difference.

[0045] Step 2.2 Obtain the final noise index NI: Normalize the initial noise index obtained in step 2.1. In addition, in order to avoid zero values, we do not simply add them together, but use an exponential function to normalize them, and finally obtain the final noise index NI of each method. The specific formula is shown in (3).

[0046]

[0047] Where k represents the kth traditional method.

[0048] Step 3: Traditional methods for coarse segmentation and refinement

[0049] This step involves performing coarse segmentation on all preprocessed images using the five traditional segmentation methods mentioned above, and then refining the coarse segmentation results.

[0050] Step 3.1: Traditional coarse segmentation: All images are segmented using five mainstream segmentation methods: threshold-based OTSU segmentation, variable model-based active contour segmentation, region-based watershed segmentation, region-based seed growth segmentation, and clustering-based K-means segmentation. The specific formulas are shown in (4), (5), (6), (7) and (8).

[0051] Mask1(x,y)′=W1(u(x,y)) (4)

[0052] Mask2(x,y)′=W2(u(x,y)) (5)

[0053] Mask3(x,y)′=W3(u(x,y)) (6)

[0054] Mask4(x,y)′=W4(u(x,y)) (7)

[0055] Mask5(x,y)′=W5(u(x,y)) (8)

[0056] Where u(x,y) represents the pixel value at coordinates (x,y), W1 represents the threshold-based OTSU segmentation method, and Mask1(x,y)′ represents the coarse segmentation result for this pixel (1 for segmentation target, 0 for background). W2 represents the active contour segmentation method based on a variable model, and Mask2(x,y)′ represents the coarse segmentation result for this pixel (1 for segmentation target, 0 for background). W3 represents the region-based watershed segmentation method, and Mask3(x,y)′ represents the coarse segmentation result for this pixel (1 for segmentation target, 0 for background). W4 represents the region-based seed growth segmentation method, and Mask4(x,y)′ represents the coarse segmentation result for this pixel (1 for segmentation target, 0 for background). W5 represents the clustering-based K-means method, and Mask5(x,y)′ represents the coarse segmentation result for this pixel (1 for segmentation target, 0 for background).

[0057] Step 3.2: Refine the coarse segmentation result: The coarse segmentation result is refined pixel by pixel using a KNN-based method. The specific rules are as follows: ① The label of the center pixel is the same as the labels of all its neighboring pixels: the label of the center pixel remains unchanged; ② The label of the center pixel is not completely the same as the labels of its neighboring pixels: the labels of the majority of the neighborhood pixels are assigned to the center pixel. The specific formula is shown in (9).

[0058]

[0059] Among them, Mask k (x,y) represents the segmentation result after refining the point (x,y) (1 represents the segmentation target, 0 represents the background), Mask k (x,y)′ represents the coarse segmentation result (1 for segmentation target, 0 for background), and A represents the label of most pixels in the set of neighboring pixels of point (x,y) (1 for segmentation target, 0 for background).

[0060] Step 4: Integration of Traditional Methods

[0061] This step obtains the segmentation results refined by the five traditional methods in step 3.2, and combines them with the weights of the five methods calculated by the noise index NI in step 2. Pixel values ​​are calculated point by point to finally obtain the fused segmentation result. Specifically, it is divided into three stages.

[0062] Step 4.1: Obtaining the fusion weights: Convert the noise index NI of the five traditional methods into fusion weights w, and then normalize the weights. The specific formula is shown in (10).

[0063]

[0064] Where w k For the fusion weight of the k-th method, NI k Let be the noise index of the k-th method.

[0065] Step 4.2 Segmentation result fusion: Using pixels as the unit, multiply the five segmentation results of the pixel (1 for segmentation target, 0 for background) and the fusion weight w corresponding to each result to obtain the label probability M(x,y) of the point. The specific formula is shown in (11).

[0066]

[0067] Where w k Mask is the fusion weight for the k-th method. k (x,y) represents the segmentation result refined by the k-th method.

[0068] Step 4.3 Probability Label Conversion: Pixel-by-pixel judgment. If the label probability of a pixel is greater than 0.5, it indicates that most models tend to consider this point as the segmentation target, and the label of this point is set to 1; if the label probability of a pixel is less than or equal to 0.5, it indicates that most models tend to consider this point as the background, and the label of this point is set to 0. Through this method, the final label of each pixel can be obtained, that is, the final segmentation result after fusion, as shown in the specific formula (12).

[0069]

[0070] Step 5: ROI Region Prediction

[0071] This step first uses the final segmentation result from step 4.3 as a pseudo-label, then trains a localization and segmentation network for image ROI based on the pseudo-label. This network is an improvement on the traditional U-Net, and finally uses the improved network to predict the ROI.

[0072] The network used in this step is an improved U-Net network, as shown in the attached diagram. Figure 1As shown, it adds a downsampling path to the original U-Net network. The added downsampling path changes the original first 3×3 convolution in each layer into a 5×5 convolution. Then, the generated feature map is concatenated with the feature map generated by the downsampling of the original U-Net in the skip connections and the central layer, so as to guide the network to pay more attention to the context information.

[0073] Specifically, the U-Net network is an encoder-decoder structure, corresponding to downsampling and upsampling respectively. The network has a total of five layers, divided into a downsampling part, a skip connection part, and an upsampling part. In the downsampling part, each layer first performs two convolution operations, and then performs a 2×2 max pooling operation before entering the next layer. The two convolutions in the original downsampling path are both 3×3 convolutions, as shown in formula (13). The added downsampling path changes the first 3×3 convolution of each layer in the original downsampling path into a 5×5 convolution, as shown in formula (14). The fifth layer is the central layer. After two convolutions, pooling is no longer performed. Instead, the feature maps of different scales generated are unified in size and then stitched together to obtain a fused feature map, which is then sent to the upsampling part, as shown in formula (15). The skip connection part is to cut and stitch the feature map of the corresponding downsampling part of each layer into the corresponding upsampling part. In this invention, since a new downsampling path has been added, it is necessary to first unify the size of the feature maps of the same number of layers in the two paths and stitch them together before sending them to the upsampling part, as shown in formula (16). In the upsampling part, each layer first performs two 3×3 convolution operations, and then performs bilinear interpolation to upsample the feature map, as shown in formula (17). Finally, after the output of the first layer decoder, a 1×1 convolution is used to change the feature map channel to 1, that is, the pixel has only two values ​​(1 for segmentation target, 0 for background), thus obtaining the final segmentation result, as shown in formula (18). Except for the last 1×1 convolution, after each convolution, a batch normalization operation is used. This normalizes the input of the subsequent activation function, making the batch follow a standard normal distribution (mean of 1, standard deviation of 0), making the values ​​more stable. After batch normalization, a Rectified Linear Unit (ReLU) is used as the activation function, making the model converge faster. C 0,1 and C 0,2 This is the original input image.

[0074] C i+1,1 =p2(conv3(conv3(C i,1 (13)

[0075] C i+1,2 =p2(conv3(conv5(C i,2 (14)

[0076]

[0077]

[0078] Di = conv3(conv3(UP(D) i+1 (17)

[0079] O = conv1(D1) (18)

[0080] Among them, C i+1,1 C is the output of the encoder at layer i+1 of the original path. i+1,2 For the output of the encoder at layer i+1 of the newly added path, C i,skip C5 is the output of the skip connection of layer i, C5 is the output of the encoder of layer 5, i.e., the central layer, and D is the output of the skip connection of layer i. i is the output of the i-th layer decoder, D5 is the output of the 5th layer (the central layer decoder), D1 is the output of the 1st layer decoder, O is the final segmentation result; p2 is the 2×2 max pooling operation, conv1 is the 1×1 convolution operation, conv3 is the 3×3 convolution operation, conv5 is the 5×5 convolution operation, and UP is the upsampling operation, i.e., bilinear interpolation. This is a lateral connection operation on the feature map, i.e., element-wise addition.

[0081] This invention primarily targets image data with high ambiguity and varying target sizes, such as blurred pollen data and satellite imagery data. In the traditional segmentation part, it enhances the robustness and feature capture ability of traditional methods by fusing multiple different traditional segmentation methods, thereby improving the accuracy of pseudo-labels. In the deep learning part, it guides the segmentation network to retain more information by adding downsampling paths of different scales, thereby improving the target region segmentation prediction performance. The steps described in the specific embodiments of the invention can be modified, but the system architecture does not depart from the basic spirit of the invention. Therefore, the current embodiments should be regarded as exemplary rather than limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description.

[0082] References:

[0083] [1]Otsu Nobuyuki.A Threshold Selection Method from Gray-LevelHistograms[J].IEEE Transactions on Systems,Man,and Cybernetics,1979,9(1).

[0084] [2]Alain Tremeau,NathalieBorel.A region growing and merging algorithmto color segmentation[J].Pattern Recognition,1997,30(7).

[0085] [3]Olaf Ronneberger,PhilippFischer,ThomasBrox.U-Net:ConvolutionalNetworks for Biomedical Image Segmentation.[J].CoRR,2015,abs / 1505.04597。

Claims

1. A multi-scale semi-supervised object segmentation method based on noise index guidance, characterized in that Comprising the following steps: Step 1: image preprocessing The purpose of image preprocessing is: ① unify the size of all original maps; ② denoise all images; Step 2: noise index NI acquisition This step takes the preprocessed and labeled image as input, segments the labeled image to obtain the initial noise index, and then normalizes to obtain the final noise index, as follows: Step 2.1: Obtain the initial noise index NI': For a given 512*512 size, denoised and labeled original image, use five traditional methods of threshold-based OTSU segmentation, variable model-based active contour segmentation, region-based watershed segmentation, region-based seed growth segmentation, and clustering-based K-means segmentation to segment it, and compare it with the true label to calculate the proportion of the intersection and union in the original label, and obtain the initial noise index NI' corresponding to the segmentation method, as shown in (1) and (2); D' k = |{y = 1} U {y' = 1}| k - |{y = 1} n {y' = 1}| k (1) NI' k = D' k / {y = 1} k (2) where y is the image ground truth label, y' is the image target segmentation result, k is the kth traditional method, D' is the Dice coefficient, and D'k is the Dice coefficient of the kth traditional method. k is the intersection over union; where 1 is the target and 0 is the background. Step 2.2: Obtain the final noise index NI: Normalize the initial noise index NI' obtained in 2.1 to obtain the final noise index NI of the five traditional methods; Step 3: Traditional method coarse segmentation and refinement, as follows: Step 3.1: Traditional method coarse segmentation: Segment all images using five major segmentation methods of threshold-based OTSU segmentation, variable model-based active contour segmentation, region-based watershed segmentation, region-based seed growth segmentation, and clustering-based K-means segmentation to obtain coarse segmentation results; Step 3.2: Coarse segmentation result refinement: Refine the coarse segmentation result pixel by pixel using the KNN-based method, with the following specific rules: ① The label of the center pixel is the same as that of all its adjacent pixels: the label of the center pixel remains unchanged; ② The label of the center pixel is not completely the same as that of its adjacent pixels: assign the majority label in the neighborhood to the center pixel; Step 4: Traditional method fusion This step obtains the segmentation results of the five traditional methods after refinement in step 3.2, and combines them with the weights of the five methods calculated from the final noise index NI in step 2 to calculate the pixel value point by point, and finally obtains the fused segmentation result. Specifically, this step is divided into three stages, as follows: Step 4.1: Fusion weight acquisition: Convert the noise index NI of the five traditional methods into fusion weights, and then normalize the fusion weights; Step 4.2: Segmentation result fusion: Take each pixel point as a unit, multiply the segmentation results of each pixel point under the five methods by its corresponding fusion weight to obtain the label probability of that point, 1 for segmentation target and 0 for background; Step 4.3: Probability label conversion: If the label probability of the point is greater than 0.5, it means that most models tend to regard the point as a segmentation target, so the label of the point is set to 1; if the label probability of the point is less than or equal to 0.5, it means that most models tend to regard the point as background, so the label of the point is set to 0; Through this method, the final fused segmentation result is obtained; Step 5: Target region prediction This step first takes the final segmentation result in step 4.3 as a pseudo label, then trains an image ROI positioning segmentation network based on the pseudo label, the network is improved on the traditional U-Net, and finally uses the improved U-Net network to predict the ROI; The network used in this step is an improved U-Net network, which adds a down-sampling path to the original U-Net network. The first 3*3 convolution of each layer of the original down-sampling path is changed to 5*5 convolution in the added down-sampling path. Then the generated feature map is spliced with the feature map generated by the original U-Net down-sampling in the jump connection and the center layer.

2. A multi-scale semi-supervised object segmentation method based on noise index guidance, characterized in that Comprising the following steps: Step 1: image preprocessing The purpose of image preprocessing is to unify the size of all images in the data set and to carry out noise reduction processing; histogram equalization and anisotropic diffusion filtering are adopted to preprocess the image, the heat transfer coefficient R of anisotropic diffusion filtering is set to 15, the iteration number N is set to 10, and the smoothing coefficient lambla is set to 0.15; then, the size of the final preprocessed image is unified to 512*512; Step 2: noise index NI acquisition Step 2.1: obtaining initial noise index NI': for a given 512*512 size, noise-reduced, and real-labeled original image, five traditional methods of threshold-based OTSU segmentation, variable model-based active contour segmentation, region-based watershed segmentation, region-based seed growth segmentation, and clustering-based K-means segmentation are used for segmentation, and compared with the real label, the proportion of the intersection and union difference in the original label is calculated to obtain the initial noise index NI' corresponding to the segmentation method, the specific formula is shown in (1) and (2); D' l = |{y = 1} U {y' = 1}| k - |{y = 1} n {y' = 1}| k (1) NI' k = D' k / {y = 1} k (2) where y is the image ground truth label, y' is the image target segmentation result, k is the kth traditional method, D' is the Dice coefficient, and D'k is the Dice coefficient of the kth traditional method. k is the intersection over union; where 1 is the target and 0 is the background. Step 2.2: obtaining final noise index NI: normalizing the initial noise index obtained in step 2.1, using exponential function normalization, finally obtaining the final noise index NI of each method, the specific formula is shown in (3); Where k is the kth traditional method; Step 3: traditional method coarse segmentation and refinement Step 3.1: traditional method coarse segmentation: five mainstream segmentation methods of threshold-based OTSU segmentation, variable model-based active contour segmentation, region-based watershed segmentation, region-based seed growth segmentation, and clustering-based K-means segmentation are used to segment all images to obtain coarse segmentation results, the specific formula is shown in (4), (5), (6), (7) and (8); Mask1(x,y)' = W1(u(x,y)) (4) Mask2(x,y)' = W2(u(x,y)) (5) Mask3(x,y)' = W3(u(x,y)) (6) Mask4(x,y)' = W4(u(x,y)) (7) Mask5(x,y)' = W5(u(x,y)) (8) Where u(x, y) represents the pixel point value of coordinates (x, y), W1 represents the OTSU threshold-based segmentation method, Mask1(x, y)' represents the coarse segmentation result of the pixel point; W2 represents the active contour segmentation method based on the variable model, Mask2(x, y)' represents the coarse segmentation result of the pixel point; W3 represents the region-based watershed segmentation method, Mask3(x, y)' represents the coarse segmentation result of the pixel point; W4 represents the region-based seed growth segmentation method, Mask4(x, y)' represents the coarse segmentation result of the pixel point; W5 represents the K-means method based on clustering, Mask5(x, y)' represents the coarse segmentation result of the pixel point; Step 3.2: Refine the coarse segmentation result: use the KNN-based method to refine the coarse segmentation result pixel by pixel, and the specific rules are as follows: ① The label of the center pixel is the same as that of all its adjacent pixels: the label of the center pixel remains unchanged; ② The label of the center pixel is not completely the same as that of its adjacent pixels: assign the majority label in the neighborhood to the center pixel; the specific formula is shown in (9); wherein Mask k (x, y) is the refined segmentation result of point (x, y), Mask k (x, y)′ represents the coarse segmentation result, and A represents the label of the majority of pixels in the set of adjacent pixels of point (x, y). Step 4: Fusion of traditional methods Step 4.1: Obtain the fusion weight: convert the noise index NI of the five traditional methods into the fusion weight w, and then normalize the weight; the specific formula is shown in (10); where w k is the fusion weight of the kth method, N k is the noise index of the kth method; Step 4.2: Fusion of segmentation results: take the pixel point as the unit, multiply the five segmentation results of the pixel point by the fusion weight w corresponding to each result, and obtain the label probability M(x, y) of the point; the specific formula is shown in (11); where w k is the fusion weight of the kth method, Mask k (x, y) is the refined segmentation result of the kth method Step 4.3: Convert the probability label: judge pixel by pixel, if the label probability of the pixel point is greater than 0.5, it means that most models tend to regard the point as the segmentation target, then the label of the point is set to 1; if the label probability of the pixel point is less than or equal to 0.5, it means that most models tend to regard the point as the background, then the label of the point is set to 0; in this way, the final label of each pixel point, i.e. the final segmentation result after fusion, can be obtained; the specific formula is shown in (12); Step 5: ROI region prediction This step first takes the final segmentation result in step 4.3 as the pseudo label, and then trains the image ROI positioning segmentation network based on the pseudo label; the network is improved on the traditional U-Net, and finally the improved network is used to predict the ROI; The network used in this step is an improved U-Net network. Specifically, the U-Net network is an encoder-decoder structure, corresponding to down-sampling and up-sampling respectively. The network has a total of five layers, which are divided into a down-sampling part, a skip connection part and an up-sampling part. The down-sampling part first performs two convolution operations at each layer, and then performs a 2x2 max-pooling operation and enters the next layer. The original down-sampling path has two 3x3 convolutions, as shown in equation (13). The added down-sampling path changes the first 3x3 convolution of each layer of the original down-sampling path to a 5x5 convolution, as shown in equation (14). The fifth layer is the center layer, which does not pool after two convolutions, but instead uniformly sizes the generated feature maps of different scales to obtain a fused feature map, which is then sent to the up-sampling part, as shown in equation (15). The skip connection part is to crop and splice the feature maps of the corresponding down-sampling part at each layer into the corresponding up-sampling part. Since a new down-sampling path is added, the feature maps of the same layer from the two paths need to be uniformly sized and spliced before being sent to the up-sampling part, as shown in equation (16). The up-sampling part first performs two 3x3 convolution operations at each layer, and then performs bilinear interpolation on the feature maps, as shown in equation (17). Finally, after the output of the first layer decoder, a 1x1 convolution is used to change the channel of the feature map to 1, i.e. the pixel has only two values, thereby obtaining the final segmentation result, as shown in equation (18). Except for the last 1x1 convolution, batch normalization is used after each convolution, which normalizes the input of the subsequent activation function to make the batch normally distributed. After batch normalization, the linear rectifier unit ReLU is used as the activation function. C 0,1 and C 0,2 is the original input image. C i+1,1 = p2(conv3(conv3(C i,1 ))) (13) C i+1,2 = p2(conv3(conv5(C i,2 ))) (14) D i = conv3(conv3(UP(D i+1 ))) (17) O = conv1(D1) (18) wherein C i+1,1 is the output of the original path i+1 layer encoder, C i+1,2 is the output of the new path i+1 layer encoder, C i,skip is the output of the i-th layer skip connection, C5 is the output of the 5-th layer, i.e. the center layer encoder, D i is the output of the i-th layer decoder, D5 is the output of the 5-th layer, i.e. the center layer decoder, D1 is the output of the 1-st layer decoder, O is the final segmentation result; p2 is a 2x2 max-pooling operation, conv1 is a 1x1 convolution operation, conv3 is a 3x3 convolution operation, conv5 is a 5x5 convolution operation, UP is an up-sampling operation, i.e. bilinear interpolation; is a feature map horizontal concatenation operation, i.e. element-wise addition.

Images