Cloud snow layer image segmentation method and system based on edge optimization, and storage medium

Abstract

The invention relates to the technical field of remote sensing image processing, in particular to a cloud snow layer image segmentation method and system based on edge optimization and a storage medium, and the method comprises the steps: segmenting a foreground region containing a cloud layer and a snow layer on a remote sensing image through a ResNet network, and a background region; extracting the edge information of the cloud layer and the snow layer in the foreground region by using an edge detection operator to obtain the edge information of the foreground region; and fusing the edge information into a graph cut algorithm framework, and segmenting a cloud layer and a snow layer in the foreground region. According to the method, firstly, the ResNet stage is utilized to ensure that the cloud snow foreground is completely detected, then the edge optimization graph cutting stage is utilized to accurately guide the boundary by utilizing the mixed edge weight, so that the segmentation contour is highly matched with the real ground feature edge in the image, respective advantages of deep learning and the graph cutting model are fully utilized, and the image segmentation efficiency is improved. And high-precision and high-robustness automatic segmentation of the cloud layer and the snow layer in the remote sensing image is realized.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing technology, specifically to a cloud and snow layer image segmentation method, system, and storage medium based on edge optimization. Background Technology

[0002] Accurate segmentation of clouds and snow layers in remote sensing images is a core preprocessing step in fields such as meteorological monitoring, climate research, glacier surveys, and land cover analysis. However, due to the high reflectivity of clouds and snow in both visible and near-infrared bands ("different objects in the same spectrum"), and the fact that their edges often exhibit blurred, gradual, and nonlinear characteristics, traditional single-technical approaches face severe challenges. For example, methods based on deep learning (such as semantic segmentation networks like ResNet) can learn from a large number of samples to understand the high-level semantics and contextual features of clouds and snow, achieving robust separation between the foreground (clouds and snow) and the background (land surface). However, their segmentation boundaries are often relatively coarse, and their accuracy in locating complex texture changes and blurred boundaries within clouds and snow is limited, easily leading to smoothed boundaries or loss of details. Methods based on traditional image processing (such as thresholding, region growing, and classical graph cut) can utilize spectral features (such as brightness and NDSI) for segmentation, but they are sensitive to initial values ​​and are prone to over-segmentation or under-segmentation in cloud and snow texture areas (such as wavy clouds and shadowed snow), and they are difficult to effectively utilize the edge structure information of the image itself.

[0003] Therefore, current technologies are insufficient to achieve fine and accurate segmentation of cloud and snow layers in remote sensing images, especially the mixed cloud and snow layers, resulting in insufficient segmentation accuracy of cloud and snow in remote sensing images. Summary of the Invention

[0004] The purpose of this invention is to provide a cloud and snow layer image segmentation method, system, and storage medium based on edge optimization, so as to solve the technical problem that it is difficult to achieve fine and accurate segmentation of cloud and snow layers, especially cloud and snow mixed layers, in remote sensing images in the prior art.

[0005] To solve the above-mentioned technical problems, the present invention specifically provides the following technical solution:

[0006] A cloud and snow layer image segmentation method based on edge optimization includes the following steps:

[0007] The ResNet network was used to segment the foreground region containing clouds and snow, as well as the background region, in remote sensing images;

[0008] Edge detection operators are used to extract edge information of clouds and snow in the foreground region to obtain the edge information of the foreground region;

[0009] The edge information is fused into the graph cut algorithm framework to segment the cloud layer and snow layer in the foreground region.

[0010] As a preferred embodiment of the present invention, the method for extracting edge information includes:

[0011] Gradient calculation is performed on the remote sensing image I located in the foreground region to obtain the horizontal gradient. Vertical gradient ,in, , , Calculate the Sobel operator respectively , The convolution kernel is denoted by F, where F is the feature image of the remote sensing image I. The brightness features of remote sensing image I, The normalized differential snow cover index (NDSI) for the remote sensing image I. , They are respectively And the weighting coefficient of NDSI, ;

[0012] Based on horizontal gradient and vertical gradient Calculate the gradient magnitude and gradient direction ;

[0013] Canny edge detection is performed on the feature image F of remote sensing image I to obtain a binary edge map U, where edge pixel values ​​are 1 and non-edge pixel values ​​are 0.

[0014] Based on gradient magnitude Using the binary edge map U, the edge weight map of the remote sensing image I is calculated. ,in, For gradient sensitivity, Enhance the parameters for Canny.

[0015] As a preferred embodiment of the present invention, the method for fusing the edge information into a graph cut algorithm framework and segmenting the cloud and snow layers in the foreground region includes:

[0016] Construction Graph ,in, Let E be the set of nodes consisting of snow pixels (source node s) and cloud pixels (sink node t), and let E be the set of nodes consisting of terminal edges connecting pixels to either source node s or sink node t. and the neighborhood edges connecting adjacent pixel nodes The set of edges formed;

[0017] Construction Graph Energy function ,in, For data items, representing pixels. Assigned tags The cost, The smoothing term for fusing edge information represents adjacent pixels. and The penalty for being assigned different labels Represents the set of adjacent pixel pairs. It is a positive coefficient that balances the data terms and the smoothing term;

[0018] ,in, Let be the probability density function for snow pixel label 1. Let be the probability density function for the cloud pixel label 0. The feature value of pixel p in the feature image;

[0019] ,in, The strength coefficient of the smoothing term. , These are the weight values ​​of pixels p and q in the edge weight map, respectively. Let q be the feature value of pixel q in the feature image. For feature similarity parameters, The distance between pixels;

[0020] For each pixel in graph G From the source node To pixel Connect an edge, and the capacity allocation is denoted as . From pixels Remittance node Connect an edge, and the capacity allocation is denoted as . And for each pair of adjacent pixels Connect them with an undirected edge, and the capacity allocation is denoted as . This makes the minimum cut of the graph correspond to the minimization of the energy function;

[0021] Solve for the source node using the maximum flow algorithm. Remittance node The maximum flow and minimum cut divide the remote sensing image I into two parts: with the source node The connected parts are marked as snow pixel label 1, and the sink node The connected parts are marked as cloud pixel label 0.

[0022] As a preferred embodiment of the present invention, the and All follow a Gaussian distribution, where:

[0023] ;

[0024] ;

[0025] , These are the mean and standard deviation of the characteristic values ​​of the snow layer, respectively. , These are the mean and standard deviation of the characteristic values ​​of the cloud layer, respectively.

[0026] As a preferred embodiment of the present invention, the remote sensing image I is divided into two parts by minimum cut and then post-processed to remove small regions, fill holes and smooth boundaries.

[0027] As a preferred embodiment of the present invention, the segmentation results of the graph cut algorithm framework that integrates edge information are combined with the segmentation results of the ResNet network to serve as the segmentation results of cloud and snow layers in remote sensing images.

[0028] As a preferred embodiment of the present invention, the brightness feature ,in, These represent the surface reflectance of pixels in remote sensing image I in the red, green, and blue spectral bands, respectively.

[0029] As a preferred embodiment of the present invention, the normalized differential snow cover index ,in, The surface reflectance is in the blue spectral band. The reflectance is the shortwave infrared band 1.

[0030] As a preferred embodiment of the present invention, the present invention provides a cloud and snow layer image segmentation system based on edge optimization, applied to a cloud and snow layer image segmentation method based on edge optimization, the system comprising:

[0031] The foreground segmentation unit is used to segment the foreground region, which includes clouds and snow, and the background region on remote sensing images using a ResNet network;

[0032] An edge information extraction unit is used to extract the edge information of the cloud and snow layers in the foreground region using an edge detection operator to obtain the edge information of the foreground region;

[0033] The cloud and snow differentiation unit is used to fuse the edge information into the graph cut algorithm framework to segment the cloud layer and snow layer in the foreground region.

[0034] As a preferred embodiment of the present invention, the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement a cloud and snow layer image segmentation method based on edge optimization.

[0035] Compared with the prior art, the present invention has the following advantages:

[0036] This invention first utilizes the ResNet stage to ensure that the cloud and snow foreground is completely detected. Then, it uses the edge optimization graph cut stage to accurately guide the boundary with mixed edge weights, so that the segmented contour highly matches the edge of the real ground features in the image. It makes full use of the advantages of deep learning and graph cut models to achieve high-precision and robust automatic segmentation of clouds and snow in remote sensing images. Attached Figure Description

[0037] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0038] Figure 1 A flowchart of the cloud and snow layer image segmentation method based on edge optimization provided in an embodiment of the present invention;

[0039] Figure 2 A block diagram of a cloud and snow layer image segmentation system based on edge optimization provided in an embodiment of the present invention;

[0040] Figure 3 The image segmentation result of cloud and snow layer based on adjacency relationship provided in the embodiments of the present invention;

[0041] Figure 4 The cloud and snow layer image segmentation results based on the envelopment relationship provided in the embodiments of the present invention. Detailed Implementation

[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] like Figure 1 As shown, this invention provides a cloud and snow layer image segmentation method based on edge optimization, comprising the following steps:

[0044] The ResNet network was used to segment the foreground region containing clouds and snow, as well as the background region, in remote sensing images;

[0045] Edge detection operators are used to extract edge information of clouds and snow in the foreground region to obtain the edge information of the foreground region;

[0046] Edge information is fused into the graph cut algorithm framework to segment clouds and snow layers in the foreground region.

[0047] The clouds and snow have significant differences in spectral features compared to the background. Therefore, existing segmentation networks can achieve good segmentation results between the cloud and snow layers and the background. Thus, this invention applies a pre-trained ResNet segmentation network (such as a ResNet-UNet variant, or other existing segmentation networks, which are not limited here) to process the entire remote sensing image. The network outputs a binary segmentation mask, where a pixel value of 1 represents the foreground region composed of clouds and snow, and 0 represents the ground background. By utilizing the semantic understanding ability of deep learning models for complex scenes, an accurate region of interest is provided for subsequent fine processing, which greatly reduces the risk of subsequent algorithms producing incorrect segmentation of background features.

[0048] After extracting the mixed layer of clouds and snow, this invention abandons the deep learning processing method (because its segmentation boundary is often relatively coarse, and its positioning accuracy for complex texture changes and blurred boundaries inside clouds and snow is limited, which can easily lead to boundary smoothing or loss of details) in order to perform fine segmentation processing on this mixed region. Instead, it chooses to combine edge information with the traditional graph cut algorithm framework for segmentation guidance, avoiding the shortcomings of traditional graph cut algorithms in cloud and snow texture areas (such as wavy clouds and shadow snow) which are prone to over-segmentation or under-segmentation and have difficulty in effectively utilizing the edge structure information of the image itself.

[0049] Methods for extracting edge information include:

[0050] Gradient calculation is performed on remote sensing image I located in the foreground region to obtain the horizontal gradient. Vertical gradient ,in, , , Calculate the Sobel operator respectively , The convolution kernel is denoted by F, where F is the feature image of the remote sensing image I. The brightness features of remote sensing image I, The normalized differential snow cover index (NDSI) for remote sensing image I. , They are respectively And the weighting coefficient of NDSI, ;

[0051] Based on horizontal gradient and vertical gradient Calculate the gradient magnitude and gradient direction ;

[0052] Canny edge detection is performed on the feature image F of remote sensing image I to obtain a binary edge map U, where edge pixel values ​​are 1 and non-edge pixel values ​​are 0.

[0053] Based on gradient magnitude Using the binary edge map U, calculate the edge weight map of the remote sensing image I. ,in, For gradient sensitivity, Enhance the parameters for Canny.

[0054] This invention calculates the luminance features of each pixel. and Normalized Difference Snow Cover Index Linear fusion into a comprehensive feature map . Simultaneously responding to the brightness of clouds and the high NDSI value of snow, maximizing the difference in cloud and snow characteristics.

[0055] And calculate in the extraction of edge information gradient magnitude To obtain continuous edge intensity information, for Perform Canny detection to obtain a binarized and structured edge map. Merging M and U forms an edge weight map. The exponential part reduces the weights in the strong gradient region; multiplying by Later at the edge of the Canny ( The weighting was further suppressed. Therefore, The lower the value, the greater the likelihood that the location is the true boundary of the clouds and snow.

[0056] Specifically, the index part: ,gradient The larger the value, the smaller the value of this item, corresponding to a lower weight. Controlling the descent speed, i.e. The larger the value, the less sensitive it is to gradients. It can be set to 0.5-1.5 times the average value of the image gradient magnitude based on image statistics, or set near the valley between the two main peaks (corresponding to flat areas and edge areas) in the image gradient magnitude histogram based on histogram analysis.

[0057] Canny enhancements: At the edge of the Canny ( ): Multiplier is At non-edge locations ( ): Multiplier is 1, parameter The enhancement level of the Canny edges is controlled and is usually set to... Within the range, adjust according to Canny edge quality.

[0058] The combination of the two makes Simultaneously subject to continuous gradient and discrete Canny edges The effect is that at the Canny edges, the weights are multiplied by an additional factor. Factors, further reduce weights, when When, it degenerates into pure gradient weights; when At that time, the weights at the edges of the Canny are forced to be 0.

[0059] This invention is applicable to scenarios where small areas of snow are segmented from large cloud layers. Specifically, in segmenting mixed cloud and snow layers, the cloud layer is used as the background and the snow layer as the foreground for fine-grained segmentation, ultimately forming the segmentation result. Edge information is introduced into the graph cut algorithm to optimize edges, resulting in a relatively small gradient magnitude M at texture locations, thus increasing the edge weight. The cost of the smoothing term remains relatively large. The algorithm still tends to assign the same label to these texture regions, keeping them as a whole. Only those true contours whose gradients are significantly greater than the texture changes are selected as boundaries, avoiding the possibility that traditional graph cut algorithms might mistakenly treat these texture changes as boundaries, leading to oversegmentation (splitting a whole cloud into many fragments).

[0060] Methods that integrate edge information into a graph cut algorithm framework for cloud and snow layer segmentation in the foreground region include:

[0061] Construction Graph ,in, Let E be the set of nodes consisting of snow pixels (source node s) and cloud pixels (sink node t), and let E be the set of nodes consisting of terminal edges connecting pixels to either source node s or sink node t. and the neighborhood edges connecting adjacent pixel nodes The set of edges formed;

[0062] Construction Graph Energy function ,in, For data items, representing pixels. Assigned tags The cost, The smoothing term for fusing edge information represents adjacent pixels. and The penalty for being assigned different labels It represents a set of adjacent pixel pairs (usually using 4-neighborhood or 8-neighborhood). It is a positive coefficient (0.5~2.0) for balancing data terms and smoothing terms.

[0063] ,in, Let be the probability density function for snow pixel label 1. Let be the probability density function for the cloud pixel label 0. The feature value of pixel p in the feature image;

[0064] ,in, The strength coefficient of the smoothing term. , These are the weight values ​​of pixels p and q in the edge weight map, respectively. Let q be the feature value of pixel q in the feature image. The feature similarity parameter is usually set to the standard deviation of the feature image F. The distance between pixels (the distance between pixels is 1 for 4-neighborhood, and 1 for 8-neighborhood) is... ), ;

[0065] Part 1: Average Marginal Weights This is the core component of the edge optimization concept. It directly incorporates the edge weight map calculated from the image. Information. The lower the value, the greater the probability that pixel p is located at a real edge (such as the boundary of clouds or snow). By averaging the weights of adjacent pixels p and q, it measures the probability that the "edge" connecting these two pixels is located at an edge.

[0066] The first part serves as a spatially adaptive modulator. In regions where the value is very low (at strong edges), this value is very small, thus reducing the overall cost of the smoothing term. They become very small. When minimizing the total energy, the algorithm tends to let the segmentation boundary cross these "low-cost" regions, thus forcing the final segmentation contour to align with the true visual edges in the image.

[0067] exist In areas with high values ​​(flat cloud interiors or snow surfaces), this value is very large, leading to... Very high. This imposes a huge penalty on breaking labels at this point (i.e., creating boundaries), effectively suppressing false, broken segments caused by noise or texture within objects, ensuring the integrity of the segmented region.

[0068] Use the average value instead of a single pixel. The value provides a certain degree of smoothness and noise resistance, making the determination of edge positions more robust.

[0069] Part Two: Feature Similarity Penalty This is based on pixel-level feature similarity. and These are the values ​​of pixels p and q on a feature map (such as a brightness or NDSI fusion map). This exponential function measures the similarity between the two: when... and When they are very close, the function value approaches 1, indicating "identical characteristics". As the difference between the two increases, the function value decays from 1 to 0, and the decay rate is determined by the parameter. control. The smaller the value, the more sensitive it is to differences.

[0070] It works in conjunction with the edge weight term to jointly determine whether a boundary should be defined between two pixels. Even if two pixels are located in non-edge regions (where the first part has a large value), if their feature values ​​differ significantly (where this term's value is close to 0), the overall... It may also not be too high, thus allowing segmentation at points of abrupt feature change (even if not at strong gradient edges). This captures the segmentation logic based on region characteristics.

[0071] In the internal texture regions of clouds or snow, although there may be slight gradient variations (causing the first part) (Not a maximum value), but the feature values ​​𝐹 between pixels are usually still similar (this value is close to 1). The product of the two will still maintain a relatively high value. This suppresses the misclassification of subtle textures as object boundaries. Only those locations that simultaneously satisfy the criteria of being "located in a low edge weight region" and having "significant feature differences" are considered the most reasonable segmentation boundary candidates. Compared to hard thresholding, the exponential function provides a smooth, probabilistic similarity measure, making the energy function easier to optimize.

[0072] Part 3: Distance Normalization , It is the Euclidean distance between pixels p and q. In a standard image grid, for a 4-neighborhood (top, bottom, left, right)... 1; For diagonal adjacency in an 8-neighborhood, This term is a geometric normalization factor.

[0073] Without this, the "connection" between diagonally adjacent pixel pairs is naturally longer than that between horizontally / vertically adjacent pixel pairs. Without normalization, the algorithm might unintentionally assign different importance to diagonal connections, leading to directional biases in the segmentation boundaries (e.g., a preference for horizontal and vertical paths over the shortest straight path). Dividing by the distance ensures that the boundary cost per unit length is uniform in physical space, making the segmentation boundaries geometrically fair and isotropic.

[0074] It embodies the intuitive idea that "the penalty should be proportional to the boundary length." In a continuous domain, the boundary cost is the integral along the boundary curve. In a discrete graphical model, The summation approximates the measure of the boundary length, making energy minimization directly related to finding a shorter segmentation boundary that conforms to image features.

[0075] These three parts in the smoothing term Multiplication creates a powerful and sophisticated constraint mechanism: when the algorithm considers whether to draw a dividing line between adjacent pixels p and q, it comprehensively calculates a "cost". This cost is very low (encouraging segmentation) in the following cases: (p, q) is located on an edge detected by Canny with a large image gradient (small value in the first part), and the feature values ​​of p and q are significantly different (small value in the second part), while this judgment is independent of the spatial distance between pixels (normalized by the third part). Conversely, this cost is very high (suppressing segmentation) in the following cases: (p, q) is located in a flat and uniform region of the image (large value in the first part), or the feature values ​​of p and q are very similar (large value in the second part).

[0076] This invention, through its design, enables the graph cut algorithm to find an optimal segmentation boundary that both fits the obvious visual edges in the image, respects the consistency of features within the region, and is geometrically simple and reasonable. This is the fundamental reason why the "edge-optimization-based" segmentation method can obtain accurate and complete segmentation results when dealing with complex scenes such as clouds and snow.

[0077] For each pixel in graph G From the source node To pixel Connect an edge, and the capacity allocation is denoted as . From pixels Remittance node Connect an edge, and the capacity allocation is denoted as . And for each pair of adjacent pixels Connect them with an undirected edge, and the capacity allocation is denoted as . This makes the minimum cut of the graph correspond to the minimization of the energy function;

[0078] Use maximum flow algorithms (such as Ford-Fulkerson or Push-relabel) to solve for the source node. Remittance node The maximum flow and minimum cut divide the remote sensing image I into two parts: with the source node The connected parts are marked as snow pixel label 1, and the sink node The connected parts are marked as cloud pixel label 0.

[0079] and All follow a Gaussian distribution, where:

[0080] ;

[0081] ;

[0082] , These are the mean and standard deviation of the characteristic values ​​of the snow layer, respectively. , These are the mean and standard deviation of the characteristic values ​​of the cloud layer, respectively.

[0083] In graph cuts, the data term measures the cost of assigning a single pixel to a certain label (in this case, "cloud" or "snow"), based on the pixel's own features (such as brightness, NDSI, etc.). Specifically, it reflects the degree of matching between the pixel's feature values ​​and a statistical model of a certain label. It utilizes the statistical properties of pixel feature values ​​to independently classify each pixel into the most likely (highest probability) category. Because the negative log-likelihood has a small value when the probability is high and a large value when the probability is low, minimizing the sum of the data terms is equivalent to maximizing the joint likelihood probability of all pixels.

[0084] However, simply having data terms leads to independent classification of each pixel, potentially resulting in noise and isolated points. Therefore, a smoothing term is needed to make adjacent and similar pixels tend to have the same label, thus obtaining smooth and continuous segmented regions.

[0085] Therefore, based on the pixel's feature values, the negative logarithmic probability of the pixel belonging to either cloud or snow is calculated as the cost of assigning the pixel the corresponding label. Its advantage lies in providing a classification prior based on a statistical model, ensuring that the segmentation result is consistent with the statistical distribution of the pixel's feature values.

[0086] The parameters (mean and variance) of the Gaussian distribution can be estimated by initial segmentation (e.g., using a simple thresholding method or from the initial segmentation of ResNet), or by iterative estimation (such as the EM algorithm) alternating with graph cuts, without limitation.

[0087] This invention transforms the cloud-snow separation problem into a problem involving the energy function within a mixing region. The minimization problem, where the label Represents snow, Represents the cloud. Among them, data items... Based on eigenvalues Assume that the labels for clouds and snow each follow a Gaussian distribution. and . The value is the negative log-likelihood, which drives the pixel to be assigned to the more likely category (cloud or snow) based on its feature value.

[0088] When adjacent pixels There will be a cost when different labels are assigned. The cost is related to the average edge weights between them. Proportional: at the edge of reality ( Small, low cost, allows for boundary delineation here. In uniform regions or weakly textured areas ( Large (and costly), it strongly suppresses segmentation at this point, thereby maintaining the internal integrity of the cloud or snow region and avoiding excessive segmentation.

[0089] This invention transforms the above energy minimization problem into a minimum cut problem on a network flow graph, and solves it efficiently using the maximum flow algorithm to obtain a fine label map of whether each pixel is a cloud (0) or snow (1).

[0090] The remote sensing image I is divided into two parts by minimum cut, and then post-processing is performed to remove small regions, fill holes, and smooth the boundaries.

[0091] The segmentation results of the graph cut algorithm framework that integrates edge information are combined with the segmentation results of the ResNet network to obtain the segmentation results of cloud and snow layers in remote sensing images. The resulting fine cloud and snow segmentation map is logically combined with the foreground-background segmentation map obtained by ResNet to obtain the final three-class segmentation results (background, cloud, and snow). Morphological post-processing (small region removal, hole filling, and boundary smoothing) is used to improve the spatial consistency of the results.

[0092] Brightness characteristics ,in, These represent the surface reflectance of pixels in remote sensing image I in the red, green, and blue spectral bands, respectively.

[0093] Normalized Difference Snow Cover Index ,in, The surface reflectance is in the blue spectral band. The reflectance is the shortwave infrared band 1.

[0094] like Figure 3 and Figure 4 As shown, Figure 3 In the image, a is the original remote sensing image, showing large areas of cloud pixels adjacent to small areas of snow pixels; b is the edge map detected by Canny; c is the segmentation result of the cloud-snow mixture region (white) and the background region (black) obtained by the ResNet network; and d is the final segmentation result of clouds (green), snow (red), and background (black).

[0095] Similarly, Figure 4 In the image, a is the original remote sensing image, which shows a large area of ​​cloud pixels surrounding a small area of ​​snow pixels. b is the edge map detected by Canny. c is the segmentation result of the cloud-snow mixture region (white) and the background region (black) obtained by the ResNet network. d is the final segmentation result of the cloud (green), snow (red), and background (black).

[0096] Figure 3 and Figure 4 The mid-level clouds and snow layer are adjacent in a band, with wavy textures at the cloud edges. The snow layer edges are gradually transitioned (without clear boundaries) due to the influence of cloud shadows. Figure 3 and Figure 4 The results show that the Canny edge map in Figure 3b accurately extracts the core edges of the clouds and snow (excluding texture interference). In the final result in Figure 3d, the cloud and snow boundaries highly match the Canny edges, without any "boundary drift" phenomenon. This avoids the defect of traditional methods that misjudge cloud textures as boundaries due to the lack of edge information, resulting in false segmentation blocks. This method... Modulation smoothing term, increasing the cost of texture region smoothing term. Increase the intensity and force consistent texture area labels, retaining only the realistic cloud and snow boundaries.

[0097] Figure 4 The area of ​​the snow layer is only a small part of the cloud-snow mixture area (small target), and it is completely covered by thick clouds. The difference in spectral reflectance between the snow layer and the cloud layer is small, and traditional methods are prone to misidentifying the snow layer as the cloud layer.

[0098] In Figure 4c, the ResNet foreground segmentation accurately excluded the ground background and completely extracted the cloud and snow mixed area, laying the foundation for subsequent fine segmentation. In the final result of Figure 4d, small areas of snow were completely detected without omission or breakage, and the snow layer boundary was smooth (after post-processing optimization), verifying that the feature image F can effectively amplify the differences in cloud and snow features (the snow layer has a higher NDSI value, and the cloud layer has a higher brightness). b (Higher); Gaussian distributed data item Dp(Lp) can accurately classify similar spectral pixels based on statistical properties.

[0099] This invention can effectively distinguish small snow pixels within a large area of ​​cloud pixels, regardless of whether they are adjacent or enclosed. In other words, in mixed cloud and snow scenes (especially in scenes with large cloud areas and small snow areas), this invention can accurately achieve cloud and snow segmentation, or accurately identify small snow areas within a large cloud area, without causing over-segmentation (dividing a whole cloud into many fragments).

[0100] Therefore, this invention enables the graph cut algorithm to perform "low-cost segmentation" at real boundaries and "high-cost suppressed segmentation" in textured regions, thereby improving the fit between the segmented contour and the edge of the real ground feature.

[0101] This invention ensures complete detection of cloud and snow foreground in the ResNet stage, and utilizes hybrid edge weights in the edge optimization graph cut stage. Precise boundary guidance ensures that the segmented contour closely matches the edges of real features in the image, overcoming the shortcomings of blurred boundaries in deep learning models and fragmented boundaries in traditional methods.

[0102] In other words, the ResNet stage accurately excludes background areas (such as the ground surface and water bodies), allowing subsequent graph cut algorithms to focus only on the cloud and snow mixed areas, thus improving computational efficiency. At the same time, the fine segmentation results of graph cut correct the coarse boundaries of ResNet (such as the jagged edges of the snow layer segmented by ResNet, which are smooth and continuous after graph cut optimization), achieving a synergistic effect of "coarse segmentation preserving integrity and fine segmentation preserving accuracy".

[0103] This invention is achieved through By modulating the smoothing term, the algorithm can effectively distinguish between real cloud and snow boundaries and texture fluctuations within clouds or on the snow surface. In textured regions, due to the gradient magnitude... Relatively small, The value is large, resulting in a higher cost for the smoothing term. The high resolution forces consistent labeling within the region. It can segment large, textured cloud or snow layers into complete single objects, rather than multiple fragments, resulting in a more physically realistic outcome and significantly reducing over-segmentation.

[0104] This invention provides a cloud and snow layer image segmentation system based on edge optimization, applied to a cloud and snow layer image segmentation method based on edge optimization. The system includes:

[0105] The foreground segmentation unit is used to segment the foreground region, which includes clouds and snow, and the background region on remote sensing images using a ResNet network;

[0106] The edge information extraction unit is used to extract the edge information of clouds and snow in the foreground region using edge detection operators to obtain the edge information of the foreground region;

[0107] The cloud and snow separation unit is used to fuse edge information into the graph cut algorithm framework to segment the cloud and snow layers in the foreground region.

[0108] The present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement a cloud and snow layer image segmentation method based on edge optimization.

[0109] This invention first utilizes the ResNet stage to ensure that the cloud and snow foreground is completely detected. Then, it uses the edge optimization graph cut stage to accurately guide the boundary with mixed edge weights, so that the segmented contour highly matches the edge of the real ground features in the image. It makes full use of the advantages of deep learning and graph cut models to achieve high-precision and robust automatic segmentation of clouds and snow in remote sensing images.

[0110] The above embodiments are merely exemplary embodiments of this application and are not intended to limit this application. The scope of protection of this application is defined by the claims. Those skilled in the art can make various modifications or equivalent substitutions to this application within its substance and scope of protection, and such modifications or equivalent substitutions should also be considered to fall within the scope of protection of this application.

Claims

1. A cloud and snow layer image segmentation method based on edge optimization, characterized in that, Includes the following steps: The ResNet network was used to segment the foreground region containing clouds and snow, as well as the background region, in remote sensing images; Edge detection operators are used to extract edge information of clouds and snow in the foreground region to obtain the edge information of the foreground region; The edge information is fused into the graph cut algorithm framework to segment the cloud layer and snow layer in the foreground region.

2. The cloud and snow layer image segmentation method based on edge optimization according to claim 1, characterized in that: The method for extracting the edge information includes: Gradient calculation is performed on the remote sensing image I located in the foreground region to obtain the horizontal gradient. Vertical gradient ,in, , , Calculate the Sobel operator respectively , The convolution kernel is denoted by F, where F is the feature image of the remote sensing image I. The brightness features of remote sensing image I, The normalized differential snow cover index (NDSI) for the remote sensing image I. , They are respectively And the weighting coefficient of NDSI, ; Based on horizontal gradient and vertical gradient Calculate the gradient magnitude and gradient direction ; Canny edge detection is performed on the feature image F of remote sensing image I to obtain a binary edge map U, where edge pixel values ​​are 1 and non-edge pixel values ​​are 0. Based on gradient magnitude Using the binary edge map U, the edge weight map of the remote sensing image I is calculated. ,in, For gradient sensitivity, Enhance the parameters for Canny.

3. The cloud and snow layer image segmentation method based on edge optimization according to claim 2, characterized in that: The method for fusing the edge information into a graph cut algorithm framework and segmenting the cloud and snow layers in the foreground region includes: Construction Graph ,in, Let E be the set of nodes consisting of snow pixels (source node s) and cloud pixels (sink node t), and let E be the set of nodes consisting of terminal edges connecting pixels to either source node s or sink node t. and the neighborhood edges connecting adjacent pixel nodes The set of edges formed; Construction Graph Energy function ,in, For data items, representing pixels. Assigned tags The cost, The smoothing term for fusing edge information represents adjacent pixels. and The penalty for being assigned different labels Represents the set of adjacent pixel pairs. It is a positive coefficient that balances the data terms and the smoothing term; ,in, Let be the probability density function for snow pixel label 1. Let be the probability density function for the cloud pixel label 0. The feature value of pixel p in the feature image; ,in, The strength coefficient of the smoothing term. , These are the weight values ​​of pixels p and q in the edge weight map, respectively. Let q be the feature value of pixel q in the feature image. For feature similarity parameters, The distance between pixels; For each pixel in graph G From the source node To pixel Connect an edge, and the capacity allocation is denoted as . From pixels Remittance node Connect an edge, and the capacity allocation is denoted as . And for each pair of adjacent pixels Connect them with an undirected edge, and the capacity allocation is denoted as . This makes the minimum cut of the graph correspond to the minimization of the energy function; Solve for the source node using the maximum flow algorithm. Remittance node The maximum flow and minimum cut divide the remote sensing image I into two parts: with the source node The connected parts are marked as snow pixel label 1, and the sink node The connected parts are marked as cloud pixel label 0.

4. The cloud and snow layer image segmentation method based on edge optimization according to claim 3, characterized in that: The and All follow a Gaussian distribution, where: ； ； , These are the mean and standard deviation of the characteristic values ​​of the snow layer, respectively. , These are the mean and standard deviation of the characteristic values ​​of the cloud layer, respectively.

5. The cloud and snow layer image segmentation method based on edge optimization according to claim 3, characterized in that: The remote sensing image I is divided into two parts by minimum cut, and then post-processing is performed to remove small regions, fill holes, and smooth the boundaries.

6. The cloud and snow layer image segmentation method based on edge optimization according to claim 3, characterized in that: The segmentation results of the graph cut algorithm framework that integrates edge information are combined with the segmentation results of the ResNet network to serve as the segmentation results of cloud and snow layers in remote sensing images.

7. The cloud and snow layer image segmentation method based on edge optimization according to claim 2, characterized in that: The brightness feature ,in, These represent the surface reflectance of pixels in remote sensing image I in the red, green, and blue spectral bands, respectively.

8. The cloud and snow layer image segmentation method based on edge optimization according to claim 2, characterized in that: The normalized difference snow cover index ,in, The surface reflectance is in the blue spectral band. The reflectance is the shortwave infrared band 1.

9. A cloud and snow layer image segmentation system based on edge optimization, characterized in that, The system, applied to the edge-optimized cloud and snow layer image segmentation method according to any one of claims 1-9, comprises: The foreground segmentation unit is used to segment the foreground region, which includes clouds and snow, and the background region on remote sensing images using a ResNet network; An edge information extraction unit is used to extract the edge information of the cloud and snow layers in the foreground region using an edge detection operator to obtain the edge information of the foreground region; The cloud and snow differentiation unit is used to fuse the edge information into the graph cut algorithm framework to segment the cloud layer and snow layer in the foreground region.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the method as described in any one of claims 1-8.

Images