Remote sensing image semantic segmentation method based on deep learning model

Abstract

The invention relates to the technical field of image processing, in particular to a remote sensing image semantic segmentation method based on a deep learning model. Acquiring multispectral remote sensing image data of the target area, wherein the multispectral remote sensing image data comprises image information of a plurality of spectral bands; constructing a ground feature recognizer, wherein the ground feature recognizer is used for recognizing the ground feature category in the multispectral remote sensing image data; calling a preset window size sequence, and carrying out ground feature recognition labeling on the multispectral remote sensing image data in combination with a ground feature recognizer to obtain a ground feature category labeling result of the target area; and obtaining a surface feature semantic segmentation result of the target area according to the surface feature category labeling result. And high-precision and high-efficiency ground feature semantic segmentation of the multispectral remote sensing image of the target area is realized.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a semantic segmentation method for remote sensing images based on a deep learning model. Background Technology

[0002] Traditional semantic segmentation of remote sensing images primarily relies on pixel-based methods such as thresholding and edge detection. These methods calculate the differences in grayscale values ​​between pixels and use techniques like thresholding and edge detection to segment ground features; for example, they use grayscale thresholds from single-band images to distinguish between water and land. However, traditional semantic segmentation largely depends on single-band or a few-band image data, failing to fully utilize the rich spectral information in multispectral images. Furthermore, traditional segmentation methods often employ single-scale analysis, making them unsuitable for segmenting large-scale remote sensing images. Summary of the Invention

[0003] This invention addresses the technical problems of existing technologies failing to fully utilize the rich spectral information in multispectral images and being unable to adapt to the segmentation needs of large-scale remote sensing images by providing a remote sensing image semantic segmentation method based on a deep learning model.

[0004] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: In a first aspect, the present invention provides a method for semantic segmentation of remote sensing images based on a deep learning model, comprising: acquiring multispectral remote sensing image data of a target area, wherein the multispectral remote sensing image data contains image information of multiple spectral bands; constructing a land cover identifier, wherein the land cover identifier is used to identify land cover categories in the multispectral remote sensing image data; retrieving a preset window size sequence, and combining the land cover identifier to perform land cover identification and annotation on the multispectral remote sensing image data to obtain land cover category annotation results for the target area; and obtaining a land cover semantic segmentation result for the target area based on the land cover category annotation results.

[0005] Optionally, acquiring multispectral remote sensing image data of the target area includes: acquiring a preset spectral band set; acquiring remote sensing images of the target area based on the preset spectral band set to obtain multiple raw spectral images of the target area; and preprocessing the multiple raw spectral images to obtain the multispectral remote sensing image data.

[0006] The preprocessing of the plurality of original spectral images to obtain the multispectral remote sensing image data includes: performing radiometric correction on the plurality of original spectral images to obtain a plurality of first corrected spectral images; performing geometric correction on the plurality of first corrected spectral images to obtain a plurality of second corrected spectral images; performing registration processing on the plurality of second corrected spectral images to ensure pixel alignment in each spectral band to obtain a plurality of registered spectral images; and performing normalization processing on the plurality of registered spectral images to obtain the multispectral remote sensing image data.

[0007] Optionally, constructing a land cover identifier includes: acquiring multiple land cover categories, performing multispectral remote sensing image data retrieval for each land cover category, constructing a sample single-class land cover multispectral dataset and a sample land cover category label set; using the sample single-class land cover multispectral dataset as training input and the sample land cover category label set as training labels to train a deep neural network to generate the land cover identifier.

[0008] Optionally, a preset window size sequence is retrieved, and the land cover identifier is used to perform land cover identification and annotation on the multispectral remote sensing image data to obtain the land cover category annotation results for the target area. This includes: retrieving the preset window size sequence, which includes N preset window sizes; extracting a first-level preset window size from the N preset window sizes in descending order of size, and dividing the multispectral remote sensing image data into windows based on the first-level preset window size to obtain multiple first-level window data; and using the land cover identifier to annotate the multiple first-level window data to obtain the first-level labeled results. The process involves extracting secondary preset window sizes from the N preset window sizes and dividing the primary unlabeled data into multiple secondary window data based on these secondary preset window sizes. The feature identifier is then used to label the multiple secondary window data, resulting in a secondary labeled result set and secondary unlabeled data. This process of window size extraction, window division, and feature labeling is repeated until the preset window size sequence is processed or the multispectral remote sensing image data is labeled. Based on the labeled result sets at each level, a feature category labeling result for the target area is formed.

[0009] The process of annotating multiple primary window data using the feature identifier to obtain a primary labeled result set and primary unlabeled data includes: extracting first primary window data from the multiple primary window data; calculating the spectral consistency index of the first primary window data; determining whether the spectral consistency index meets a preset homogeneity condition; if it does, identifying the first primary window data as homogeneous primary window data; extracting the statistical features of the homogeneous primary window data in each spectral band to form a window feature vector; inputting the window feature vector into the feature identifier to obtain a feature category prediction result; using the feature category prediction result as the feature label of the homogeneous primary window data and adding the homogeneous primary window data to the primary labeled result set; if the spectral consistency index does not meet the preset homogeneity condition, adding the first primary window data to the primary unlabeled data; and processing the remaining primary window data sequentially to complete the feature annotation processing of all primary window data.

[0010] The calculation of the spectral consistency index of the first-level window data includes: extracting all pixels in the first-level window data and obtaining the reflectance value of each pixel in each spectral band. The variance of the reflectance values ​​of all pixels in each spectral band is calculated to obtain the band consistency index corresponding to each spectral band; the spectral consistency index of the first-level window data is obtained based on the band consistency index corresponding to each spectral band.

[0011] The process of obtaining the semantic segmentation result of the target area based on the land cover category labeling results includes: extracting the labeled result sets at each level from the land cover category labeling results, obtaining the spatial location information and land cover category identifier of each labeled window; mapping the corresponding land cover category identifier to the corresponding spatial location of the multispectral remote sensing image data based on the spatial location information to obtain a land cover distribution map of the target area, wherein the land cover distribution map records the land cover category identifier corresponding to each spatial location; and outputting the land cover distribution map as the semantic segmentation result of the target area.

[0012] By implementing this invention, it is possible to acquire multispectral remote sensing image data of a target area. The multispectral remote sensing image data contains image information of multiple spectral bands. The multispectral data contains richer spectral information of ground features and can distinguish different ground features more finely compared to single-band images. A ground feature identifier is constructed to identify the types of ground features in multispectral remote sensing image data. The ground feature identifier based on deep neural networks has powerful feature learning and fitting capabilities and can automatically learn the deep spectral and spatial features of ground features from massive multispectral data. Compared with traditional identification methods based on manually designed features, it is more adaptable. A preset window size sequence is retrieved, and the land cover identifier is used to identify and label land cover in the multispectral remote sensing image data to obtain the land cover category labeling results of the target area. A multi-scale window division method is adopted. Large-size windows can quickly process large areas of homogeneous regions and improve labeling efficiency; small-size windows can finely process heterogeneous regions and improve the land cover labeling accuracy of complex regions, thus achieving a balance between efficiency and accuracy. Based on the land feature category labeling results, the semantic segmentation results of the target area are obtained, and the discrete window labeling results are transformed into a continuous land feature distribution map, which intuitively presents the spatial distribution of various features in the target area, making the segmentation results more practical and facilitating further applications such as land feature area statistics and change detection.

[0013] In summary, by implementing this invention, high-precision and high-efficiency semantic segmentation of ground features in multispectral remote sensing images of target areas can be achieved. Attached Figure Description

[0014] Figure 1 A flowchart illustrating a remote sensing image semantic segmentation method based on a deep learning model provided by this invention; Figure 2 This is a schematic diagram illustrating the process of labeling land cover categories in a remote sensing image semantic segmentation method based on a deep learning model provided by the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0017] In the description of this invention, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed herein.

[0018] Example 1, as Figure 1 As shown, this embodiment of the invention provides a method for semantic segmentation of remote sensing images based on a deep learning model, including: S100: Acquire multispectral remote sensing image data of the target area, wherein the multispectral remote sensing image data contains image information of multiple spectral bands; S200: Construct a land cover identifier, which is used to identify land cover categories in multispectral remote sensing image data; S300: Retrieve a preset window size sequence, and combine it with the land feature identifier to perform land feature identification and annotation on the multispectral remote sensing image data to obtain the land feature category annotation result of the target area; S400: Obtain the semantic segmentation result of the target area based on the land feature category labeling results.

[0019] In step S100 of this application embodiment, acquiring multispectral remote sensing image data of the target area includes: A preset spectral band set is obtained, and remote sensing images of the target area are acquired based on the preset spectral band set to obtain multiple original spectral images of the target area. The multiple original spectral images are preprocessed to obtain the multispectral remote sensing image data.

[0020] In this embodiment, the purpose of step S100 is to collect multi-dimensional spectral information and eliminate data interference, thereby providing high-quality and reliable basic data support for subsequent land cover identification and semantic segmentation. The specific process of step S100 is divided into two core steps: raw data acquisition and data preprocessing.

[0021] First, a preset spectral band set needs to be obtained. Based on this preset spectral band set, remote sensing images of the target area are acquired, resulting in multiple raw spectral images of the target area. That is, a "preset spectral band set" is set according to the actual needs of land cover identification. For example, in land use surveys where there is a need to distinguish between vegetation, water bodies, and buildings, the preset spectral band set needs to include the visible light red, green, and blue bands, as well as near-infrared and short-wave infrared bands.

[0022] Then, using remote sensing equipment such as satellites and drones equipped with corresponding spectral sensors, images are synchronously acquired for the target area according to a preset set of spectral bands, ultimately resulting in "multiple original spectral images" corresponding to the preset number of spectral bands. Each preset spectral band corresponds to one original image, ensuring that the image of each spectral band covers the same target area.

[0023] Furthermore, the multiple original spectral images need to be preprocessed to obtain the multispectral remote sensing image data.

[0024] In step S100 of this embodiment, the multiple original spectral images are preprocessed to obtain the multispectral remote sensing image data, including: Radiometric correction is performed on the multiple original spectral images to obtain multiple first corrected spectral images; Geometric correction is performed on the plurality of first corrected spectral images to obtain a plurality of second corrected spectral images; The multiple second-corrected spectral images are registered to ensure that the pixel positions of each spectral band are aligned, resulting in multiple registered spectral images. The multiple registered spectral images are normalized to obtain the multispectral remote sensing image data.

[0025] Specifically, the first step is to perform radiometric correction on the multiple original spectral images to obtain multiple first-corrected spectral images. This involves eliminating radiometric interference from factors other than the ground objects themselves, and correcting deviations in radiometric values ​​caused by factors such as sensor response errors, atmospheric scattering / absorption, and changes in solar altitude angle using a radiometric correction algorithm. This ensures that the pixel radiometric values ​​of each original spectral image accurately reflect the reflectivity of the ground objects, resulting in "multiple first-corrected spectral images." The radiometric correction algorithm used is existing technology and will not be described in detail here.

[0026] Furthermore, geometric correction needs to be performed on the multiple first-corrected spectral images to obtain multiple second-corrected spectral images. This involves correcting the geometric positional deviations of the first-corrected spectral images. Addressing image distortions caused by terrain undulations, Earth curvature, and sensor attitude changes, and using ground control points as a reference, methods such as affine transformation and polynomial fitting are employed to correct the "first-corrected spectral images" to conform to the geographic coordinate system, resulting in "multiple second-corrected spectral images." This ensures that the geometric position of the image matches the actual location of ground features.

[0027] Furthermore, the multiple second-corrected spectral images need to be registered to ensure that the pixel positions of each spectral band are aligned, resulting in multiple registered spectral images. This achieves pixel alignment across multiple bands. Since sensors in different spectral bands may have slight imaging time differences or viewing angle differences, an image from one band is used as a reference. Feature matching algorithms such as SIFT and SURF are used to find the corresponding pixels in other band images and the reference image, adjusting the image positions so that the same ground feature is at the same pixel coordinates in all band images, resulting in "multiple registered spectral images".

[0028] Finally, the multiple registered spectral images need to be normalized to obtain the multispectral remote sensing image data. This involves unifying the numerical range of the multi-band data. The original pixel values ​​of different spectral bands may be on different orders of magnitude due to differences in sensor sensitivity; for example, the visible light band value range is 0-255, and the near-infrared band value range is 0-1000. Through methods such as linear normalization and max-min normalization, the pixel values ​​of all registered spectral images are mapped to the same standard range, such as 0-1, eliminating the brightness imbalance problem between bands and ultimately forming the "multispectral remote sensing image data". The final "multispectral remote sensing image data" is a three-dimensional data cube with dimensions of "height (rows) × width (columns) × number of spectral bands". All subsequent steps are based on this three-dimensional data cube.

[0029] In step S200 of this application embodiment, constructing a land feature identifier includes: Multiple land cover categories are obtained, and multispectral remote sensing image data retrieval is performed for each land cover category to construct a multispectral dataset of a single land cover category and a label set of the land cover category. The sample single-class land cover multispectral dataset is used as training input, and the sample land cover category label set is used as training labels to train a deep neural network to generate the land cover identifier.

[0030] In this embodiment of the application, the purpose of step S200 is to construct an intelligent model capable of accurately identifying various land features in multispectral remote sensing images, providing a reliable classification tool for the subsequent annotation process of land feature semantic segmentation.

[0031] First, it is necessary to obtain training data for the land cover identifier, that is, to obtain multiple land cover categories, and to perform multispectral remote sensing image data retrieval for each land cover category to construct a sample single-class land cover multispectral dataset and a sample land cover category label set.

[0032] Specifically, based on the actual application scenario, "multiple land feature categories" can be determined. For example, urban planning needs to identify buildings, roads, green spaces, etc.; agricultural monitoring needs to identify cultivated land, forest land, irrigation areas, etc., to clarify the target types that the land feature identifier needs to cover, and avoid data redundancy or omission of key categories.

[0033] Then, for each defined land cover category, the corresponding multispectral remote sensing image data is retrieved. For example, for the "vegetation" category, multispectral images containing different vegetation types such as forests, grasslands, and farmland are retrieved; for the "water body" category, multispectral images containing different water body forms such as rivers, lakes, and reservoirs are retrieved, ensuring that the samples for each category are diverse and cover land cover images under different terrain, lighting, and seasonal conditions.

[0034] Finally, the retrieved multispectral images for each land cover category are organized into "sample single-class land cover multispectral datasets," with each land cover category corresponding to an independent sample single-class land cover multispectral dataset for convenient subsequent targeted training. Image samples in each sample single-class land cover multispectral dataset are manually or semi-automatically labeled to clearly define the land cover category corresponding to each sample, such as labeling "Sample A - Vegetation" and "Sample B - Water Body," forming a "sample land cover category label set" that corresponds one-to-one with the sample single-class land cover multispectral dataset, ensuring that each training sample has a clear learning objective. Each sample single-class land cover multispectral dataset contains 10,000 multispectral image samples and corresponding sample land cover category labels. As a single-class land cover dataset, each single-class land cover dataset is divided into a training set, a validation set, and a test set in a 7:2:1 ratio. These sets are used to train the land cover identifier.

[0035] Furthermore, the sample single-class land cover multispectral dataset needs to be used as training input, and the sample land cover category label set needs to be used as training labels to train the deep neural network and generate the land cover identifier.

[0036] For different types of ground feature identifiers, the U-Net deep neural network can be used to build them. The U-Net deep neural network has a symmetrical "encoder-decoder" structure, which can extract deep spectral features and spatial texture features from multispectral remote sensing images through the encoding layer, and accurately reconstruct the spatial location information of ground features through the decoding layer.

[0037] The feature identifier consists of two parts: an encoding layer and a decoding layer.

[0038] The structure of the coding layer includes: The input layer is used to receive multispectral image data from the aforementioned "sample single-type land cover multispectral dataset". The number of input channels is consistent with the number of bands in the preset spectral band set. If the preset spectral band set contains five bands: red, green, blue, near-infrared, and shortwave infrared, then the number of input channels is 5. The input image size is set to 256×256 pixels. Convolutional block 1 contains two consecutive convolutional layers. The parameters of each convolutional layer are 32 kernels, 3×3 kernel size, stride 1, and padding 1. Each convolutional layer is followed by a batch normalization layer and a ReLU activation function. The convolutional block is followed by a max pooling layer with a 2×2 kernel size and stride 2, which is used to implement feature downsampling and dimensionality compression. Convolutional block 2 has the same structure as convolutional block 1, but the number of convolutional kernels is increased to 64, while the other parameters remain unchanged. It further extracts the deep spectral-spatial fusion features of ground objects. Convolutional block 3 has an increased number of convolutional kernels to 128, while the other parameters are the same as those of convolutional block 1, which enhances the ability to distinguish features of similar land features. Convolutional block 4 increases the number of convolutional kernels to 256, while the other parameters are the same as those of convolutional block 1, capturing the subtle textures and spectral differences of ground features; The bottleneck layer consists of two convolutional layers with 512 kernels, a kernel size of 3×3, a stride of 1, and padding of 1. It is followed by a batch normalization layer and a ReLU activation function, serving as the core layer for feature extraction and integrating all feature information from the first four convolutional blocks.

[0039] The structure of the decoding layer includes: Deconvolution block 1 contains one "deconvolution layer" with 256 kernels, a kernel size of 2×2, and a stride of 2, which realizes feature upsampling. After upsampling, it is connected to the output feature map of the coding layer convolution block 4 by skip connection. After concatenation, it is processed by two convolution layers, a batch normalization layer, and a ReLU activation function.

[0040] Deconvolution block 2 has 128 convolution kernels and the rest of the structure is the same as deconvolution block 1. It is connected to the output feature map of the coding layer convolution block 3 to restore the medium-scale spatial features of ground objects.

[0041] Deconvolution block 3 has 64 convolution kernels in the deconvolution layer, and the rest of the structure is the same as deconvolution block 1. It is connected to the output feature map of the coding layer convolution block 2 in a skip connection to improve the accuracy of ground feature edge localization.

[0042] Deconvolution block 4 has 32 convolution kernels and the rest of the structure is the same as deconvolution block 1. It is connected to the output feature map of coding layer convolution block 1 to accurately restore the pixel-level location information of ground objects.

[0043] The output layer consists of one convolutional layer with the same number of convolutional kernels as the aforementioned "multiple land cover categories", a kernel size of 1×1, and a stride of 1. It is followed by a Softmax activation function to output the probability that each pixel belongs to a different land cover category.

[0044] For the feature recognition parameter settings, the Adam optimizer was selected, with a learning rate of 0.001, β1 parameter of 0.9, β2 parameter of 0.999, and weight decay coefficient of 0.0001 to prevent overfitting. The loss function adopted was the cross-entropy loss function combined with the Dice loss function, with a weight ratio of 1:1, to ensure the model's recognition accuracy for various features.

[0045] For training the feature identifier, the batch size is set to 16, and the total number of training rounds is set to 100. The training data consists of the aforementioned training set, validation set, and test set. When the feature category recognition accuracy on the validation set remains stable at 95% or higher for 5 consecutive rounds, and the recognition accuracy on the test set reaches 94% or higher, the model is considered to have converged, and the feature identifier is obtained.

[0046] It is important to note that the aforementioned method of extracting statistical features as input to the land cover identifier is one feasible implementation. However, in another implementation, to fully leverage the powerful feature extraction capabilities of deep neural networks, the homogeneous window data itself—that is, a three-dimensional data block with dimensions of "window size × window size × number of bands"—can be used as the direct input to the land cover identifier (U-Net). Accordingly, if the homogeneous window three-dimensional data block is directly input, the land cover identifier will process the multispectral image three-dimensional data block and output the land cover category prediction result for the entire window.

[0047] like Figure 2 As shown, in step S300 of this embodiment, a preset window size sequence is retrieved, and the land cover identifier is used to perform land cover identification and annotation on the multispectral remote sensing image data to obtain the land cover category annotation result of the target area, including: Retrieve a preset window size sequence, wherein the preset window size sequence includes N preset window sizes; Extract a first-level preset window size from the N preset window sizes in descending order of size, and divide the multispectral remote sensing image data into windows based on the first-level preset window size to obtain multiple first-level window data; The land feature identifier is used to annotate the multiple first-level window data to obtain a set of first-level labeled results and first-level unlabeled data. Continue to extract secondary preset window sizes from the N preset window sizes, and divide the primary unlabeled data into windows based on the secondary preset window sizes to obtain multiple secondary window data; The land feature identifier is used to annotate the multiple secondary window data to obtain a set of secondary labeled results and a set of secondary unlabeled data. Repeat the window size extraction, window division, and feature labeling process until the preset window size sequence is processed or the multispectral remote sensing image data is labeled. Based on the labeled result sets at each level, the feature category labeling results for the target area are formed.

[0048] In this embodiment of the application, the purpose of step S300 is to balance the efficiency and accuracy of land cover identification and annotation through a multi-size window layering processing strategy, so as to ensure that all types of land cover in the multispectral remote sensing image can be accurately annotated, and to provide a foundation for the subsequent generation of complete and reliable land cover semantic segmentation results.

[0049] First, a preset window size sequence needs to be retrieved, which includes N preset window sizes. That is, based on the resolution of the remote sensing image of the target area, such as 15 meters / pixel, 30 meters / pixel, etc., and the scale of common land features, such as large areas of farmland and small areas of residential land, a sequence containing N preset window sizes is set. For example, N=4, and the size sequence can be set to 256×256 pixels, 128×128 pixels, 64×64 pixels, and 32×32 pixels, ensuring that the sequence covers the needs from rapid processing of large areas to fine processing of small areas.

[0050] Then, a first-level preset window size needs to be extracted from the N preset window sizes in descending order of size, and the multispectral remote sensing image data is divided into windows based on the first-level preset window size to obtain multiple first-level window data. The dimension of each first-level window data is "first-level preset window size × first-level preset window size × number of spectral bands".

[0051] Specifically, the preset window size sequence needs to be sorted according to the "decreasing size" rule. In the example above, the preset window size sequence sorted according to the "decreasing size" rule is: 256×256 pixels, 128×128 pixels, 64×64 pixels, and 32×32 pixels.

[0052] Then, from the preset window size sequence, the first size, i.e., the largest size, such as 256×256 pixels, is extracted in descending order as the "first-level preset window size". Based on the first-level preset window size, the "multispectral remote sensing image data" obtained in step S100 is divided into sliding windows, dividing the entire image into multiple non-overlapping "first-level window data". For example, a 1024×1024 pixel image can be divided into 16 first-level window data by dividing it into 256×256 pixel windows. The sliding step size of the sliding window is consistent with the window size to avoid duplication or omission.

[0053] Furthermore, the feature identifier is used to annotate the multiple primary window data to obtain a set of primary labeled results and a set of primary unlabeled data.

[0054] In step S300 of this embodiment, the land feature identifier is used to annotate the multiple primary window data to obtain a primary labeled result set and primary unlabeled data, including: Extract the first-level window data from the plurality of first-level window data, and calculate the spectral consistency index of the first-level window data; Determine whether the spectral consistency index meets the preset homogeneity condition. If it does, then determine the first-level window data as homogeneous first-level window data. Extract the statistical features of the homogeneous first-level window data in each spectral band to form a window feature vector; The window feature vector is input into the land cover identifier to obtain the land cover category prediction result; The predicted land cover category results are used as land cover labels for the homogeneous first-level window data, and the homogeneous first-level window data is added to the first-level labeled result set; If the spectral consistency index does not meet the preset homogeneity condition, the first-level window data is added to the first-level unlabeled data. Process the remaining first-level window data sequentially to complete the feature annotation processing of all first-level window data.

[0055] In step S300 of this embodiment, calculating the spectral consistency index of the first-level window data includes: Extract all pixels within the first-level window data and obtain the reflectance value of each pixel in each spectral band; The variance of the reflectance values ​​of all pixels in each spectral band is calculated to obtain the band consistency index corresponding to each spectral band. The spectral consistency index of the first-level window data is obtained based on the band consistency index corresponding to each spectral band.

[0056] In step S300 of this application embodiment, the purpose of the above sub-step is to ensure that the land cover labeling of the first-level window data is accurate and efficient through the combination logic of "spectral consistency screening + accurate feature recognition", while screening out heterogeneous areas that need further refinement, so as to provide clear targets for subsequent small-size window labeling.

[0057] First, all pixels within the first-level window data need to be extracted, and the reflectance value of each pixel in each spectral band needs to be obtained. That is, according to the preset first-level window size, such as 256×256 pixels, corresponding to the spatial dimension of "rows × columns", the spatial region of the first-level window is cropped from the spatial dimension (rows × columns) of the three-dimensional data cube according to the coordinate range. For example, if the spatial coordinate range of the first-level window is "rows: 1-256, columns: 1-256", then all pixels within this range are extracted. For each pixel, its reflectance value in all spectral bands is obtained, thus obtaining a vector containing the complete spectral information of that pixel.

[0058] Then, for each pixel within the aforementioned spatial range, its reflectance values ​​in all spectral bands, such as bands 1-5, are extracted to form the three-dimensional sub-cube data corresponding to the first-level window. The structure is "256 rows × 256 columns × 5 bands," meaning it includes the spatial location and full-band reflectance information of all pixels within the window. In this example, the spatial range comprises 256 × 256 = 65536 pixels, with each pixel's coordinates being "row i, column j," where 1 ≤ i ≤ 256 and 1 ≤ j ≤ 256.

[0059] Furthermore, it is necessary to calculate the variance of the reflectance values ​​of all pixels within each spectral band to obtain the band consistency index corresponding to each spectral band.

[0060] First, it is necessary to traverse each band of the first-level window's 3D sub-cube, such as bands 1 to 5. For a single band, taking band 1 as an example, extract the reflectance values ​​of all pixels in that band to form a two-dimensional reflectance matrix. The two-dimensional reflectance matrix has a structure of "256 rows × 256 columns". The value in "row i, column j" of the matrix corresponds to the reflectance of the pixel in "row i, column j" within the window in band 1. Repeat the above operation to generate a corresponding two-dimensional reflectance matrix for each band, ultimately obtaining multiple reflectance matrices consistent with the number of bands. For example, 5 bands correspond to 5 "256 × 256" matrices.

[0061] Next, for the reflectivity matrix of a single band, such as the "256×256" matrix of band 1, the reflectivity values ​​of all pixels in the matrix are extracted into a one-dimensional array, with a total of 256×256=65536 values.

[0062] Calculate the variance of the one-dimensional array, where the variance = (sum of squared differences between each value and the array mean ÷ total number of values). This variance is the "band consistency index" corresponding to the band. The smaller the variance, the more uniform the reflectivity of the window pixels within the band and the more uniform the ground features.

[0063] Following the above method, the band consistency index for all bands is calculated sequentially to obtain the variance values ​​corresponding to the five bands.

[0064] Then, the band consistency index for all bands, namely the variance values ​​corresponding to the above 5 bands, is calculated comprehensively. For example, the average value of all variances can be taken to obtain the "spectral consistency index" of the first-level window data. This index comprehensively reflects the uniformity of ground features in all bands within the window.

[0065] Furthermore, it is necessary to determine whether the spectral consistency index meets a preset homogeneity condition. If it does, the first-level window data is determined as homogeneous first-level window data. That is, the calculated spectral consistency index is compared with a preset spectral consistency threshold to determine whether the preset homogeneity condition is met. For example, the preset threshold can be an average variance ≤ 0.05. The preset spectral consistency threshold can be adjusted according to the annotation accuracy requirements in actual applications.

[0066] Furthermore, if the spectral consistency index meets the preset homogeneity condition, then for each band, such as bands 1-5, the statistical characteristics of the reflectance matrix of that band are calculated, such as the mean, median, and standard deviation. Three features are extracted for each band, and a total of 15 features are extracted for the five bands. These features are arranged in the order of "band 1 mean, band 1 median, band 1 standard deviation, band 2 mean... band 5 standard deviation" to form a 15-dimensional "window feature vector".

[0067] Then, the feature vector of the window is input into the land cover identifier constructed by S200, and the "land cover category prediction result" corresponding to the window is output, such as "vegetation-grassland" etc.

[0068] Furthermore, the predicted land cover category results are bound to the spatial coordinate range of the first-level window, such as "row 1-256, column 1-256", as the land cover label for that window. At the same time, the 3D sub-cube data, spatial coordinates, and label categories of that window are added to the "first-level labeled result set".

[0069] If the spectral consistency index does not meet the preset homogeneity condition, the data from the first-level window is added to the first-level unlabeled data. That is, the 3D sub-cube data and its spatial coordinate range, such as "rows 1-256, columns 1-256", from the first-level window are directly added to the "first-level unlabeled data", and reserved for subsequent secondary cropping and processing of the 3D sub-cube using a smaller second-level window, such as 128×128 pixels.

[0070] Finally, the remaining first-level window data needs to be processed sequentially to complete the feature annotation processing for all first-level window data. That is, for the remaining first-level window data, the same process of 3D sub-cube extraction, spectral consistency index calculation, homogeneity judgment, and classification annotation is performed until all first-level windows are processed, ultimately forming a "first-level labeled result set" containing the annotation information of all homogeneous windows and a "first-level unlabeled data" containing the data of all heterogeneous windows.

[0071] Furthermore, it is necessary to extract secondary preset window sizes from the N preset window sizes, and to divide the primary unlabeled data into windows based on the secondary preset window sizes to obtain multiple secondary window data. That is, from the aforementioned preset window size sequence, such as "256×256, 128×128, 64×64, 32×32", the second size after the primary size, such as 128×128 pixels, is extracted in descending order of size as the "secondary preset window size".

[0072] Then, based on the preset size of the secondary window, the spatial range of the aforementioned unlabeled primary data is divided into non-overlapping, fully covered sliding windows, where the sliding step size equals the window size. For example, a 256×256 pixel unlabeled primary window can be divided into 128×128 pixel windows, resulting in four secondary windows with spatial coordinates of "row 1-128, column 1-128", "row 1-128, column 129-256", "row 129-256, column 1-128", and "row 129-256, column 129-256". For each secondary window, its corresponding 3D data cube sub-block is extracted, such as a 128×128×5 band, forming "multiple secondary window data".

[0073] Furthermore, it is necessary to fully adopt the labeling logic of the first-level window and combine the feature identifier and spectral consistency to label the multiple second-level window data.

[0074] Specifically, for a single secondary window of data, such as a 128×128×5 band sub-block, the pixel reflectance of each band is extracted, and the variance of each band is calculated as a band consistency index. These are then combined to obtain the spectral consistency index of the window, i.e., the aforementioned mean variance. If the spectral consistency index of the window meets a preset threshold, such as ≤0.05, it is determined to be a homogeneous window. Its statistical features for each band are extracted to form a feature vector, which is input into the land cover identifier to obtain the category prediction result. The window is then classified into the "secondary labeled result set". If the spectral consistency index of the window does not meet the threshold, it is determined to be a heterogeneous window and is directly classified into the "secondary unlabeled data".

[0075] Repeat the window size extraction, window division, and feature labeling process until the preset window size sequence is processed or the multispectral remote sensing image data is labeled. Based on the labeled result sets at each level, the feature category labeling results for the target area are formed.

[0076] In step S400 of this embodiment, obtaining the semantic segmentation result of the target area based on the land feature category labeling result includes: Extract the labeled result sets at each level from the land feature category labeling results, and obtain the spatial location information and land feature category identifier of each labeled window; Based on the spatial location information, the corresponding land cover category identifier is mapped to the corresponding spatial location of the multispectral remote sensing image data to obtain a land cover distribution map of the target area. The land cover distribution map records the land cover category identifier corresponding to each spatial location. The aforementioned land cover distribution map is output as the semantic segmentation result of the land cover in the target area.

[0077] In this embodiment of the application, the purpose of step S400 is to transform discrete multi-scale window annotation information into a continuous and complete spatial distribution map of ground features by integrating spatial coordinate mapping and results, thereby achieving pixel-level accurate classification of ground feature categories in the target area and providing intuitive and accurate basic data for subsequent applications.

[0078] First, it is necessary to extract the labeled result sets at each level from the land feature category labeling results, and obtain the spatial location information and land feature category identifier of each labeled window. That is, to collect all the "labeled result sets" generated in step S300, including the first-level, second-level... Nth-level labeled result sets.

[0079] Then, for each labeled window, the spatial location information and land cover category identifier of each labeled window are extracted from the labeled result set. The spatial location information is the specific coordinate range of the labeled window in the multispectral remote sensing image, such as "row a1-a2, column b1-b2", corresponding to the pixel range of the spatial dimension. The land cover category identifier is the land cover category of the window output by the land cover identifier, such as "farmland", "water body", "building", etc. It is ensured that the spatial location and category identifier of each labeled window are bound one-to-one, providing a data foundation for subsequent mapping.

[0080] Furthermore, it is necessary to map the corresponding land cover category identifier to the corresponding spatial location in the multispectral remote sensing image data based on the spatial location information to obtain a land cover distribution map of the target area. The land cover distribution map records the land cover category identifier corresponding to each spatial location.

[0081] First, based on the original spatial coordinate system (row × column) of the multispectral remote sensing image, it is necessary to associate the "spatial location information" of each labeled window, such as "row a1-a2, column b1-b2", with the pixel coordinates of the original image to clarify the location of all pixels covered by the window.

[0082] Then, the "feature category identifier" of the labeled window is assigned to all pixels covered by that window. For example, if a secondary labeled window has a spatial location of "row 1-128, column 1-128" and a category identifier of "grassland", then all pixels within the range of "row 1-128, column 1-128" in the original image will be labeled with the feature category of "grassland". If there is spatial overlap between windows of different levels, such as a large window partially overlapping a small window, the "small window priority" principle is followed—the feature category identifier of the smaller window is used to ensure the accuracy of the mapping results.

[0083] Finally, through the above mapping process, each pixel in the original multispectral remote sensing image—that is, the intersection of all rows and columns in the spatial dimension—is assigned a unique land cover category identifier, forming a "land cover distribution map." The structure of this map is consistent with the spatial dimension of the original image, such as 1000 rows × 1000 columns. The value of each pixel is its corresponding land cover category identifier, which can be represented by encoding, such as 1 = farmland, 2 = water body, etc., completely recording the land cover categories of all spatial locations within the target area.

[0084] Next, the land cover distribution map is output as the semantic segmentation result of the target area. The output format can be a visual image, that is, pixels are assigned different colors according to land cover categories to intuitively show the spatial distribution of land covers.

[0085] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0086] Those skilled in the art will understand that embodiments of the present invention can provide methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0087] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0088] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0089] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0090] Although preferred embodiments of the invention have been described, those skilled in the art, once they have learned the basic inventive concept, can make other changes and modifications to these embodiments.

[0091] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of this invention and its equivalents, this invention also intends to include these modifications and variations.

Claims

1. A semantic segmentation method for remote sensing images based on a deep learning model, characterized in that, The method includes: Acquire multispectral remote sensing image data of the target area, wherein the multispectral remote sensing image data contains image information of multiple spectral bands; Construct a land cover identifier, which is used to identify land cover categories in multispectral remote sensing image data; A preset window size sequence is retrieved, and the land cover identifier is used to perform land cover identification and annotation on the multispectral remote sensing image data to obtain the land cover category annotation results for the target area, including: Retrieve a preset window size sequence, wherein the preset window size sequence includes N preset window sizes; Extract a first-level preset window size from the N preset window sizes in descending order of size, and divide the multispectral remote sensing image data into windows based on the first-level preset window size to obtain multiple first-level window data; The land feature identifier is used to annotate the multiple first-level window data to obtain a set of first-level labeled results and first-level unlabeled data. Continue to extract secondary preset window sizes from the N preset window sizes, and divide the primary unlabeled data into windows based on the secondary preset window sizes to obtain multiple secondary window data; The land feature identifier is used to annotate the multiple secondary window data to obtain a set of secondary labeled results and a set of secondary unlabeled data. Repeat the window size extraction, window division and land cover labeling process until the preset window size sequence is processed or the multispectral remote sensing image data is labeled. Based on the labeled result set at each level, the land cover category labeling result of the target area is formed. The semantic segmentation results of the target area are obtained based on the land feature category labeling results.

2. The method according to claim 1, characterized in that, Acquire multispectral remote sensing image data of the target area, including: A preset spectral band set is obtained, and remote sensing images of the target area are acquired based on the preset spectral band set to obtain multiple original spectral images of the target area. The multiple original spectral images are preprocessed to obtain the multispectral remote sensing image data.

3. The method according to claim 2, characterized in that, The multiple original spectral images are preprocessed to obtain the multispectral remote sensing image data, including: Radiometric correction is performed on the multiple original spectral images to obtain multiple first corrected spectral images; Geometric correction is performed on the plurality of first corrected spectral images to obtain a plurality of second corrected spectral images; The multiple second-corrected spectral images are registered to ensure that the pixel positions of each spectral band are aligned, resulting in multiple registered spectral images. The multiple registered spectral images are normalized to obtain the multispectral remote sensing image data.

4. The method according to claim 1, characterized in that, Constructing a feature identifier includes: Multiple land cover categories are obtained, and multispectral remote sensing image data retrieval is performed for each land cover category to construct a multispectral dataset of a single land cover category and a label set of the land cover category. The sample single-class land cover multispectral dataset is used as training input, and the sample land cover category label set is used as training labels to train a deep neural network to generate the land cover identifier.

5. The method according to claim 1, characterized in that, The land feature identifier is used to annotate the multiple primary window data to obtain a primary labeled result set and primary unlabeled data, including: Extract the first-level window data from the plurality of first-level window data, and calculate the spectral consistency index of the first-level window data; Determine whether the spectral consistency index meets the preset homogeneity condition. If it does, then determine the first-level window data as homogeneous first-level window data. Extract the statistical features of the homogeneous first-level window data in each spectral band to form a window feature vector; The window feature vector is input into the land cover identifier to obtain the land cover category prediction result; The predicted land cover category results are used as land cover labels for the homogeneous first-level window data, and the homogeneous first-level window data is added to the first-level labeled result set; If the spectral consistency index does not meet the preset homogeneity condition, the first-level window data is added to the first-level unlabeled data. Process the remaining first-level window data sequentially to complete the feature annotation processing of all first-level window data.

6. The method according to claim 5, characterized in that, Calculate the spectral consistency index of the first-level window data, including: Extract all pixels within the first-level window data and obtain the reflectance value of each pixel in each spectral band; The variance of the reflectance values ​​of all pixels in each spectral band is calculated to obtain the band consistency index corresponding to each spectral band. The spectral consistency index of the first-level window data is obtained based on the band consistency index corresponding to each spectral band.

7. The method according to claim 1, characterized in that, Based on the land cover category labeling results, the semantic segmentation results of the target area are obtained, including: Extract the labeled result sets at each level from the land feature category labeling results, and obtain the spatial location information and land feature category identifier of each labeled window; Based on the spatial location information, the corresponding land cover category identifier is mapped to the corresponding spatial location of the multispectral remote sensing image data to obtain a land cover distribution map of the target area. The land cover distribution map records the land cover category identifier corresponding to each spatial location. The aforementioned land cover distribution map is output as the semantic segmentation result of the land cover in the target area.

Images