A method and device for classifying bare land in remote sensing images

Abstract

The embodiment of the application discloses a remote sensing image bare land classification method and device, the method comprises the following steps: obtaining a remote sensing image to be classified, the size of the remote sensing image to be classified is 256*256 pixels; performing feature extraction processing on the remote sensing image to be classified to obtain a first feature image and a second feature image with different levels; performing a hole convolution processing on the first feature image to obtain a deep feature image of the remote sensing image to be classified; performing splicing processing on the second feature image and the deep feature image to obtain a spliced feature image; and according to the spliced feature image, classifying the pixel points in the remote sensing image to be classified to obtain the classification result of each pixel point in the remote sensing image to be classified, wherein the classification result at least includes a bare land or a background category.

Description

TECHNICAL FIELD

[0001] The present application relates to, but is not limited to, the technical field of remote sensing image processing, and in particular to a remote sensing image bare land classification method and device. BACKGROUND

[0002] With the high-speed growth of population and economy, especially the acceleration of urbanization, unreasonable land use methods have caused serious waste of land resources and pollution of the ecological environment. Bare land is an important part of land cover types and has an important impact on the urban environment. On the one hand, the existence of a large number of bare lands leads to low land use efficiency and waste of land resources; on the other hand, bare land lacks vegetation protection and is prone to cause ecological problems such as dust raising and soil erosion.

[0003] Therefore, it is particularly important to quickly and accurately locate bare land areas for fine management of the urban environment and optimal allocation of land resources. SUMMARY

[0004] Based on the problems in the related art, the embodiments of the present application provide a remote sensing image bare land classification method and device.

[0005] The technical scheme of the embodiments of the present application is implemented as follows:

[0006] The embodiments of the present application provide a remote sensing image bare land classification method, which comprises:

[0007] Obtaining a to-be-classified remote sensing image, the size of the to-be-classified remote sensing image being 256x256 pixels;

[0008] Performing feature extraction processing on the to-be-classified remote sensing image to obtain first feature images and second feature images with different levels;

[0009] Performing hole convolution processing on the first feature images to obtain deep feature images of the to-be-classified remote sensing image;

[0010] Performing splicing processing on the second feature images and the deep feature images to obtain spliced feature images;

[0011] According to the spliced feature images, classifying all pixel points in the to-be-classified remote sensing image to obtain a classification result of each pixel point in the to-be-classified remote sensing image, the classification result at least including a bare land or a background category.

[0012] In some embodiments, the feature extraction processing on the to-be-classified remote sensing image to obtain first feature images and second feature images with different levels comprises:

[0013] The channel attention and spatial attention mechanism is used to perform attention feature extraction on the remote sensing image to be classified to obtain an attention feature image.

[0014] The feature extraction network is used to perform feature extraction on the attention feature image to obtain a feature image.

[0015] The channel attention and spatial attention mechanism is used to perform attention feature extraction on the feature image to obtain the first feature image and the second feature image with different levels.

[0016] In some embodiments, the channel attention and spatial attention mechanism is used to perform attention feature extraction on the remote sensing image to be classified to obtain an attention feature image, including:

[0017] The channel attention module is used to perform maximum pooling processing and average pooling processing on the remote sensing image to be classified respectively to obtain a channel maximum pooling feature image and a channel average pooling feature image.

[0018] The shared fully connected layer is used to perform feature extraction on the channel maximum pooling feature image and the channel average pooling feature image respectively to obtain a third feature image and a fourth feature image.

[0019] The third feature image and the fourth feature image are sequentially subjected to element addition processing and activation processing to obtain an initial channel feature image.

[0020] The initial channel feature image and the remote sensing image to be classified are subjected to element multiplication processing to obtain a channel feature image.

[0021] The spatial attention module is used to perform maximum pooling processing and average pooling processing on the channel feature image respectively to obtain a spatial maximum pooling feature image and a spatial average pooling feature image.

[0022] The spatial maximum pooling feature image and the spatial average pooling feature image are subjected to splicing processing to obtain a spatial splicing feature image.

[0023] The spatial splicing feature image is sequentially subjected to dimension reduction processing and activation processing to obtain a spatial feature image.

[0024] The spatial feature image and the channel feature image are subjected to element multiplication processing to obtain an attention feature image.

[0025] In some embodiments, the feature extraction network is used to perform feature extraction on the attention feature image to obtain a feature image, including:

[0026] The attention feature image is subjected to dimension increasing processing to obtain a dimension-increased feature image.

[0027] The up-sampling feature image is down-sampled by a deep convolutional layer to obtain a down-sampled feature image.

[0028] The down-sampled image is reduced in dimension by a point-wise convolutional layer to obtain the feature image.

[0029] In some embodiments, the first feature image is processed by the hole convolution to obtain the deep feature image of the remote sensing image to be classified, including:

[0030] According to the hole spatial pyramid pooling, the first feature image is processed by parallel hole convolution to obtain five hole convolution feature images.

[0031] The five hole convolution feature images are processed by point-wise convolution to obtain the deep feature image of the remote sensing image to be classified.

[0032] In some embodiments, according to the spliced feature image, all pixel points in the remote sensing image to be classified are classified to obtain the classification result of each pixel point in the remote sensing image to be classified, including:

[0033] The spliced feature image is processed by convolution with a convolution kernel of a preset size to obtain a convolution feature image.

[0034] The convolution feature image is processed by up-sampling to obtain the classification result of each pixel point in the remote sensing image to be classified.

[0035] In some embodiments, the method further includes:

[0036] A remote sensing image is obtained, the remote sensing image including at least two image blocks of 256x256 pixels;

[0037] According to a preset image size, the remote sensing image is cropped to obtain at least two remote sensing images to be classified and position information of each remote sensing image to be classified on the remote sensing image;

[0038] Correspondingly, after obtaining the classification result of each pixel point in the remote sensing image to be classified, the method further includes:

[0039] According to the position information of each remote sensing image to be classified on the remote sensing image, the classified remote sensing image to be classified is spliced to obtain a classification image.

[0040] According to geographic information of the remote sensing image, the classification image is reconstructed in geographic information to obtain a remote sensing classification image.

[0041] According to the classification result of each pixel point in the remote sensing classification image, a bare land classification image corresponding to the remote sensing classification image is determined.

[0042] In some embodiments, the method further comprises:

[0043] At least one thousand sample remote sensing images including bare land are obtained;

[0044] The bare land and non-bare land in the sample remote sensing images are labeled to obtain labeled sample remote sensing images;

[0045] The labeled sample remote sensing images are determined as a sample data set.

[0046] In some embodiments, the method is implemented through a bare land semantic segmentation model, which at least includes a feature extraction module, a dilated convolution module, a splicing module, and a classification module; the bare land semantic segmentation model is obtained through the following steps:

[0047] The sample data set is input into the bare land semantic segmentation model, the sample data set including a sample image, the size of the sample image being 256x256 pixels;

[0048] The feature extraction module is used to perform feature extraction processing on the sample image to obtain first feature sample images and second feature sample images with different levels;

[0049] The dilated convolution module is used to perform dilated convolution processing on the first feature sample images to obtain deep feature sample images of the sample image;

[0050] The splicing module is used to splice the second feature sample images and the semantic feature sample images to obtain spliced feature sample images;

[0051] The classification module is used to classify all pixel points in the sample image according to the spliced feature sample images to obtain sample classification results of each pixel point in the sample image, the sample classification results at least including bare land or background categories;

[0052] The sample classification results are input into a preset loss model to obtain a loss result;

[0053] According to the loss result, parameters in the feature extraction module, the dilated convolution module, the splicing module, and the classification module are corrected to obtain a trained bare land semantic segmentation model.

[0054] Embodiments of the present application provide a bare land classification device, which comprises:

[0055] An acquisition module is configured to acquire a remote sensing image to be classified, wherein the remote sensing image to be classified has a size of 256*256 pixels.

[0056] A feature extraction module is configured to perform feature extraction processing on the remote sensing image to be classified to obtain first feature images and second feature images having different levels.

[0057] A dilated convolution module is configured to perform dilated convolution processing on the first feature images to obtain deep feature images of the remote sensing image to be classified.

[0058] A splicing module is configured to perform splicing processing on the second feature images and the deep feature images to obtain spliced feature images.

[0059] A classification module is configured to perform classification on all pixel points in the remote sensing image to be classified according to the spliced feature images to obtain a classification result of each pixel point in the remote sensing image to be classified, wherein the classification result at least includes a bare land category or a background category.

[0060] An embodiment of the present application provides a remote sensing image bare land classification device, which comprises a processor and a memory, wherein the memory stores a computer program capable of running on the processor, and the processor implements the remote sensing image bare land classification method when executing the computer program.

[0061] An embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the remote sensing image bare land classification method.

[0062] An embodiment of the present application provides a computer program product, which comprises executable instructions stored in a computer readable storage medium, and when a processor of a bare land classification device reads the executable instructions from the computer readable storage medium and executes the executable instructions, the remote sensing image bare land classification method is implemented.

[0063] The bare land classification method and device for remote sensing images provided by the embodiments of the present application obtain a to-be-classified remote sensing image with a size of 256*256 pixels, perform feature extraction processing on the to-be-classified remote sensing image, obtain first feature images and second feature images with different levels, perform hole convolution processing on the first feature images, and obtain deep feature images of the to-be-classified remote sensing image; perform splicing processing on the first feature images and the deep feature images, obtain spliced feature images, classify the pixel points in the to-be-classified remote sensing image according to the spliced feature images, and obtain the classification results of each pixel point in the to-be-classified remote sensing image. In this way, the embodiments of the present application obtain feature images with different levels by performing feature extraction on the to-be-classified remote sensing image, and take into account the boundary information and deep semantic information of the to-be-classified remote sensing image, so that the classification can be performed through richer features when the bare land is classified, and the accuracy of the classification results is improved. Meanwhile, the embodiments of the present application realize pixel-level classification of the to-be-classified remote sensing image, so that the classification results can fully meet the demand for monitoring bare land in fine management of urban environment.

[0064] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, rather than limiting the present application. BRIEF DESCRIPTION OF DRAWINGS

[0065] The drawings incorporated into the specification and forming part of the specification, show embodiments consistent with the present application, and together with the specification, serve to explain the technical solutions of the present application.

[0066] Figure 1 is an implementation flow diagram of a bare land classification method for remote sensing images provided by the embodiments of the present application;

[0067] Figure 2 is an implementation flow diagram of a bare land classification method for remote sensing images provided by the embodiments of the present application;

[0068] Figure 3 is an implementation flow diagram of a bare land classification method for remote sensing images provided by the embodiments of the present application;

[0069] Figure 4 is a Deeplabv3+_M_CBAM model structure diagram provided by the embodiments of the present application;

[0070] Figure 5 is an M_CBAM model structure diagram provided by the embodiments of the present application;

[0071] Figure 6 is a CBAM model structure diagram provided by the embodiments of the present application;

[0072] Figure 7is a variation curve of average intersection over union precision in a three-model training process provided by an embodiment of the present application.

[0073] Figure 8 is a bare land classification result schematic diagram provided by an embodiment of the present application.

[0074] Figure 9 is a remote sensing image bare land classification device schematic diagram provided by an embodiment of the present application.

[0075] Figure 10 is a hardware entity schematic diagram of a remote sensing image bare land classification device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0076] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings, and the described embodiments should not be regarded as limiting the present application, and all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0077] In the following description, “some embodiments” are described, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0078] In the following description, the terms “first\second\third” are only to distinguish similar objects, and do not represent a specific order of the objects, and it can be understood that “first\second\third” can be interchanged in a specific order or sequence as allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0079] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0080] In domestic and foreign research, bare land is defined as “land with soil surface and basically no vegetation coverage”, and remote sensing technology provides effective technical support for monitoring large-scale bare land due to its advantages of large range, macroscopic and dynamic.

[0081] The remote sensing classification research in the related art includes two categories. The first category is land cover mapping based on medium and low resolution images and sampling supervised classification method. For example, according to 1km (kilometer) data of a medium resolution imaging spectrometer (MODIS), a global land cover product by national mapping organizations (GLCNMO) is produced, in which bare land includes gravel, rock and sand; or based on a deep learning method, for example, an annual land cover product is produced by using 500m (meter) data of MODIS, in which bare land includes soil, sand, rock or snow exposed land. The second category is a bare soil index extracted only for bare land. For example, a bare soil index model is constructed by using multispectral data, for example, a bare soil index model is constructed by using 30m multispectral data of a satellite imaging system (Landsat TM), and the extraction accuracy is between 92.16% and 95.95%. It can be seen that in the first category of research, the classification granularity is coarse, and bare rock, saline-alkali land, sandy land and the like are classified as bare land. In addition, the existing land use / cover public data set cannot meet the needs of fine management of urban environment in terms of data accuracy and update cycle. The bare soil index in the second category of research is mostly constructed depending on short-wave infrared and medium infrared bands, which is not suitable for the extraction of bare land in the current sub-meter high-resolution remote sensing image.

[0082] The related art uses high-resolution remote sensing images for bare land classification. Although the details of the ground objects in the remote sensing image are clearly reflected, the extraction of bare land also causes various interferences, resulting in great difficulty in extracting bare land. First, most of the bare land itself has no fixed geometric shape, varies in size, and has a broken boundary. Second, there are some small bare land patches in the managed areas under the management methods such as covering, planting trees and grass, and the reappearance of bare land due to seasonal changes in the grass-planting area, the exposure of bare land due to the damage or poor covering of the covering net, and the like, forming a complex background. Third, buildings and bare land have similar spectral characteristics on high-resolution images. Therefore, for management needs, the small amount of bare land in the managed area is not the object of concern for bare land, but background information, and therefore needs to be excluded.

[0083] The remote sensing image analysis technology in the related art cannot meet the application needs of bare land monitoring for environmental management, and currently no related reports have been found on the bare land extraction research and application of such large-scale high-resolution remote sensing images. Therefore, it is still a technical problem in practical applications to separate bare land from other land cover types on high-resolution remote sensing images with rich semantic information, especially with the increase of image resolution and the dramatic increase of data volume, the reliability and efficiency of remote sensing information extraction are also facing severe challenges.

[0084] Based on the problems in the related art, the embodiment of the present application provides a remote sensing image bare land classification method, by acquiring a to-be-classified remote sensing image with a size of 256x256 pixels, performing feature extraction processing on the to-be-classified remote sensing image, obtaining first feature images and second feature images with different levels, performing hole convolution processing on the first feature images to obtain deep feature images of the to-be-classified remote sensing image; performing splicing processing on the first feature images and the deep feature images to obtain spliced feature images, and classifying at least two pixel points in the to-be-classified remote sensing image according to the spliced feature images to obtain the classification result of each pixel point in the to-be-classified remote sensing image. In this way, the embodiment of the present application obtains feature images with different levels by performing feature extraction on the to-be-classified remote sensing image, and takes into account the boundary information and deep semantic information of the to-be-classified remote sensing image, so that when classifying the bare land, the classification can be performed through more abundant features, and the accuracy of the classification result is improved; at the same time, the embodiment of the present application realizes pixel-level classification of the to-be-classified remote sensing image, so that the classification result can fully meet the demand for monitoring the bare land in the fine management of the urban environment.

[0085] The remote sensing image bare land classification method provided by the embodiment of the present application can be executed by an electronic device such as a bare land classification device, wherein the electronic device can be a notebook computer, a tablet computer, a desktop computer, and the like, or can be implemented as a server. The server can be a stand-alone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud storage, network services, cloud communication, middleware services, security services, and basic cloud computing services such as big data and artificial intelligence platforms.

[0086] In the following, an exemplary application of the bare land classification device implemented as a server will be described, and the technical solutions in the embodiment of the present application will be described clearly and completely with reference to the drawings in the embodiment of the present application.

[0087] Figure 1 is a flowchart of an implementation process of a remote sensing image bare land classification method provided by the embodiment of the present application, as shown in Figure 1 , the method is implemented by steps S101 to S105:

[0088] Step S101, acquiring a to-be-classified remote sensing image, the size of the to-be-classified remote sensing image is 256x256 pixels.

[0089] In some embodiments, due to the large geographical range, large number of pixels and high resolution of the remote sensing image, a large amount of calculation is required for the server to perform feature extraction using a deep learning model, which greatly reduces the calculation efficiency. Therefore, in the embodiment of the present application, a cropping operation is performed on the remote sensing image before feature extraction to obtain a plurality of standard-sized remote sensing images to be classified, and then feature extraction is performed on the plurality of remote sensing images to be classified, which can improve the calculation efficiency and speed up the feature extraction rate.

[0090] In some embodiments, the remote sensing image can be a multispectral high-resolution remote sensing image or other image after system radiation correction and system geometric correction.

[0091] In the embodiment of the present application, the remote sensing image can be obtained first, and the remote sensing image can be cropped according to a preset image size to obtain at least two remote sensing images to be classified and position information of each remote sensing image to be classified on the remote sensing image. Here, the preset image size can be the size of the image required for input during feature classification, for example, the preset image size is 256x256 pixels. The remote sensing image can be cropped according to the preset image size to obtain at least two remote sensing images to be classified and position information of each remote sensing image to be classified on the remote sensing image. The size of each remote sensing image to be classified can be 256x256 pixels; the position information can be the row and column information of the remote sensing image to be classified on the remote sensing image, for example, the remote sensing image is cropped into ten rows and ten columns according to the preset image size, and the position information of the remote sensing image to be classified can be the third row and the fourth column.

[0092] In the embodiment of the present application, after classifying each remote sensing image to be classified, the classified images can be spliced according to the position information of each remote sensing image to be classified to obtain a classified remote sensing image.

[0093] Step S102, performing feature extraction processing on the remote sensing image to be classified to obtain first feature images and second feature images with different levels.

[0094] Here, the MobileNet V2 can be used to perform feature extraction processing on the remote sensing image to be classified. MobileNet V2 is a lightweight feature extraction model that can reduce model parameters and avoid model overfitting. At the same time, to exclude the interference of non-interest areas such as dry grassland, bare soil under the forest and buildings, the Convolutional Block Attention Module (CBAM) module is added to the starting layer and the last layer of MobileNet V2 respectively, and MobileNet V2 network is optimized, so that the texture and other detail features of bare land are enhanced, and the constraint on the spatial features and channel features of bare land is increased in the generation stage of the first feature image and the second feature image.

[0095] In the embodiment of the present application, after the CBAM+Mobilenet V2 is used for feature extraction processing of the remote sensing image to be classified, first feature images and second feature images with different levels can be obtained. The first feature images have deep features and are used to provide semantic information of the remote sensing image to be classified. The second feature images have shallow features and are used to provide boundary information of the remote sensing image to be classified.

[0096] The different level feature images obtained by the embodiment of the present application for feature extraction of the remote sensing image to be classified take into account the boundary information and semantic information of the remote sensing image to be classified, so that more features are provided when bare land classification is performed, and the accuracy of the classification result is improved.

[0097] In step S103, the first feature image is subjected to a dilated convolution processing to obtain a deep feature image of the remote sensing image to be classified.

[0098] In the embodiment of the present application, by performing dilated convolution processing on the first feature image, convolution layers with different dilation coefficients are provided, and dilated convolution features of different scales in the first feature image can be extracted to obtain a deep feature image with semantic features of the remote sensing image to be classified.

[0099] In the embodiment of the present application, step S103 can be implemented by step S1031 and step S1032.

[0100] In step S1031, the first feature image is subjected to parallel dilated convolution processing according to Atrous Spatial Pyramid Pooling to obtain five dilated convolution feature images.

[0101] In some embodiments, by using Atrous Spatial Pyramid Pooling (ASPP), the first feature image is subjected to parallel dilated convolution processing with different dilation coefficients and the same convolution kernel size, so that receptive fields of different scales can be obtained and multi-scale information can be extracted.

[0102] In the embodiment of the present application, the parallel dilated convolution processing of the first feature image can at least include dimension reduction of the first feature image by a 1x1 convolution layer, superposition of corresponding dilated convolution layers with different dilation factors (atrous rates) to extract features of the first feature image at different scales, for example, superposition of corresponding 3x3 dilated convolution layers with given dilation factors of 6, 12 and 18 to extract feature images at different scales, and pooling processing of the first feature image by a dilated pooling layer to obtain a pooled image.

[0103] Here, the first feature image is respectively subjected to a 1x1 convolution layer, a three-given-dilation-factor dilated convolution layer and a pooling processing, and five dilated convolution feature images can be obtained.

[0104] The number of convolution layers with different dilation factors in the dilated spatial convolution pyramid pooling can be determined according to requirements in the embodiments of the present application, so that different numbers of dilated convolution feature images are obtained.

[0105] In step S1032, the five dilated convolution feature images are subjected to point-by-point convolution processing, and the deep feature image of the remote sensing image to be classified is obtained.

[0106] In some embodiments, after obtaining five dilated convolution feature images through the dilated spatial convolution pyramid pooling, at least two dilated convolution feature images can be subjected to feature superposition, and a superposition feature image with an input channel being 5 times the input channel of the first feature image can be obtained. The superposition feature image is subjected to point-by-point convolution processing (which can be 1x1 convolution), and the superposition feature image is reduced in dimension, so that the deep feature image of the remote sensing image to be classified is obtained.

[0107] In step S104, the second feature image and the deep feature image are subjected to splicing processing, and a spliced feature image is obtained.

[0108] In the embodiments of the present application, the deep feature image can be subjected to bilinear up-sampling by 4 times, and a first spliced image is obtained; the second feature image can be subjected to channel reduction processing through a 1x1 convolution layer, and a second spliced image with the same number of channels as the first spliced image is obtained. The first spliced image and the second spliced image are spliced to obtain a feature spliced image.

[0109] In step S105, all pixel points in the remote sensing image to be classified are classified according to the spliced feature image, and a classification result of each pixel point in the remote sensing image to be classified is obtained, and the classification result at least includes a bare land or background category.

[0110] In some embodiments, the spliced feature image can be subjected to a 3x3 convolution to refine the features in the spliced feature image, and finally bilinear up-sampling by 4 times is performed to obtain the classification result of each pixel point in the remote sensing image to be classified, and the classification result includes a bare land and a background category.

[0111] In the embodiments of the present application, the classification of at least two pixel points in the remote sensing image to be classified according to the spliced feature image can be implemented through step S1051 and step S1052.

[0112] In step S1051, the spliced feature image is subjected to convolution processing through a convolution kernel with a preset size, and a convolution feature image is obtained.

[0113] In the embodiment of the present application, the spliced feature image can be subjected to convolution processing by a preset size of convolution kernel (for example, a 3x3 convolution kernel), to refine the features of the spliced feature image, and obtain a convolution feature image.

[0114] Step S1052, performing up-sampling processing on the convolution feature image to obtain the classification result of each pixel point in the to-be-classified remote sensing image.

[0115] In some embodiments, after obtaining the classification result of each pixel point in the to-be-classified remote sensing image, the embodiment of the present application can further have steps S1 to S3:

[0116] Step S1, splicing the classified to-be-classified remote sensing image according to the position information of each to-be-classified remote sensing image on the remote sensing image, to obtain a classification image.

[0117] In some embodiments, all classified to-be-classified remote sensing images can be spliced according to the row and column information of each to-be-classified remote sensing image on the remote sensing image, to obtain a classification image. The classification image has the same size as the remote sensing image, and the classification image shows the position and area of the bare land on the remote sensing image.

[0118] Step S2, reconstructing the geographic information of the classification image according to the geographic information of the remote sensing image, to obtain a remote sensing classification image.

[0119] In some embodiments, the geographic information of the remote sensing image can at least include geographic information such as house buildings, roads, forests, and lakes. The geographic information of the corresponding pixel point in the classification image is reconstructed according to the geographic information of each pixel point in the remote sensing image, to obtain a remote sensing classification image having both geographic information and classification information.

[0120] Step S3, determining the bare land classification image corresponding to the remote sensing classification image according to the classification result of each pixel point in the remote sensing classification image.

[0121] Since the remote sensing classification image has the classification result of each pixel point, the classification result at least includes the bare land or background category, so the bare land classification image corresponding to the remote sensing classification image can be determined intuitively, and then the bare land grid extraction result is generated. The different level feature images obtained by the embodiment of the present application by performing feature extraction on the to-be-classified remote sensing image take into account the boundary information and semantic information of the to-be-classified remote sensing image, so when classifying the bare land, the classification can be performed by using more abundant features, to improve the accuracy of the classification result. Meanwhile, the embodiment of the present application realizes pixel-level classification of the to-be-classified remote sensing image, so that the classification result can fully meet the demand for monitoring the bare land in the fine management of the urban environment.

[0122] In some embodiments, MobileNet V2 is used to replace the original backbone network of DeeplabV3+, and a channel and spatial attention module is added to optimize MobileNet V2, so that the texture and other detail features of bare land are enhanced to effectively distinguish bare land from background objects. Based on the above embodiments, Figure 2 is a flowchart of an implementation of a remote sensing image bare land classification method provided by an embodiment of the present application, as Figure 2 shown, step S102 can be implemented by steps S201 to S203:

[0123] Step S201, through the channel attention and spatial attention mechanism, attention feature extraction is performed on the remote sensing image to be classified to obtain an attention feature image.

[0124] In the embodiments of the present application, CBAM is added to MobileNet V2, and the constraints on spatial features and channel features of the intermediate feature map of the backbone network of DeepLabv3 are increased in the generation stage. CBAM contains two independent sub-modules, namely a channel attention module (CAM) and a spatial attention module (SAM), which respectively pay attention to the channel and spatial dimensions. In the embodiments of the present application, the channel attention module and the spatial attention module can be combined in a serial manner to improve the feature extraction accuracy.

[0125] In the embodiments of the present application, the CBAM module is integrated into the related network architecture, which not only saves parameters and computing power, but also optimizes the related network architecture.

[0126] In some embodiments, the attention feature extraction on the remote sensing image to be classified through the channel attention and spatial attention mechanism can be implemented by steps S2011 to S2018:

[0127] Step S2011, through the channel attention module, maximum pooling processing and average pooling processing are respectively performed on the remote sensing image to be classified to obtain a channel maximum pooling feature image and a channel average pooling feature image.

[0128] In the embodiments of the present application, first, the channel attention module is used to extract attention features from the remote sensing image to be classified, and the maximum pooling processing (global max pooling) and the average pooling processing (global average pooling) based on the width (width) and the height (height) are performed on the remote sensing image to be classified to obtain a channel maximum pooling feature image and a channel average pooling feature image.

[0129] In the embodiment of the present application, the CBAM introduces position information by performing global pooling on the channel, and each position contains information of a local region in the remote sensing image to be classified.

[0130] Step S2012, performing feature extraction on the channel maximum pooling feature image and the channel average pooling feature image respectively through a shared fully connected layer, to obtain a third feature image and a fourth feature image.

[0131] In some embodiments, the shared fully connected layer can be a two-layer neural network (Shared MLP), which can realize feature conversion and information reorganization. The shared fully connected layer respectively converts the dimensions of the channel maximum pooling feature image and the channel average pooling feature image, for example, the dimension of the channel maximum pooling feature image is C1 dimension, and the shared fully connected layer converts C1 dimension to C2 dimension, each dimension of C2 dimension integrates all C1 dimension information, but different dimensions in C2 dimension use different information emphases.

[0132] Step S2013, sequentially performing element addition processing and activation processing on the third feature image and the fourth feature image, to obtain an initial channel feature image.

[0133] Step S2014, performing element multiplication processing on the initial channel feature image and the remote sensing image to be classified, to obtain a channel feature image.

[0134] In the embodiment of the present application, the third feature image and the fourth feature image can be sequentially subjected to element-wise addition operation and then subjected to sigmoid activation operation processing, to obtain an initial channel feature image, and then the initial channel feature image and the remote sensing image to be classified are subjected to element-wise multiplication processing, to obtain a channel feature image. The channel feature image is the output feature image of the channel attention module, which is the input image of the spatial attention module.

[0135] Step S2015, performing maximum pooling processing and average pooling processing on the channel feature image respectively through a spatial attention module, to obtain a spatial maximum pooling feature image and a spatial average pooling feature image.

[0136] In the embodiment of the present application, the spatial attention module can be used to extract attention features from the channel feature image, and perform maximum pooling processing (global max pooling) and average pooling processing (global average pooling) on the channel feature image based on the channel, to obtain a spatial maximum pooling feature image and a spatial average pooling feature image.

[0137] Step S2016, the spatial maximum pooling feature image and the spatial average pooling feature image are spliced to obtain a spatial splicing feature image.

[0138] The embodiment of the application can splice the spatial maximum pooling feature image and the spatial average pooling feature image based on a channel to obtain a spatial splicing feature image.

[0139] Step S2017, the spatial splicing feature image is sequentially subjected to dimension reduction processing and activation processing to obtain a spatial feature image.

[0140] The embodiment of the application can perform dimension reduction processing on the spatial splicing feature image through a 7x7 convolution layer to reduce the dimension to one channel, and then perform sigmoid activation to generate a spatial feature image.

[0141] Step S2018, the spatial feature image and the channel feature image are subjected to element multiplication processing to obtain an attention feature image.

[0142] Finally, the spatial feature image and the channel feature image are subjected to element multiplication processing to obtain an attention feature image, which is the output feature image of the CBAM module.

[0143] Step S202, the attention feature image is subjected to feature extraction through a feature extraction network to obtain a feature image.

[0144] Step S2021, the attention feature image is subjected to dimension increase processing to obtain a dimension-increased feature image.

[0145] Step S2022, the dimension-increased feature image is subjected to down-sampling processing through a deep convolution layer to obtain a down-sampled feature image.

[0146] Step S2023, the down-sampled image is subjected to dimension reduction processing through a point-wise convolution layer to obtain the feature image.

[0147] In some embodiments, when the step length is 1, the remote sensing image to be classified is first subjected to CBAM module processing to obtain an attention feature image, and then sequentially subjected to 1x1 convolution layer dimension increase, deep convolution layer and point-wise convolution layer dimension reduction processing to obtain a feature image. The feature image and the attention feature image are subjected to residual operation to realize fusion of the features processed by MobileNet V2 and CBAM, to obtain a fusion image, and then subjected to CBAM module processing again to obtain the output feature image of the backbone network of DeepLabv3. When the step length is 2, there is no residual structure, and the remaining steps are the same as those when the step length is 1.

[0148] Step S203, attention feature extraction is performed on the feature image through the channel attention and spatial attention mechanism to obtain the first feature image and the second feature image with different levels.

[0149] In the embodiment of the application, after obtaining the fusion image, attention feature extraction is performed on the fusion image again through the channel attention and spatial attention mechanism to obtain the first feature image and the second feature image with different levels.

[0150] In the embodiment of the application, MobileNet V2 is used as the backbone network, which reduces the calculation parameter amount by 90%, accelerates the fitting speed of the model, and avoids the overfitting phenomenon. Secondly, the CBAM module is added to the starting layer of MobileNet V2 respectively, the MobileNet V2 network is optimized, the interference of non-interest areas such as meadow, forest bare soil and buildings is excluded, and the classification accuracy is improved.

[0151] In some embodiments, the remote sensing image bare land classification method provided by the foregoing embodiments is implemented through a bare land semantic segmentation model, and the bare land semantic segmentation model at least includes a feature extraction module, a dilated convolution module, a splicing module and a classification module; the bare land semantic segmentation model is obtained through steps S301 to S307:

[0152] Step S301, input sample data set into the bare land semantic segmentation model, the sample data set includes a sample image, and the size of the sample image is 256*256 pixels.

[0153] In the embodiment of the application, about 1000 sample remote sensing images including bare land can be obtained first, the bare land and the background category in the sample remote sensing image are labeled to obtain a labeled sample remote sensing image, and the labeled sample remote sensing image after labeling is determined as a sample data set.

[0154] In some embodiments, the bare land in the sample remote sensing image can be labeled by using a semantic segmentation labeling tool of a deep learning platform, and the generated bare land and background binary classification segmentation label is saved in the corresponding labeled sample remote sensing image.

[0155] Step S302, feature extraction is performed on the sample image through the feature extraction module to obtain first feature sample images and second feature sample images with different levels.

[0156] Step S303, the first feature sample image is processed through the dilated convolution module to obtain a deep feature sample image of the sample image.

[0157] In step S304, the first feature sample image and the deep feature sample image are spliced by the splicing module to obtain a spliced feature sample image.

[0158] In step S305, the pixel points in the sample image are classified according to the spliced feature sample image by the classification module to obtain a sample classification result of each pixel point in the sample image, and the sample classification result at least includes a bare land or background category.

[0159] In step S306, the sample classification result is input into a preset loss model to obtain a loss result.

[0160] In the embodiment of the present application, the preset loss model includes a loss function, the similarity between the sample classification result and a pre-labeled sample data set is calculated by the loss function, and the obtained similarity is determined as the loss result.

[0161] In some embodiments, the similarity between the sample classification result and the pre-labeled sample data set can be valued from 0 to 100%. Here, a similarity threshold can be pre-set, when the similarity between the sample classification result and the pre-labeled sample data set is greater than the similarity threshold, it indicates that the bare land semantic segmentation model can accurately predict the bare land in the sample data set, so there is no need to further train the bare land semantic segmentation model, that is, the training of the bare land semantic segmentation model can be stopped; when the similarity between the sample classification result and the pre-labeled sample data set is less than or equal to the similarity threshold, it indicates that the bare land semantic segmentation model cannot accurately predict the bare land in the sample data set, so it is necessary to continue to further train the bare land semantic segmentation model.

[0162] In some embodiments, a convergence condition of the bare land semantic segmentation model training process can also be set, for example, the convergence condition can be that when the training duration reaches a duration threshold, the training of the bare land semantic segmentation model is stopped, or the convergence condition can be that when the training times reaches a times threshold, the training of the bare land semantic segmentation model is stopped, or the convergence condition can be that when the similarity between the sample classification result output by the bare land semantic segmentation model and the pre-labeled sample data set is greater than the above-mentioned similarity threshold, the training of the bare land semantic segmentation model is stopped.

[0163] In step S307, the parameters in the feature extraction module, the dilated convolution module, the splicing module and the classification module are corrected according to the loss result to obtain a trained bare land semantic segmentation model.

[0164] Here, the parameters in the feature extraction module, the dilated convolution module, the splicing module and the classification module are corrected, that is, the parameters of the bare land semantic segmentation model are corrected, so that the predicted bare land classification of the bare land semantic segmentation model after inputting the sample data set into the bare land semantic segmentation model is the same as the bare land classification labeled in the sample data set.

[0165] The bare land semantic segmentation model is trained by the sample data set, and the trained bare land semantic segmentation model is obtained, so that when the bare land classification is performed, the remote sensing image is classified by the trained bare land semantic segmentation model, and the accuracy of the bare land classification is improved.

[0166] The bare land classification method of the remote sensing image is further applied in an actual scene.

[0167] The bare land classification method of the remote sensing image provided in the embodiments of the present application first can use high-resolution remote sensing images (i.e. remote sensing images) to establish a bare land sample set (i.e. a sample data set); secondly, the bare land semantic segmentation model constructed by the bare land sample set is trained and tested to obtain a final bare land semantic segmentation model; then the large-format high-resolution remote sensing image is cropped to meet the requirements of the bare land semantic segmentation model for the input image; thereafter, the cropped high-resolution remote sensing image block (i.e. the remote sensing image to be classified) is subjected to semantic segmentation (i.e. bare land classification), and a segmentation result is obtained; finally, the geographic location information of the segmentation result is restored, and a bare land grid extraction result is finally obtained.

[0168] The bare land classification method of the remote sensing image provided in the embodiments of the present application can be implemented through steps S10 to S60:

[0169] S10, a bare land sample set is established by using a large-format high-resolution remote sensing image that has been subjected to radiation correction and geometric correction.

[0170] In the embodiments of the present application, a multi-spectral high-resolution remote sensing image that has been subjected to system radiation correction and system geometric correction is obtained, a certain number of image regions containing bare land are selected from the remote sensing image, a deep learning semantic segmentation labeling tool is used to label the bare land and the background categories in the selected regions (wherein the bare land is labeled as bare land and the background region is labeled as non-bare land), the generated binary classification segmentation label is saved in the corresponding picture file, and a bare land sample set is constructed in this way.

[0171] S20, a Deeplabv3+_M_CBAM bare land segmentation model is constructed based on a Deeplabv3+ deep learning model.

[0172] In the embodiment of the present application, the backbone network Xception of DeepLabv3+ is replaced by a lightweight MobileNet V2 network to reduce the original model parameter quantity, avoid overfitting, and improve the training efficiency of the model. In order to exclude the interference of non-interest areas such as dry grassland, bare soil under forest and buildings, a Convolutional Block Attention Module (CBAM) is used to optimize the MobileNet V2 network, and a DeepLabv3+_M_CBAM model is established. The model is trained and tested, and finally a DeepLabv3+_M_CBAM bare land semantic segmentation model is obtained.

[0173] In the embodiment of the present application, Figure 4 is a DeepLabv3+_M_CBAM model structure diagram provided by the embodiment of the present application, as Figure 4 shown, the DeepLabv3+_M_CBAM model is composed of an encoding structure 401 (Encoder) and a decoding structure 402 (Decoder). The input is a remote sensing image to be classified with a fixed size of 256x256 pixels. The encoding structure 401 includes a backbone network 4011 (Backbone), an Atrous Spatial Pyramid Pooling (ASPP) 4012, and a 1x1 convolution layer 4013. The decoding structure 402 includes a splicing layer 4021, a convolution layer 4022, and an up-sampling layer 4023. The remote sensing image to be classified is obtained by the encoding structure 401 and the decoding structure 402 to obtain the bare land prediction result.

[0174] In some embodiments, the Atrous Spatial Pyramid Pooling 4012 is used to extract features of different scales of the dilated convolution, and the 1x1 convolution layer is used to compress the multiple scale features (i.e. dilated convolution feature images) output in the ASPP into a feature layer (i.e. deep feature image), and input into the decoding structure 402. The decoding structure 402 performs splicing operation on the shallow features and the deep features (i.e. the first feature image and the second feature image), and then passes through the convolution layer 4022 and the up-sampling layer 4023 to obtain the final bare land semantic segmentation output image.

[0175] In some embodiments, in order to reduce the model parameters, the embodiment of the present application uses MobileNet V2 as the backbone network of the DeepLabv3+_M_CBAM model. MobileNet V2 is a lightweight network in the field of semantic segmentation, and has small model size and strong feature extraction capability, and has good performance in semantic segmentation and target detection tasks.

[0176] In some embodiments, to exclude the interference of non-interesting areas such as bare land, bare soil under forest and buildings, the CBAM module is added to the starting layer of the MobileNet V2 network to obtain an M_CBAM model, and the M_CBAM model is used as the backbone network of the Deeplabv3+_M_CBAM model. The CBAM module is connected in series with a spatial attention module (Spatial Attention Module) and a channel attention module (Channel Attention Module). Given an intermediate feature map, the attention map can be inferred along two independent dimensions in turn, and then the attention map is multiplied with the input feature map to perform adaptive feature optimization.

[0177] Figure 5 is a schematic diagram of the M_CBAM model structure provided by the embodiments of the present application, as shown in Figure 5 When the step (stride) is 1 block, the input picture (Input, i.e. the picture to be classified) is first processed by the CBAM module to obtain a feature map 501 (i.e. an attention feature map); then the feature map 501 is sequentially processed by a 1x1 convolution layer 502 to increase the dimension, a deep convolution layer 503, and a point-by-point convolution layer 504 to reduce the dimension, to obtain a corresponding feature map, and the two feature maps are subjected to residual operation through a residual structure 505 to realize the fusion of the feature maps processed by MobileNet V2 and CBAM; finally, the output feature map of the backbone network is obtained by processing through the CBAM module again. When the step is 2 blocks, there is no residual structure 505, and the remaining steps are the same as when the step is 1 block.

[0178] Figure 6 is a schematic diagram of the CBAM model structure provided by the embodiments of the present application, as shown in Figure 6As shown, the CBAM model includes a serial channel attention module 601 and a spatial attention module 602. Among them, the input feature map (i.e. the remote sensing image to be classified) is subjected to maximum pooling 6011 and average pooling 6012 respectively, to obtain two feature maps, and then the two feature maps (third feature image and fourth feature image) are sent into a shared fully connected layer 6013. The features output by the shared fully connected layer 6013 are subjected to an element-wise addition operation, and then subjected to a sigmoid activation operation to generate an initial channel feature image. Finally, the initial channel feature image and the input feature map are subjected to an element-wise multiplication operation to generate a channel feature image. The channel feature image output by the channel attention module 601 is taken as the input feature map of the spatial attention module 602. First, a channel-based maximum pooling and average pooling 6021 is performed to obtain two feature maps (i.e. a spatial maximum pooling feature image and a spatial average pooling feature image), and then the two feature maps are spliced based on the channel to obtain a spatial splicing feature image. Then the spatial splicing feature image is subjected to a convolution operation by a convolution layer 6022 to reduce the dimension to 1 channel. Then a sigmoid activation operation is performed to generate a spatial feature image. Finally, the spatial feature image and the channel feature image are multiplied to obtain the finally generated feature image (i.e. the attention feature image).

[0179] The embodiment of the present application selects the semantic segmentation model DeepLabv3+ as the basic framework, replaces the backbone network Xception with the lightweight MobileNet V2 network, and optimizes the MobileNet V2 network by combining the channel and spatial attention mechanisms of the CBAM module, so that the model has stronger discrimination ability for interfering ground objects, and builds a DeepLabv3+_M_CBAM bare land semantic segmentation model, solving the problem of interference of multiple complex backgrounds in the extraction of bare land from remote sensing images, and avoiding model overfitting.

[0180] S30, cropping the whole high-resolution remote sensing image to be segmented, and naming the cropped image block according to the row and column numbers.

[0181] In some embodiments, the input image size of the related deep learning semantic segmentation model is generally several hundred or several thousand pixels. For example, the input image size of the DeepLabv3+_M_CBAM model used in the embodiment of the present application is 256x256 pixels, which is very different from the size of large remote sensing images. Therefore, the remote sensing image needs to be cropped before being input into the network to obtain a standard image block (i.e. a remote sensing image to be classified), and the standard image block is named according to the row and column numbers of the cropped image.

[0182] S40, based on the DeepLabv3+_M_CBAM model, extracting the bare land in each image block after cropping to obtain the pixel-level bare land binary classification result of the corresponding image block.

[0183] In some embodiments, the cropped standard image (i.e. remote sensing image to be classified) block is sequentially input into the trained bare land extraction model DeepLabv3+_M_CBAM (i.e. bare land semantic segmentation model), and bare land semantic segmentation is performed for each standard image block to obtain the bare land classification extraction result.

[0184] S50, the classification results are spliced according to the row and column numbers of the image to obtain the bare land classification result of the whole image.

[0185] In some embodiments, each standard image block can be spliced according to the cropping row and column numbers to obtain the spliced classification result.

[0186] S60, the classification result of the spliced whole image is restored to the geographical position information to obtain the bare land raster classification map of the whole image.

[0187] In some embodiments, the geographical position information of the spliced classification result can be restored according to the spatial coordinate system of the original remote sensing image, and finally the bare land raster extraction result is generated.

[0188] The embodiments of the present application aim at the problem that the related deep learning model cannot be directly input into the large-format remote sensing image. Through the two steps of cropping of high-resolution remote sensing image and restoration of geographical position information of bare land semantic segmentation result, the problems that the deep learning model cannot directly detect the large-format remote sensing image and the segmentation result is not conducive to classification statistics are solved. Through the remote sensing image bare land classification method and device proposed in the present application, the rapid, high-precision and intelligent extraction of bare land in large-format high-resolution remote sensing image can be realized.

[0189] The remote sensing image bare land classification method provided in this application improves upon the DeepLabv3+ model. It proposes and constructs the DeepLabv3+_M_CBAM bare land semantic segmentation model, along with its complete technical process. This accelerates the convergence speed of DeepLabv3+_M_CBAM during prediction, avoids model overfitting, and enhances the model's ability to fuse channel and spatial features. It effectively avoids interference from complex background features in bare land extraction. The method can automatically extract bare land information for environmental governance from large-format, high-resolution remote sensing images with high accuracy, speed, and intelligence. By extracting bare land from large-format, high-resolution remote sensing images, it quickly obtains bare land raster classification results, improving the automation and intelligent processing efficiency and accuracy of bare land extraction. This provides target areas for precise bare land governance and contributes to the refined management of the urban environment and the rational utilization of land resources.

[0190] Figure 7 These are the curves showing the change in average intersection-union ratio (IU) accuracy during the training process of the three models provided in the embodiments of this application. From... Figure 7 As can be seen, the Mean Intersection over Union (MIoU) accuracy curves of the three models—DeepLabv3+_Xception, DeepLabv3+_MobileNet V2, and DeepLabv3+_M_CBAM—gradually stabilize with increasing training iterations, but their characteristics of change differ significantly. The DeepLabv3+_MobileNet V2 model achieves over 80% accuracy after the first training iteration, demonstrating the learning advantage of lightweight networks on small datasets, and reaches the optimal accuracy of DeepLabv3+_Xception after the 20th training iteration. After introducing the CBAM module to improve the MobileNet V2 network, DeepLabv3+_M_CBAM achieves higher accuracy in the early stages of training, and its final training accuracy is higher than the previous two models, indicating that the introduction of the CBAM module helps the model learn the features of bare land faster and better.

[0191] The method for classifying bare land in remote sensing images provided in the embodiments of this application Figure 8 This is a schematic diagram of the bare land classification results provided in an embodiment of this application, wherein, Figure 8 Image a shows a large-format, high-resolution remote sensing image of a village in a certain city, after radiometric and geometric correction. The classification result after bare land classification is shown in image b, where the black areas represent the extracted bare land regions. Figure 8It can be known that the bare land classification method of the remote sensing image provided in the embodiment of the application has a Mean Pixel Accuracy (MPA) of 92.98%, and a Frames Per Second (FPS) of 42.99f / s, realizes rapid, high-precision and intelligent extraction of large-format high-resolution remote sensing images, and the example verifies that the method provided in the embodiment of the application is effective and achieves real-time processing speed.

[0192] Based on the above embodiment, the embodiment of the application provides a bare land classification device of a remote sensing image; Figure 9 is a schematic diagram of a bare land classification device of a remote sensing image provided in the embodiment of the application, as Figure 9 shown, the device 90 includes an acquisition module 901, a feature extraction module 902, a hollow convolution module 903, a splicing module 904 and a classification module 905.

[0193] The acquisition module 901 is configured to acquire a remote sensing image to be classified, wherein the remote sensing image to be classified has a size of 256x256 pixels.

[0194] The feature extraction module 902 is configured to perform feature extraction processing on the remote sensing image to be classified to obtain first feature images and second feature images with different levels.

[0195] The hollow convolution module 903 is configured to perform hollow convolution processing on the first feature images to obtain deep feature images of the remote sensing image to be classified.

[0196] The splicing module 904 is configured to perform splicing processing on the second feature images and the deep feature images to obtain spliced feature images.

[0197] The classification module 905 is configured to classify pixel points in the remote sensing image to be classified according to the spliced feature images to obtain classification results of each pixel point in the remote sensing image to be classified, wherein the classification results at least include a bare land or a background category.

[0198] In some embodiments, the feature extraction module 902 is further configured to perform attention feature extraction on the remote sensing image to be classified through a channel attention and a spatial attention mechanism to obtain attention feature images, perform feature extraction on the attention feature images through a feature extraction network to obtain feature images, and perform attention feature extraction on the feature images through the channel attention and the spatial attention mechanism to obtain the first feature images and the second feature images with different levels.

[0199] In some embodiments, the feature extraction module 902 is further configured to perform maximum pooling processing and average pooling processing on the remote sensing image to be classified respectively by a channel attention module to obtain a channel maximum pooling feature image and a channel average pooling feature image; perform feature extraction on the channel maximum pooling feature image and the channel average pooling feature image respectively by a shared fully connected layer to obtain a third feature image and a fourth feature image; perform element addition processing and activation processing on the third feature image and the fourth feature image in sequence to obtain an initial channel feature image; perform element multiplication processing on the initial channel feature image and the remote sensing image to be classified to obtain a channel feature image; perform maximum pooling processing and average pooling processing on the channel feature image respectively by a spatial attention module to obtain a spatial maximum pooling feature image and a spatial average pooling feature image; perform splicing processing on the spatial maximum pooling feature image and the spatial average pooling feature image to obtain a spatial splicing feature image; perform dimension reduction processing and activation processing on the spatial splicing feature image in sequence to obtain a spatial feature image; and perform element multiplication processing on the spatial feature image and the channel feature image to obtain an attention feature image.

[0200] In some embodiments, the feature extraction module 902 is further configured to perform dimension increasing processing on the attention feature image to obtain a dimension-increased feature image; perform down-sampling processing on the dimension-increased feature image by a deep convolution layer to obtain a down-sampled feature image; and perform dimension reduction processing on the down-sampled image by a point-wise convolution layer to obtain the feature image.

[0201] In some embodiments, the cavity convolution module is further configured to perform parallel cavity convolution processing on the first feature image according to cavity spatial convolution pyramid pooling to obtain five cavity convolution feature images; and perform point-wise convolution processing on the five cavity convolution feature images to obtain the deep feature image of the remote sensing image to be classified.

[0202] In some embodiments, the classification module is further configured to perform convolution processing on the splicing feature image by a convolution kernel with a preset size to obtain a convolution feature image; and perform up-sampling processing on the convolution feature image to obtain a classification result of each pixel point in the remote sensing image to be classified.

[0203] In some embodiments, the apparatus further includes: a first acquisition module configured to acquire a remote sensing image, the remote sensing image including at least two image blocks with a size of 256x256 pixels; and a cropping module configured to crop the remote sensing image according to a preset image size to obtain at least two remote sensing images to be classified and position information of each remote sensing image to be classified on the remote sensing image.

[0204] Correspondingly, after obtaining the classification result of each pixel point in the to-be-classified remote sensing image, the device further comprises:

[0205] a splicing module configured to splice the classified to-be-classified remote sensing image according to the position information of each to-be-classified remote sensing image on the remote sensing image to obtain a classified image;

[0206] In some embodiments, the device further comprises: a first acquisition module configured to acquire at least one thousand sample remote sensing images including bare land; a labeling module configured to label the bare land and non-bare land (i.e. background category) in the sample remote sensing images to obtain labeled sample remote sensing images; and a determination module configured to determine the labeled sample remote sensing images as a sample data set.

[0207] In some embodiments, the method is implemented by a bare land semantic segmentation model, which at least comprises a feature extraction module, a dilated convolution module, a splicing module and a classification module; and the bare land semantic segmentation model training device comprises: a first input module configured to input a sample data set into the bare land semantic segmentation model, wherein the sample data set comprises a sample image, and the size of the sample image is 256x256 pixels; a feature extraction module configured to perform feature extraction processing on the sample image by the feature extraction module to obtain first feature sample images and second feature sample images with different levels; a dilated convolution module configured to perform dilated convolution processing on the first feature sample images by the dilated convolution module to obtain deep feature sample images of the sample image; a splicing module configured to perform splicing processing on the second feature sample images and the deep feature sample images by the splicing module to obtain a spliced feature image; a classification module configured to classify all pixel points in the sample image according to the spliced feature image by the classification module to obtain a sample classification result of each pixel point in the sample image, wherein the sample classification result at least comprises bare land or a background category; a second input module configured to input the sample classification result into a preset loss model to obtain a loss result; and a correction module configured to correct parameters in the feature extraction module, the dilated convolution module, the splicing module and the classification module according to the loss result to obtain a trained bare land semantic segmentation model.

[0208] The description of the above device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the device embodiments of the present application, please refer to the description of the method embodiments of the present application for understanding.

[0209] It should be noted that, in the embodiments of the present application, if the remote sensing image bare land classification method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application can be embodied in the form of a software product, which is stored in a storage medium, includes a number of instructions for making an electronic device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the methods described in the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk, and various storage media that can store program codes. Thus, the embodiments of the present application are not limited to any specific hardware and software combination.

[0210] The embodiments of the present application provide a remote sensing image bare land classification device, which comprises a memory and a processor, the memory stores a computer program capable of running on the processor, and the processor implements the remote sensing image bare land classification method when executing the computer program.

[0211] The embodiments of the present application provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the remote sensing image bare land classification method. The computer readable storage medium can be transitory or non-transitory.

[0212] The embodiments of the present application provide a computer program product, which comprises a non-transitory computer readable storage medium storing a computer program, and the computer program is read and executed by a computer to implement some or all steps of the above method. The computer program product can be implemented by hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in an optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK) and the like.

[0213] It should be noted that, Figure 10 is a hardware entity schematic diagram of a remote sensing image bare land classification device provided by the embodiments of the present application, as shown in Figure 10As shown, the hardware entity of the remote sensing image bare land classification device 100 includes a processor 101, a communication interface 102 and a memory 103, wherein the processor 101 generally controls the operation of the remote sensing image bare land classification device 100.

[0214] The communication interface 102 can enable the bare land classification device to communicate with other terminals or servers through a network.

[0215] The memory 103 is configured to store instructions and applications executable by the processor 101, and can also cache data (for example, image data) to be processed by the processor 101 and modules in the bare land classification device 10, which can be implemented by FLASH or Random Access Memory (RAM). The processor 101, the communication interface 102 and the memory 103 can perform data transmission through a bus 104.

[0216] It should be noted that the above description of the storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of the present application, please refer to the description of the method embodiments of the present application.

[0217] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of each process does not mean the execution order, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above sequence number of the embodiments of the present application is only for description, not representing the advantages and disadvantages of the embodiments.

[0218] It should be noted that in this document, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0219] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other manners. The embodiments described above are merely exemplary, for example, the division of the units is only a logical function division, and there can be another division manner for the actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the various components shown or discussed can be through some interfaces, and there can be an indirect coupling or communication connection between the various components, and there can be an electrical, mechanical or other form of connection.

[0220] The units described as separate components above can or can not be physically separate, and the components shown as units can or can not be physical units; they can be located in one place or distributed on a plurality of network units; and some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0221] In addition, all the functional units in the embodiments of the present application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be realized in the form of hardware, or in the form of hardware plus software functional units.

[0222] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by a program instructing related hardware, and the foregoing program can be stored in a computer readable storage medium, and the program executes the steps of the above-mentioned method embodiments when executed; and the foregoing storage medium includes a mobile storage device, a read only memory (ROM), a magnetic disc or an optical disc, and various storage medium that can store program codes. Or, when the integrated units of the present application are realized in the form of software functional modules and sold or used as independent products, they can also be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for making an electronic device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the methods described in the embodiments of the present application. The foregoing storage medium includes a mobile storage device, a ROM, a magnetic disc or an optical disc, and various storage medium that can store program codes. The above is only an embodiment of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed in the present application, which should be covered within the protection scope of the present application.

Claims

1. A method for bare soil classification of remote sensing images, characterized in that, The method comprises: acquiring a remote sensing image to be classified, the size of the remote sensing image to be classified being 256*256 pixels; performing feature extraction processing on the remote sensing image to be classified to obtain first feature images and second feature images with different levels; performing hole convolution processing on the first feature images to obtain deep feature images of the remote sensing image to be classified; performing splicing processing on the second feature images and the deep feature images to obtain spliced feature images; classifying all pixel points in the remote sensing image to be classified according to the spliced feature images to obtain a classification result of each pixel point in the remote sensing image to be classified, the classification result at least including a bare land or background category; wherein the feature extraction processing on the remote sensing image to be classified to obtain the first feature images and the second feature images with different levels comprises: performing attention feature extraction on the remote sensing image to be classified through a channel attention and a spatial attention mechanism to obtain attention feature images; performing feature extraction on the attention feature images through a feature extraction network to obtain feature images; performing attention feature extraction on the feature images through the channel attention and the spatial attention mechanism to obtain the first feature images and the second feature images with different levels.

2. The method of claim 1, wherein, The attention feature extraction on the remote sensing image to be classified through the channel attention and the spatial attention mechanism to obtain the attention feature images comprises: performing maximum pooling processing and average pooling processing on the remote sensing image to be classified through a channel attention module to obtain a channel maximum pooling feature image and a channel average pooling feature image; performing feature extraction on the channel maximum pooling feature image and the channel average pooling feature image through a shared fully connected layer to obtain a third feature image and a fourth feature image; performing element addition processing and activation processing on the third feature image and the fourth feature image in sequence to obtain an initial channel feature image; performing element multiplication processing on the initial channel feature image and the remote sensing image to be classified to obtain a channel feature image; performing maximum pooling processing and average pooling processing on the channel feature image through a spatial attention module to obtain a spatial maximum pooling feature image and a spatial average pooling feature image; performing splicing processing on the spatial maximum pooling feature image and the spatial average pooling feature image to obtain a spatial splicing feature image; performing dimension reduction processing and activation processing on the spatial splicing feature image in sequence to obtain a spatial feature image; performing element multiplication processing on the spatial feature image and the channel feature image to obtain the attention feature images.

3. The method of claim 1, wherein, The feature extraction on the attention feature images through the feature extraction network to obtain the feature images comprises: performing dimension increasing processing on the attention feature images to obtain dimension-increased feature images; performing down-sampling processing on the dimension-increased feature images through a deep convolution layer to obtain down-sampled feature images; performing dimension reduction processing on the down-sampled feature images through a point-by-point convolution layer to obtain the feature images.

4. The method of claim 1, wherein, The method further comprises: obtaining a remote sensing image, wherein the remote sensing image comprises at least two image blocks with a size of 256*256 pixels; cropping the remote sensing image according to a preset image size to obtain at least two remote sensing images to be classified and position information of each remote sensing image to be classified on the remote sensing image; 5. The method of claim 1, wherein, correspondingly, after obtaining the classification result of each pixel point in the remote sensing image to be classified, the method further comprises: splicing the classified remote sensing image to be classified according to the position information of each remote sensing image to be classified on the remote sensing image to obtain a classification image; reconstructing geographical information of the classification image according to geographical information of the remote sensing image to obtain a remote sensing classification image; 6. The method according to any one of claims 1 to 5, characterized in that, determining a bare land classification image corresponding to the remote sensing classification image according to the classification result of each pixel point in the remote sensing classification image. The method further comprises: obtaining at least one thousand sample remote sensing images comprising bare land; annotating the bare land and non-bare land in the sample remote sensing images to obtain annotated sample remote sensing images; determining the annotated sample remote sensing images after annotation as a sample data set. The method is implemented through a bare land semantic segmentation model, wherein the bare land semantic segmentation model comprises at least a feature extraction module, a dilated convolution module, a splicing module and a classification module; and the bare land semantic segmentation model is obtained through the following steps: inputting a sample data set into the bare land semantic segmentation model, wherein the sample data set comprises sample images, and the size of the sample images is 256*256 pixels; 7. The method of claim 6, wherein, performing feature extraction processing on the sample images through the feature extraction module to obtain first feature sample images and second feature sample images with different levels; performing dilated convolution processing on the first feature sample images through the dilated convolution module to obtain deep feature sample images of the sample images; performing splicing processing on the second feature sample images and the deep feature sample images through the splicing module to obtain spliced feature sample images; performing classification on all pixel points in the sample images according to the spliced feature sample images through the classification module to obtain sample classification results of each pixel point in the sample images, wherein the sample classification results comprise at least a bare land or background category.

8. The method of claim 7, wherein, ​ ​ ​ ​ ​ ​ Input the sample classification result into a preset loss model to obtain a loss result; According to the loss result, parameters in the feature extraction module, the dilated convolution module, the splicing module, and the classification module are corrected to obtain a trained bare land semantic segmentation model.

9. A device for classifying bare land in a remote sensing image, characterized in that, The apparatus includes: An acquisition module configured to acquire a remote sensing image to be classified, the remote sensing image to be classified having a size of 256x256 pixels; A feature extraction module configured to perform feature extraction processing on the remote sensing image to be classified to obtain first feature images and second feature images having different levels; A dilated convolution module configured to perform dilated convolution processing on the first feature images to obtain deep feature images of the remote sensing image to be classified; A splicing module configured to perform splicing processing on the second feature images and the deep feature images to obtain spliced feature images; A classification module configured to perform classification on all pixel points in the remote sensing image to be classified according to the spliced feature images to obtain classification results of each pixel point in the remote sensing image to be classified, the classification results including at least a bare land or background category; The feature extraction module is further configured to perform attention feature extraction on the remote sensing image to be classified through a channel attention and spatial attention mechanism to obtain attention feature images, perform feature extraction on the attention feature images through a feature extraction network to obtain feature images, and perform attention feature extraction on the feature images through the channel attention and spatial attention mechanism to obtain the first feature images and the second feature images having different levels.

Images