Land utilization type classification method, system and equipment based on multiband remote sensing image
By cropping and data conversion of multi-band remote sensing images and using integrated classification models for image interpretation, the problems of missing information and low classification accuracy of multi-band remote sensing images in the existing technology are solved, and efficient land use classification is achieved.
Patent Information
- Application Number
- CN202411200238.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-29
- Publication Date
- 2024-12-06
AI Technical Summary
The prior art cannot adapt to multi-band remote sensing images in land use classification, resulting in missing information and low classification accuracy, especially under multi-classification tasks, and the data set input and segmentation process is complicated.
By obtaining multi-band remote sensing images, performing cropping and data conversion, tensor format data of multi-dimensional features is obtained, and a preset integrated classification model is used for image interpretation and data integration, realizing multi-band feature extraction and integrated interpretation. Suitable for any multi-band or multi-channel remote sensing image.
It improves the accuracy and efficiency of land use classification, solves the problems of missing information and low classification accuracy, simplifies the data set collation and model training process, and is suitable for multi-classification tasks.
Smart Images

Figure CN119091213A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of land use classification, and in particular to a land use type classification method, system and equipment based on multi-band remote sensing images. Background Art
[0002] Land use classification can make better use of land and realize the maximum value of land. Land use classification is mostly used in urban planning, environmental assessment and other aspects. Therefore, accurate land use classification identification is of great significance. With the continuous development of science and technology in my country, remote sensing images are gradually replacing artificial surveying and mapping data. High-resolution remote sensing images are increasingly widely used in land use classification because of their rich ground information.
[0003] Although the existing technology has achieved good results by applying deep learning technology to the land classification task of remote sensing images, there are still many shortcomings. Specifically, the existing technology is mainly based on U-Net to extract land use information from high-resolution remote sensing images. However, the classification object accepted by the existing U-net network is pictures, which has great restrictions on the information of the channels of remote sensing images, that is, the number of bands of remote sensing images is limited. Only fixed bands can be input, which will cause the information of remote sensing images to be missing, especially in the case of multi-classification tasks, which will reduce the classification accuracy. Summary of the invention
[0004] The present application provides a land use type classification method, system and equipment based on multi-band remote sensing images, which realizes the multi-band feature extraction and integrated interpretation results of remote sensing images, has no restriction on the number of bands of multi-band remote sensing images, and can be applied to remote sensing images with any multi-band number or multi-channel number, thereby improving the accuracy and efficiency of land use classification, and solving the problems existing in the existing land use classification technology based on deep learning technology, such as the inability to adapt to multi-band remote sensing images, missing remote sensing image information and low classification accuracy.
[0005] In the first aspect, the present application provides a land use type classification method based on multi-band remote sensing images, comprising:
[0006] Acquiring remote sensing image data of the target land, wherein the remote sensing image data includes multi-band remote sensing images;
[0007] Performing cropping processing on the multi-band remote sensing image to obtain a tile image set corresponding to the multi-band remote sensing image, wherein the tile image set includes at least two tile images;
[0008] Performing data conversion according to each of the tile images to obtain tensor format data with multi-dimensional features;
[0009] Performing image interpretation on the tensor format data through a preset integrated classification model to obtain a tensor format classification result corresponding to the tensor format data;
[0010] Data integration is performed based on the tensor format classification results to obtain the land use type classification results of the target land.
[0011] Optionally, performing cropping processing on the multi-band remote sensing image to obtain a tile image set corresponding to the multi-band remote sensing image includes:
[0012] Get the preset cropping rules;
[0013] According to the cropping rule, the multi-band remote sensing image is cropped to obtain at least two tile images with the same resolution.
[0014] Optionally, performing data conversion according to each of the tile images to obtain tensor format data with multi-dimensional features includes:
[0015] Determine each band of the multi-band remote sensing image;
[0016] For each band, performing matrix conversion on the tile image to obtain a numerical matrix corresponding to each band in the tile image;
[0017] A tensor format conversion is performed based on the numerical matrix to obtain tensor format data with multi-dimensional features.
[0018] Optionally, performing image interpretation on the tensor format data by using a preset integrated classification model to obtain a tensor format classification result corresponding to the tensor format data includes:
[0019] Extracting backbone features from the tensor format data using the integrated classification model to obtain fused feature data;
[0020] Classification and interpretation are performed based on the fused feature data to obtain a classification result in a tensor format.
[0021] Optionally, extracting backbone features from the tensor format data by using the integrated classification model to obtain fused feature data includes:
[0022] Determining the number of channel convolutions based on the number of bands of the multi-band remote sensing image;
[0023] Based on the channel convolution number, convolving the tensor format data through the integrated classification model to obtain a valid feature layer corresponding to the channel convolution number;
[0024] Sampling and stacking are performed according to the effective feature layer to obtain fused feature data.
[0025] Optionally, performing data integration according to the tensor format classification result to obtain the land use type classification result of the target land includes:
[0026] Performing image conversion according to the tensor format classification result to obtain a target format tile image;
[0027] Image stitching is performed based on the target format tile images to obtain a classification result map of the target land as the land use type classification result.
[0028] Optionally, before performing image interpretation on the tensor format data by using a preset integrated classification model, the method further includes:
[0029] Obtain original remote sensing images of the sample area;
[0030] Perform band preprocessing based on the original remote sensing image to obtain multi-band remote sensing image samples and sample labels;
[0031] The multi-band remote sensing image samples are cropped and enhanced according to the sample labels to obtain a tile sample label data set with multi-band features, wherein the tile sample label data set includes a sample label file and remote sensing cropped image samples corresponding to the sample label files in a one-to-one manner;
[0032] Performing data conversion based on the tile sample label data set to obtain sample tensor format data and sample test data with multi-dimensional features;
[0033] Performing model training according to the sample tensor format data to obtain the integrated classification model;
[0034] The integrated classification model is subjected to segmentation testing based on the sample test data until the integrated classification model converges.
[0035] Optionally, performing model training according to the sample tensor format data to obtain the integrated classification model includes:
[0036] constructing a first initial model and a second initial model;
[0037] Performing convolution training on the sample tensor format data through the first initial model to obtain effective feature layer samples and a target feature extraction model for the number of channels corresponding to the sample tensor format data;
[0038] Performing interpretation training on the second initial model based on the effective feature layer samples to obtain a target segmentation model;
[0039] Model integration is performed according to the target feature extraction model and the target segmentation model to obtain the integrated classification model.
[0040] In the second aspect, the present application provides a land use type classification system based on multi-band remote sensing images, including:
[0041] A multi-band remote sensing image data acquisition module is used to acquire remote sensing image data of the target land, wherein the remote sensing image data includes multi-band remote sensing images;
[0042] A cropping processing module, used for performing cropping processing according to the multi-band remote sensing image to obtain a tile image set corresponding to the multi-band remote sensing image, wherein the tile image set includes at least two tile images;
[0043] A data conversion module, used for performing data conversion according to each of the tile images to obtain tensor format data with multi-dimensional features;
[0044] An image interpretation module, used to perform image interpretation on the tensor format data through a preset integrated classification model to obtain a tensor format classification result corresponding to the tensor format data;
[0045] The data integration module is used to perform data integration based on the tensor format classification result to obtain the land use type classification result of the target land.
[0046] In a third aspect, the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;
[0047] Memory, used to store computer programs;
[0048] The processor is used to implement the steps of the land use type classification method based on multi-band remote sensing images as described in any embodiment of the first aspect when executing the program stored in the memory.
[0049] In summary, the embodiment of the present application obtains a multi-band remote sensing image of the target land, performs cropping processing based on the multi-band remote sensing image, and obtains a tile image set corresponding to the multi-band remote sensing image, the tile image set contains at least two tile images, and then performs data conversion based on each tile image to obtain tensor format data with multi-dimensional features, and interprets the tensor format data through a preset integrated classification model to obtain a tensor format classification result corresponding to the tensor format data, and then integrates the data based on the tensor format classification result to obtain the land use type classification result of the target land, thereby realizing multi-band feature extraction and integrated interpretation results of the remote sensing image, without limiting the number of bands of the multi-band remote sensing image, and can be applicable to any remote sensing image with multiple bands or multiple channels, thereby improving the accuracy and efficiency of land use classification, and solving the problems of the existing land use classification technology based on deep learning technology that cannot adapt to multi-band remote sensing images, missing remote sensing image information, and low classification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0051] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0052] Figure 1 A schematic diagram of a flow chart of a land use type classification method based on multi-band remote sensing images provided in an embodiment of the present application;
[0053] Figure 2 This is a schematic diagram of the steps of a land use type classification method based on multi-band remote sensing images provided by an optional embodiment of the present application;
[0054] Figure 3 It is a flow chart of an artificial neural network model based on "vgg+U-net network" provided by an optional example of this application;
[0055] Figure 4 It is a feature layer convolution stacked data graph provided by an optional example of the present application;
[0056] Figure 5 A structural block diagram of a land use type classification system based on multi-band remote sensing images provided in an embodiment of the present application;
[0057] Figure 6It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0058] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0059] To facilitate the understanding of the embodiments of the present application, further explanation will be given below in conjunction with the drawings and specific embodiments. The embodiments do not constitute a limitation on the embodiments of the present application.
[0060] Figure 1 A flow chart of a land use type classification method based on multi-band remote sensing images provided in an embodiment of the present application. Figure 1 As shown, the land use type classification method based on multi-band remote sensing images provided in the embodiment of the present application may specifically include the following steps:
[0061] Step 110: Acquire remote sensing image data of the target land, wherein the remote sensing image data includes multi-band remote sensing images.
[0062] In this embodiment, a multi-band remote sensing image may refer to a remote sensing image containing multiple spectral bands, each spectral band may correspond to a channel. Taking a remote sensing image containing four spectral bands as an example, the number of channels corresponding to the remote sensing image of the four spectral bands is 4. This embodiment does not limit the number of spectral bands contained in the multi-band remote sensing image.
[0063] In a specific implementation, this embodiment can obtain multi-band remote sensing image data of the target land by photographing the target land. The target land may refer to land / area that needs to be classified into land use types. This embodiment does not limit the target land, the method of obtaining the multi-band remote sensing image of the target land, and the size of the target land.
[0064] Step 120 , performing cropping processing on the multi-band remote sensing image to obtain a tile image set corresponding to the multi-band remote sensing image, wherein the tile image set includes at least two tile images.
[0065] In this embodiment, the tile image can be understood as a portion of the original multi-band remote sensing image. By cropping the multi-band remote sensing image, at least two tile images can be obtained. Each tile image corresponds to a portion of the original multi-band remote sensing image. By stitching multiple tile images together, a complete remote sensing image can be obtained.
[0066] In a specific implementation, this embodiment can perform uniform and regular cropping of the multi-band remote sensing image according to the cropping quantity or cropping resolution specified by the user, and obtain multiple tile images corresponding to the specified cropping quantity, or obtain multiple tile images with the same resolution corresponding to the specified cropping resolution, as a tile image set. Preferably, this embodiment can crop the multi-band remote sensing image according to the specified cropping resolution, so as to facilitate subsequent image interpretation.
[0067] Step 130, performing data conversion according to each of the tile images to obtain tensor format data with multi-dimensional features.
[0068] Specifically, a tensor is essentially a container for storing numerical data. In this embodiment, a tensor is a basic data structure used by an integrated classification model. In this embodiment, the tensor format data is essentially the data converted from a tile image into a tensor format, which is stored in a tensor in the form of dimensions.
[0069] In a specific implementation, since the tile image is cropped from a multi-band remote sensing image, the tile image also contains multiple bands corresponding to the multi-band remote sensing image. For a multi-band tile image, the data of each band can correspond to a dimension. This embodiment converts the data of each band in the multi-band tile image into corresponding tensor format data, thereby obtaining tensor format data with multi-dimensional features.
[0070] For example, a four-band tile image with a resolution of 256*256 can be converted into a tensor format data with a size of [256, 256, 4] and four-dimensional features.
[0071] Step 140 , performing image interpretation on the tensor format data through a preset integrated classification model to obtain a tensor format classification result corresponding to the tensor format data.
[0072] In this embodiment, the integrated classification model may also be referred to as a land use classification model or an artificial neural network segmentation model (referred to as segmentation model), etc. The integrated classification model may essentially be composed of multiple neural network models with different functions. For example, the integrated classification model may include but is not limited to: a neural network model with a backbone feature extraction function and a neural network model with an interpretation function. This embodiment does not impose any restrictions on this.
[0073] In a specific implementation, this embodiment inputs tensor format data with multi-dimensional features into a pre-trained integrated classification model. The integrated classification model can perform a series of processing on the tensor format data with multi-dimensional features, such as but not limited to convolution operations, backbone feature extraction, feature fusion, and image interpretation, thereby obtaining a tensor format classification result corresponding to the band of the tile image or the dimension of the tensor format data, thereby realizing the multi-band feature extraction of the remote sensing image. Since the dimension of the tensor format data corresponds to the number of bands of the original multi-band remote sensing image, the tensor format classification result obtained by interpreting the tensor format data can also correspond to the multi-band features of the original multi-band remote sensing image, and can be effectively applied to remote sensing images with any multi-band number.
[0074] Step 150, performing data integration based on the tensor format classification result to obtain the land use type classification result of the target land.
[0075] In this embodiment, the land use type classification result may include but is not limited to a land use type classification result map. Preferably, the land use type classification result map may have the same size as the original multi-band remote sensing image, which is not limited in this embodiment.
[0076] In actual implementation, the tensor format classification results obtained after interpretation can be converted into tile images of corresponding formats by data conversion. The tile images obtained at this time are images corresponding to the tensor format classification results that have completed land use type classification. Since the tensor format classification results are obtained by image interpretation of tensor format data, and the tensor format data is obtained by data conversion based on tile images, the tile images converted according to the tensor format classification results can be spliced to obtain a land use classification result map that completes land use type classification, realizing land use classification of multi-band remote sensing images and improving the accuracy and efficiency of land use classification.
[0077] In the specific implementation, this application can build a simple land use classification system. Through the land use classification system, users only need to input the entire remote sensing image to easily obtain the interpretation results, thereby improving the classification efficiency, thereby solving the problems of low classification accuracy of existing artificial neural networks when facing multi-classification tasks and the overly complicated data set input segmentation process.
[0078] It can be seen that the embodiment of the present application obtains a multi-band remote sensing image of the target land, and performs cropping processing according to the multi-band remote sensing image to obtain a tile image set corresponding to the multi-band remote sensing image, the tile image set contains at least two tile images, and then performs data conversion according to each tile image to obtain tensor format data with multi-dimensional features, and interprets the tensor format data through a preset integrated classification model to obtain a tensor format classification result corresponding to the tensor format data, and then performs data integration according to the tensor format classification result to obtain the land use type classification result of the target land, thereby realizing the multi-band feature extraction and integrated interpretation result of the remote sensing image, and does not limit the number of bands of the multi-band remote sensing image, and can be applied to any remote sensing image with multiple bands or multiple channels, thereby improving the accuracy and efficiency of land use classification, and solving the various problems existing in the existing land use classification technology based on deep learning technology, including but not limited to: ① The problem of being unable to adapt to multi-band remote sensing images, missing remote sensing image information and low classification accuracy; ② The problem of low classification accuracy of artificial neural networks when facing multi-classification tasks and the overly complicated data set input segmentation process.
[0079] Reference Figure 2 , shows a schematic diagram of the steps of a land use type classification method based on multi-band remote sensing images provided by an optional embodiment of the present application. The method may specifically include the following steps:
[0080] Step 210: Acquire remote sensing image data of the target land, wherein the remote sensing image data includes multi-band remote sensing images.
[0081] For the description of step 210, reference may be made to the above step 110, which will not be described in detail in this embodiment.
[0082] Step 220 , performing cropping processing on the multi-band remote sensing image to obtain a tile image set corresponding to the multi-band remote sensing image, wherein the tile image set includes at least two tile images.
[0083] In actual implementation, in order to realize the cropping of multi-band remote sensing images, this embodiment can preset cropping rules to realize reasonable cropping of multi-band remote sensing images. Among them, the cropping rules may include but are not limited to quantity cropping rules and resolution cropping rules. Specifically, the quantity cropping rule can be to crop the multi-band remote sensing images according to the specified cropping quantity, wherein the cropping of the multi-band remote sensing images can be uniform and regular, and of course it can also be non-uniform, and this embodiment does not limit this. In this embodiment, the cropping quantity obtained by cropping the multi-band remote sensing images can be the same as the specified cropping quantity, and the resolutions of multiple tile images can be the same or different. Preferably, according to the specified cropping quantity, the multi-band remote sensing images are uniformly and regularly cropped to obtain multiple tile images with the same resolution, thereby facilitating subsequent processing; the resolution cropping rule, that is, according to the specified resolution, the multi-band remote sensing images are uniformly and regularly cropped to obtain multiple tile images with the same resolution.
[0084] In an optional embodiment, the embodiment of the present application performs cropping processing on the multi-band remote sensing image to obtain a tile image set corresponding to the multi-band remote sensing image, which may specifically include: obtaining preset cropping rules; and cropping the multi-band remote sensing image according to the cropping rules to obtain at least two tile images with the same resolution.
[0085] Step 230: determine each band of the multi-band remote sensing image.
[0086] Step 240 , for each band, performing matrix conversion on the tile image to obtain a numerical matrix corresponding to each band in the tile image.
[0087] Step 250, performing tensor format conversion based on the numerical matrix to obtain tensor format data with multi-dimensional features.
[0088] A unified description of steps 230 to 250 is given as follows:
[0089] In the specific implementation, the tile image contains the data of each band of the multi-band remote sensing image. To facilitate subsequent model processing, for the data of each band, this embodiment converts the data in each band into a numerical matrix by performing matrix conversion on the tile image, and can convert the numerical matrix into a tensor format, thereby obtaining tensor format data with multi-dimensional features, thereby realizing the storage of the data of each band in the form of dimensions in the tensor to meet the input requirements of the artificial neural network model.
[0090] Step 260, extracting backbone features from the tensor format data through the integrated classification model to obtain fused feature data.
[0091] In this embodiment, the integrated classification model, as a neural network model that aggregates multiple functions, may include a U-Net network model and a VGG16 (Visual Geometry Group) neural network model, that is, a backbone feature extraction structure of VGG16 is added to the U-Net network model to construct an artificial neural network model for multi-channel remote sensing image interpretation as an integrated classification model, or a segmentation model, etc.
[0092] In a specific implementation, this embodiment inputs the tensor format data into the integrated classification model, uses VGG16 of the integrated classification model to extract the backbone features, and obtains multiple effective feature layers corresponding to the number of channels, that is, multiple effective feature layers corresponding to the number of bands. Then, the effective feature layers are used for feature fusion to obtain the final feature layer as fused feature data. The fused feature data can characterize the multi-band features corresponding to the multi-band tile image, which has the same size as the input data, so that the subsequent interpretation and the re-conversion of the interpreted results into tile images can be facilitated.
[0093] In an optional embodiment, the above-mentioned extracting backbone features of the tensor format data through the integrated classification model to obtain fused feature data may specifically include: determining the number of channel convolutions based on the number of bands of the multi-band remote sensing image; based on the channel convolution number, convolving the tensor format data through the integrated classification model to obtain a valid feature layer corresponding to the channel convolution number; sampling and stacking according to the valid feature layer to obtain fused feature data.
[0094] In this embodiment, the remote sensing image data with multi-channel feature tensor format is input into the backbone feature structure of VGG16 in the integrated classification model, and backbone feature extraction is realized by using VGG16. Specifically, VGG16 can perform convolution and maximum pooling, backbone feature extraction, etc. according to the number of channels of the tensor format data input by the user, and can perform downsampling to obtain multiple preliminary effective feature layers, and then the obtained multiple preliminary effective feature layers are subjected to enhanced feature extraction in the U-net network in the integrated classification model, and the multiple preliminary effective feature layers are respectively upsampled by two times, and are stacked with the preliminary effective feature layers in sequence starting from the last feature layer, so that the feature layer finally obtained has the same size as the input data, thereby obtaining fused feature data.
[0095] In actual implementation, before the segmentation model is actually used to extract backbone features and interpret images for tensor format data, the present embodiment can also pre-train the segmentation model. On the basis of the existing U-net network model, the VGG16 backbone feature extraction structure is added to optimize the artificial neural network model training parameter adjustment process.
[0096] In an optional embodiment, before the embodiment of the present application performs image interpretation on the tensor format data through a preset integrated classification model, it may also specifically include: acquiring the original remote sensing image of the sample area; performing band preprocessing based on the original remote sensing image to obtain multi-band remote sensing image samples and sample labels; cropping and enhancing the multi-band remote sensing image samples according to the sample labels to obtain a tile sample label data set with multi-band features, wherein the tile sample label data set includes a sample label file and remote sensing cropped image samples corresponding one to the sample label file; performing data conversion based on the tile sample label data set to obtain sample tensor format data and sample test data with multi-dimensional features; performing model training based on the sample tensor format data to obtain the integrated classification model; and performing segmentation testing on the integrated classification model based on the sample test data until the integrated classification model converges.
[0097] In a specific implementation, this embodiment can conveniently complete model training through the constructed land use classification system. The model training process can refer to the following: Figure 3 The artificial neural network model flow chart shown. Specifically, this embodiment obtains original remote sensing images of sample areas such as urban and township land, performs band preprocessing such as splicing, shadow removal, and noise removal on the remote sensing images, integrates the remote sensing image bands, and forms multi-band remote sensing images as multi-band remote sensing image samples. At the same time, through manual visual interpretation of remote sensing images, the classified samples required for the study are marked in the plot base map to form sample labels. Subsequently, the corresponding multi-band remote sensing images are cropped according to the sample labels to form a sample label file consisting of multiple tile images of the same resolution and the corresponding cropped remote sensing image files, wherein the sample label file (label) corresponds to the remote sensing image file (image) sort name one-to-one, and the two together constitute a tile sample label data set containing multi-band features.
[0098] Exemplarily, to achieve model training, in this embodiment, the specific implementation process of model training may include but is not limited to: obtaining electronic computing equipment, collecting remote sensing images and establishing data sets, training and testing deep learning models, and demonstrating urban land use classification effects.
[0099] Regarding obtaining electronic computing devices, this example uses an electronic computer as the electronic computing device, and the hardware involved includes but is not limited to: 1 portable notebook: configured with a GeForceRTX3070Ti graphics card, a 16-inch monitor, 1 power supply, 1 keyboard and mouse, etc.
[0100] In this example, remote sensing images are collected and data sets are established to produce land classification sample data sets. Areas A, B, C, and D of Guangx City are used as experimental areas. The remote sensing images used are mainly derived from the 2022 Gaofen-2 remote sensing satellite images, which have four bands: red, green, blue, and near-infrared. Then, combined with the current status of land resources in the main urban area of Guangx City and the readability of remote sensing images, team members manually marked sample labels on ArcGIS software to form 11 types of sample label files. Each type of sample label has a certain number of labels, including 55 forests, 43 gardens, 31 grasslands, 40 cultivated lands, 29 unused lands, 164 residential lands, 122 commercial and service lands, 32 transportation lands, 126 industrial and mining storage lands, 107 public management and service lands, and 81 water bodies. Subsequently, the corresponding multi-band remote sensing images are cropped according to the sample labels to form a sample label file consisting of multiple tile images of the same resolution and the corresponding cropped remote sensing image files. The sample label file (label) corresponds to the remote sensing image file (image) in a one-to-one order. The two together constitute a tile sample label dataset of the main urban area of Guangx City with multi-band features.
[0101] In addition, the embodiment of the present application can also perform image enhancement on tile sample label datasets with multi-band features. Users can select image enhancement methods according to actual needs, where image enhancement may include but is not limited to: ① Rotation: Random rotation with the image center as the rotation center; ② Translation: Random translation of the image along the X-axis or Y-axis; ③ Flip: Swap the x-axis and y-axis of the image. By performing image enhancement on the tile sample label dataset, the robustness of subsequent model training can be effectively improved.
[0102] In this embodiment, the tile sample label data set containing multi-band features that increases in number after image enhancement can be divided into a training set and a test set in proportion. Then the training set and the test set are converted into a tensor data format with multi-dimensional features. In the land use classification system, the data in each band of the remote sensing image will be converted into a numerical matrix and stored in a tensor in the form of a dimension (for example, a four-band image with a resolution of 256*256 will be converted into a tensor of size [256,256,4]) to meet the input requirements of the artificial neural network model. Subsequently, the converted tensor format training set and test set can be placed in the vgg16 feature extraction network and U-net network structure for multi-band features for training and testing, and an artificial neural network segmentation model that can achieve high-precision classification is obtained as an integrated classification model. In addition, the land use classification system can also convert the remaining tile images to be classified into a tensor format, and then put them into the trained segmentation model to interpret the classification results in tensor format.
[0103] Therefore, in order to solve the problem of cumbersome preparation work for traditional artificial intelligence model training, the embodiment of the present application integrates functions such as sample cutting and graphic enhancement, and constructs a land use type classification system based on deep learning, which can be directly used for multi-band remote sensing image interpretation. This embodiment optimizes the artificial neural network model training parameter adjustment process by adding the VGG16 backbone feature extraction structure on the basis of the U-net network model, so that the U-net network model supports more than three channels of remote sensing image data set input, and users can customize the number of input channels according to their needs. It can be seen that this application integrates the data set preparation function and improves the existing artificial neural network model to improve the accuracy of the artificial neural network when facing multi-classification tasks, thereby improving the accuracy and efficiency in land use classification, thereby improving the accuracy of urban land use classification.
[0104] Optionally, an embodiment of the present application performs model training based on the sample tensor format data to obtain the integrated classification model, which may specifically include: constructing a first initial model and a second initial model; performing convolution training on the sample tensor format data through the first initial model to obtain effective feature layer samples and a target feature extraction model for the number of channels corresponding to the sample tensor format data; performing interpretation training on the second initial model based on the effective feature layer samples to obtain a target segmentation model; and performing model integration based on the target feature extraction model and the target segmentation model to obtain the integrated classification model.
[0105] In the specific implementation, Figure 3 As shown, the first initial model can be a VGG16 neural network model, and the second initial model can be a U-Net neural network model. By inputting multi-dimensional sample tensor format data into VGG16 for convolution training and backbone feature extraction, effective feature layer samples and trained VGG16 are obtained. Subsequently, the effective feature layer samples are input into U-Net for upsampling and feature fusion operations, and then interpretation training is performed to obtain a trained U-Net neural network model. On the basis of the U-Net neural network model, a VGG16 neural network model is added to perform backbone feature extraction, thereby obtaining a trained segmentation model.
[0106] Exemplarily, the artificial neural network model used in the interpretation of this system is based on the U-Net network model, adding the backbone feature extraction structure of VGG16 to construct an artificial neural network model for multi-channel remote sensing image interpretation. The specific process is as follows: the remote sensing image data with a data size of [256,256,4] and a multi-channel feature tensor format is sent to the backbone feature structure of VGG16. VGG16 can perform convolution and maximum pooling on the input data, extract backbone features, and perform downsampling to obtain five preliminary effective feature layers (conv). The five effective feature layers can be represented by conv1-conv5. Among them, the convolution operation results of conv1-conv5 can be referred to Figure 4 As shown, the convolution operation and feature extraction process of conv1-conv5 are as follows:
[0107] conv1: perform two [3,3] 64-channel convolutions to obtain a preliminary effective feature layer of [256,256,64], and then perform 2X2 maximum pooling to obtain a feature layer of [128,128,64];
[0108] conv2: perform two [3,3] 128-channel convolutions to obtain a preliminary effective feature layer of [128,128,128], and then perform 2X2 maximum pooling to obtain a feature layer of [64,64,128];
[0109] conv3: perform three [3,3] 256-channel convolutions to obtain a preliminary effective feature layer of [64,64,256], and then perform 2X2 maximum pooling to obtain a feature layer of [32,32,256];
[0110] conv4: perform three [3,3] 512-channel convolutions to obtain a preliminary effective feature layer of [32,32,512], and then perform 2X2 maximum pooling to obtain a feature layer of [16,16,512];
[0111] conv5: Perform three [3,3] 512-channel convolutions to obtain a preliminary effective feature layer of [16,16,512].
[0112] Subsequently, the five preliminary effective feature layers obtained can be upsampled twice in the enhanced feature extraction structure of the U-net network, and the fifth feature layer can be stacked with the preliminary effective feature layer in sequence, so that the final feature layer has the same size as the input data, thereby obtaining fused feature data.
[0113] Furthermore, the number of channels of the final feature layer can be adjusted to the number of classification categories (num_classes) required by the user through a 1x1 convolution, which facilitates subsequent visualization operations of the system.
[0114] Step 270, classify and interpret the fused feature data to obtain a classification result in a tensor format.
[0115] Step 280: Perform image conversion according to the tensor format classification result to obtain a tile image in a target format.
[0116] Step 290, performing image stitching based on the target format tile image to obtain a classification result map of the target land as the land use type classification result.
[0117] A unified description of steps 270 to 290 is given as follows:
[0118] The present embodiment can pre-set the image format, and the present embodiment does not limit the image format, and the corresponding image format can be set according to actual use requirements. Preferably, the image format can be png (Portable Network Graphics, portable network graphics) format.
[0119] In the specific implementation, the fusion feature data is classified and interpreted by integrating the classification model to obtain the classification result in tensor format as the classification result in tensor format. Then, the classification result in tensor format can be converted into a tile image in png format according to the preset image format, and reassembled into the classification result map of the original size according to the overall size of the image before cropping, as the land use type classification result, and output to the user.
[0120] Therefore, the embodiment of the present application adds functions such as dividing data sets and integrating interpretation results to the deep learning land use classification algorithm, and proposes a system and algorithm for effective land use classification based on deep learning for multi-classification tasks to support multi-channel remote sensing image input. Compared with the problems of low classification accuracy of artificial neural networks in the prior art when facing multi-classification tasks and overly complicated data set input segmentation process, the embodiment of the present application can effectively improve the accuracy and efficiency of land use classification. It can be seen that the embodiment of the present application achieves: 1. A simple land use classification system is adopted, so that users only need to input the entire remote sensing image to easily obtain the interpretation result, thereby improving the classification efficiency; 2. The traditional complicated classification system that relies on manual sorting of different sample data sets is transformed into a deep learning classification system that systematically sorts data sets autonomously, which simplifies the steps of data set sorting, reduces the time spent on training models, adjusting models and final classification, and improves the efficiency of land use classification;
[0121] ③ Based on the U-net network and VGG16, the backbone feature extraction structure and training parameter adjustment of the model were modified. By adding the VGG16 backbone feature extraction structure to optimize the training parameter adjustment process of the artificial neural network model, the U-net network model supports the input of remote sensing image data sets with more than three channels. Users can customize the number of input channels according to their needs, and no matter how many channels the user sets for their remote sensing image data sets, the model can be trained without having to adjust the model structure according to different data sets. Users can add feature maps to the data set in the form of adding bands according to their needs, thereby improving the accuracy of artificial neural networks when facing multi-classification tasks.
[0122] In summary, the embodiment of the present application obtains a multi-band remote sensing image of the target land, performs cropping processing according to the multi-band remote sensing image, obtains a tile image set corresponding to the multi-band remote sensing image, then determines each band of the multi-band remote sensing image, and for each band, performs matrix conversion on the tile image in the tile image set to obtain a numerical matrix corresponding to each band in the tile image, performs tensor format conversion based on the numerical matrix, obtains tensor format data with multi-dimensional features, and then extracts the main features of the tensor format data by integrating the classification model to obtain fused feature data, so as to obtain the fused feature data according to the fused feature. The data is classified and interpreted to obtain the classification result in tensor format, and image conversion is performed according to the tensor format classification result to obtain the target format tile image. Image stitching is performed based on the target format tile image to obtain the classification result map of the target land as the land use type classification result, thereby realizing the multi-band feature extraction and integrated interpretation results of the remote sensing image, without limiting the number of bands of the multi-band remote sensing image, and can be applied to any remote sensing image with multiple bands or multiple channels, thereby improving the accuracy and efficiency of land use classification and solving various problems existing in the existing land use classification technology based on deep learning technology.
[0123] It should be noted that, for the purpose of simple description, the method embodiments are expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited to the described order of actions, because according to the embodiments of the present application, certain steps may be performed in other orders or simultaneously.
[0124] like Figure 5 As shown, the embodiment of the present application also provides a land use type classification system 500 based on multi-band remote sensing images, including:
[0125] A multi-band remote sensing image data acquisition module 510 is used to acquire remote sensing image data of the target land, wherein the remote sensing image data includes multi-band remote sensing images;
[0126] A cropping processing module 520 is used to perform cropping processing according to the multi-band remote sensing image to obtain a tile image set corresponding to the multi-band remote sensing image, wherein the tile image set includes at least two tile images;
[0127] A data conversion module 530, configured to perform data conversion according to each of the tile images to obtain tensor format data with multi-dimensional features;
[0128] An image interpretation module 540 is used to perform image interpretation on the tensor format data through a preset integrated classification model to obtain a tensor format classification result corresponding to the tensor format data;
[0129] The data integration module 550 is used to perform data integration according to the tensor format classification result to obtain the land use type classification result of the target land.
[0130] Optionally, the clipping processing module 520 includes:
[0131] A clipping rule acquisition submodule is used to obtain preset clipping rules;
[0132] The image cropping submodule is used to crop the multi-band remote sensing image according to the cropping rules to obtain at least two tile images with the same resolution.
[0133] Optionally, the data conversion module 530 includes:
[0134] A band determination submodule, used for determining each band of the multi-band remote sensing image;
[0135] A matrix conversion submodule, used for performing matrix conversion on the tile image for each band to obtain a numerical matrix corresponding to each band in the tile image;
[0136] The format conversion submodule is used to perform tensor format conversion based on the numerical matrix to obtain tensor format data with multi-dimensional features.
[0137] Optionally, the image interpretation module 540 includes:
[0138] A feature extraction submodule, used for extracting main features from the tensor format data through the integrated classification model to obtain fused feature data;
[0139] The classification and interpretation submodule is used to perform classification and interpretation according to the fused feature data to obtain a classification result in a tensor format.
[0140] Optionally, the feature extraction submodule includes:
[0141] A channel convolution number determining unit, used to determine the channel convolution number based on the number of bands of the multi-band remote sensing image;
[0142] A convolution unit, configured to convolve the tensor format data through the integrated classification model based on the channel convolution number to obtain a valid feature layer corresponding to the channel convolution number;
[0143] The sampling and stacking unit is used to perform sampling and stacking according to the effective feature layer to obtain fused feature data.
[0144] Optionally, the data integration module 550 includes:
[0145] An image conversion submodule, used to perform image conversion according to the tensor format classification result to obtain a tile image in a target format;
[0146] The image stitching submodule is used to perform image stitching based on the target format tile image to obtain a classification result map of the target land as the land use type classification result.
[0147] Optionally, the land use type classification system 500 based on multi-band remote sensing images further includes:
[0148] The original remote sensing image acquisition module is used to obtain the original remote sensing image of the sample area;
[0149] A preprocessing module, used for performing band preprocessing based on the original remote sensing image to obtain multi-band remote sensing image samples and sample labels;
[0150] A cropping and enhancement module, used for cropping and enhancing the multi-band remote sensing image samples according to the sample labels to obtain a tile sample label data set with multi-band features, wherein the tile sample label data set includes a sample label file and a remote sensing cropped image sample corresponding to the sample label file in a one-to-one manner;
[0151] A data conversion module, used to perform data conversion based on the tile sample label data set to obtain sample tensor format data and sample test data with multi-dimensional features;
[0152] A model training module, used for performing model training according to the sample tensor format data to obtain the integrated classification model;
[0153] The segmentation test module is used to perform segmentation test on the integrated classification model based on the sample test data until the integrated classification model converges.
[0154] Optionally, the model training module includes:
[0155] A model building submodule, used to build a first initial model and a second initial model;
[0156] A convolution training submodule, used to perform convolution training on the sample tensor format data through the first initial model to obtain effective feature layer samples and a target feature extraction model for the number of channels corresponding to the sample tensor format data;
[0157] An interpretation training submodule, used for performing interpretation training on the second initial model based on the effective feature layer samples to obtain a target segmentation model;
[0158] The model integration submodule is used to perform model integration according to the target feature extraction model and the target segmentation model to obtain the integrated classification model.
[0159] It should be noted that the land use type classification system based on multi-band remote sensing images provided in the embodiments of the present application can execute the land use type classification method based on multi-band remote sensing images provided in any embodiment of the present application, and has the corresponding functions and beneficial effects of the execution method.
[0160] In a specific implementation, the above-mentioned land use type classification system based on multi-band remote sensing images can be integrated into a device, so that the device can obtain tensor format data from multi-band remote sensing images, and then interpret the tensor format data by integrating the classification model, and perform data integration on the tensor format classification results to obtain land use type classification results, as an electronic device, to improve the accuracy and efficiency of land use classification. The electronic device can be composed of two or more physical entities, or it can be composed of one physical entity, such as the electronic device can be a personal computer (PC), a computer, a server, etc., and the embodiments of the present application do not impose specific restrictions on this.
[0161] like Figure 6As shown, an embodiment of the present application provides an electronic device, including a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114; the memory 113 is used to store computer programs; the processor 111 is used to implement the steps of the land use type classification method based on multi-band remote sensing images provided by any of the aforementioned method embodiments when executing the program stored in the memory 113. Exemplarily, the steps of the land use type classification method based on multi-band remote sensing images may include the following steps: obtaining remote sensing image data of the target land, the remote sensing image data including a multi-band remote sensing image; performing cropping processing based on the multi-band remote sensing image to obtain a tile image set corresponding to the multi-band remote sensing image, the tile image set including at least two tile images; performing data conversion based on each of the tile images to obtain tensor format data with multi-dimensional features; performing image interpretation on the tensor format data through a preset integrated classification model to obtain a tensor format classification result corresponding to the tensor format data; performing data integration based on the tensor format classification result to obtain a land use type classification result of the target land.
[0162] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the land use type classification method based on multi-band remote sensing images provided in any of the aforementioned method embodiments are implemented.
[0163] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0164] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.
Claims
1. A land use type classification method based on multi-band remote sensing images, characterized in that: include: Acquiring remote sensing image data of the target land, wherein the remote sensing image data includes multi-band remote sensing images; Performing cropping processing on the multi-band remote sensing image to obtain a tile image set corresponding to the multi-band remote sensing image, wherein the tile image set includes at least two tile images; Performing data conversion according to each of the tile images to obtain tensor format data with multi-dimensional features; Performing image interpretation on the tensor format data through a preset integrated classification model to obtain a tensor format classification result corresponding to the tensor format data; Data integration is performed based on the tensor format classification results to obtain the land use type classification results of the target land.
2. The method according to claim 1, characterized in that The step of performing cropping processing on the multi-band remote sensing image to obtain a tile image set corresponding to the multi-band remote sensing image includes: Get the preset cropping rules; According to the cropping rule, the multi-band remote sensing image is cropped to obtain at least two tile images with the same resolution.
3. The method according to claim 1, characterized in that: The data conversion is performed according to each tile image to obtain tensor format data with multi-dimensional features, including: Determine each band of the multi-band remote sensing image; For each band, performing matrix conversion on the tile image to obtain a numerical matrix corresponding to each band in the tile image; A tensor format conversion is performed based on the numerical matrix to obtain tensor format data with multi-dimensional features.
4. The method according to claim 1, characterized in that: The image interpretation of the tensor format data by using a preset integrated classification model to obtain a tensor format classification result corresponding to the tensor format data includes: Extracting backbone features from the tensor format data using the integrated classification model to obtain fused feature data; Classification and interpretation are performed based on the fused feature data to obtain a classification result in a tensor format.
5. The method according to claim 4, characterized in that The extracting backbone features of the tensor format data by the integrated classification model to obtain fused feature data includes: Determining the number of channel convolutions based on the number of bands of the multi-band remote sensing image; Based on the channel convolution number, convolving the tensor format data through the integrated classification model to obtain a valid feature layer corresponding to the channel convolution number; Sampling and stacking are performed according to the effective feature layer to obtain fused feature data.
6. The method according to any one of claims 1 to 5, characterized in that: The data integration is performed according to the tensor format classification result to obtain the land use type classification result of the target land, including: Performing image conversion according to the tensor format classification result to obtain a target format tile image; Image stitching is performed based on the target format tile images to obtain a classification result map of the target land as the land use type classification result.
7. The method according to any one of claims 1 to 6, characterized in that: Before performing image interpretation on the tensor format data through a preset integrated classification model, the method further includes: Obtain original remote sensing images of the sample area; Perform band preprocessing based on the original remote sensing image to obtain multi-band remote sensing image samples and sample labels; The multi-band remote sensing image samples are cropped and enhanced according to the sample labels to obtain a tile sample label data set with multi-band features, wherein the tile sample label data set includes a sample label file and remote sensing cropped image samples corresponding to the sample label files in a one-to-one manner; Performing data conversion based on the tile sample label data set to obtain sample tensor format data and sample test data with multi-dimensional features; Performing model training according to the sample tensor format data to obtain the integrated classification model; The integrated classification model is subjected to segmentation testing based on the sample test data until the integrated classification model converges.
8. The method according to claim 7, characterized in that The step of performing model training according to the sample tensor format data to obtain the integrated classification model includes: constructing a first initial model and a second initial model; Performing convolution training on the sample tensor format data through the first initial model to obtain effective feature layer samples and a target feature extraction model for the number of channels corresponding to the sample tensor format data; Performing interpretation training on the second initial model based on the effective feature layer samples to obtain a target segmentation model; Model integration is performed according to the target feature extraction model and the target segmentation model to obtain the integrated classification model.
9. A land use type classification system based on multi-band remote sensing images, characterized in that: include: A multi-band remote sensing image data acquisition module is used to acquire remote sensing image data of the target land, wherein the remote sensing image data includes multi-band remote sensing images; A cropping processing module, used for performing cropping processing according to the multi-band remote sensing image to obtain a tile image set corresponding to the multi-band remote sensing image, wherein the tile image set includes at least two tile images; A data conversion module, used for performing data conversion according to each of the tile images to obtain tensor format data with multi-dimensional features; An image interpretation module, used to perform image interpretation on the tensor format data through a preset integrated classification model to obtain a tensor format classification result corresponding to the tensor format data; The data integration module is used to perform data integration based on the tensor format classification result to obtain the land use type classification result of the target land.
10. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor is used to implement the steps of the land use type classification method based on multi-band remote sensing images as described in any one of claims 1 to 8 when executing the program stored in the memory.
Citation Information
Cited By
Green tide algae biomass estimation method and system based on aerial remote sensing interpretation
CN120495873A
