Remote sensing image recognition method, device, equipment and computer-readable storage medium

Abstract

The present application provides a remote sensing image recognition method, device, equipment and computer-readable storage medium; the method includes: obtaining an initial remote sensing image to be processed; inputting the initial remote sensing image into a preset first image prediction model and a preset second image prediction model to obtain a first remote sensing prediction map and a second remote sensing prediction map; performing clustering analysis on the initial remote sensing image to obtain a clustered remote sensing image; fusing and recognizing the first remote sensing prediction map, the second remote sensing prediction map and the clustered remote sensing image to obtain the target ground objects included in the initial remote sensing image; in the technical solution of the present application, two different image prediction models are preset, and the initial remote sensing image is combined through the two image prediction models and image clustering analysis, so that when analyzing the remote sensing image, both the overall image and each pixel point in the image are considered, making the remote sensing image recognition more accurate.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular, to a method, device, equipment and computer-readable storage medium for remote sensing image recognition. Background Art

[0002] The ground objects in remote sensing images are complex and diverse. The on-map representations of the same ground object may vary greatly, and the on-map representations of different ground objects may also be relatively similar, which poses a great challenge to the recognition of remote sensing images.

[0003] Some ground objects in remote sensing images have regular shapes and clear boundaries, and are easy to segment, such as buildings, roads, etc. However, some growing ground objects such as forests, grasslands, etc. or ground objects greatly affected by growing ground objects such as sand lands, bare lands, saline-alkali lands, etc. will change greatly in both shape and sparsity due to various reasons such as time and season, resulting in the usually staggered distribution of different ground objects in remote sensing images, the blurred boundaries of different ground objects in remote sensing images, and the relatively difficult recognition of ground objects in remote sensing images; for the current recognition of different ground objects in remote sensing images, deep learning networks are adopted, and the samples for training and learning of deep learning networks are manually labeled. Although the recognition effect of ground objects in remote sensing images by deep learning networks is good, it cannot completely distinguish each pixel accurately, and the recognition accuracy of remote sensing images is not high. Summary of the Invention

[0004] This application provides a method, device, equipment and computer-readable storage medium for remote sensing image recognition, aiming to solve the technical problem that the existing remote sensing image recognition cannot analyze pixel points and has low recognition accuracy.

[0005] On the one hand, this application provides a method for remote sensing image recognition, and the method includes:

[0006] Obtain an initial remote sensing image to be processed;

[0007] Input the initial remote sensing image into a preset first image prediction model and a preset second image prediction model to obtain a first remote sensing prediction map and a second remote sensing prediction map;

[0008] Perform clustering analysis on the initial remote sensing image to obtain a clustered remote sensing image;

[0009] Fusion-recognize the first remote sensing prediction map, the second remote sensing prediction map and the clustered remote sensing image to obtain the target ground objects included in the initial remote sensing image.

[0010] In some embodiments of this application, before inputting the initial remote sensing image into a preset first image prediction model and a preset second image prediction model to obtain a first remote sensing prediction map and a second remote sensing prediction map, the method includes:

[0011] Obtain a large amount of remote sensing images, add marks to each frame of sample remote sensing images in the large amount of remote sensing images as image training samples, and summarize the image training samples to form an image training sample set;

[0012] Extract image training samples from the image training sample set, and iteratively train a first encoder and a first decoder through the image training samples to obtain a preset first image prediction model;

[0013] Extract image training samples from the image training sample set, and iteratively train a second encoder and a second decoder through the image training samples to obtain a preset second image prediction model.

[0014] In some embodiments of the present application, the fusing and identifying the first remote sensing prediction map, the second remote sensing prediction map and the clustered remote sensing image to obtain the target ground objects included in the initial remote sensing image includes:

[0015] Perform fusion dilation processing on the first remote sensing prediction map and the second remote sensing prediction map to obtain a target remote sensing prediction map;

[0016] Perform fusion recognition on the target remote sensing prediction map and the clustered remote sensing image to obtain the target ground objects included in the initial remote sensing image.

[0017] In some embodiments of the present application, the performing fusion dilation processing on the first remote sensing prediction map and the second remote sensing prediction map to obtain a target remote sensing prediction map includes:

[0018] Extract first feature data of the first remote sensing prediction map, extract second feature data of the second remote sensing prediction map, add the first feature data and the second feature data after encoding transformation to obtain encoded feature data;

[0019] Process the encoded feature data according to a preset maximum independent variable function, and fuse the processed encoded feature data to obtain a target remote sensing prediction map.

[0020] In some embodiments of the present application, the processing the encoded feature data according to a preset maximum independent variable function, and fusing the processed encoded feature data to obtain a target remote sensing prediction map includes:

[0021] Process the encoded feature data according to a preset maximum independent variable function, and obtain a classification label corresponding to the processed encoded feature data;

[0022] Compare the classification label with each pixel point in the corresponding area of the preset standard structural element;

[0023] If the pixel is the same as the standard structural element, the pixel is retained;

[0024] If the pixel is different from the standard structural element, dilation processing is performed with the pixel as the center according to the standard structural element to form dilated pixels;

[0025] The retained pixels and the dilated pixels are summarized to obtain a target remote sensing prediction map.

[0026] In some embodiments of the present application, the fusing and recognizing the target remote sensing prediction map and the clustered remote sensing image to obtain the target ground objects included in the initial remote sensing image includes:

[0027] Obtain the first subscript of the first matrix corresponding to the target remote sensing prediction map, and obtain the second subscript of the second matrix corresponding to the clustered remote sensing image;

[0028] Fuse the target remote sensing prediction map and the clustered remote sensing image according to the first subscript and the second subscript to obtain a fused remote sensing image, and recognize the fused remote sensing image to obtain the target ground objects included in the initial remote sensing image.

[0029] In some embodiments of the present application, the fusing and recognizing the first remote sensing prediction map, the second remote sensing prediction map and the clustered remote sensing image to obtain the target ground objects included in the initial remote sensing image includes:

[0030] Fuse the first remote sensing prediction map, the second remote sensing prediction map and the clustered remote sensing image to obtain a fused remote sensing image;

[0031] Obtain the pixel type of the fused remote sensing image and the number of types of the pixel type;

[0032] Input the pixel type and the number of types into a preset intersection over union calculation formula to obtain the average intersection over union of the image;

[0033] Obtain the ground object classification corresponding to the average intersection over union of the image, and use the ground object corresponding to the ground object classification as the target ground object included in the initial remote sensing image.

[0034] In some embodiments of the present application, the fusing and recognizing the first remote sensing prediction map, the second remote sensing prediction map and the clustered remote sensing image to obtain the target ground objects included in the initial remote sensing image includes:

[0035] Fuse the first remote sensing prediction map and the second remote sensing prediction map to obtain a target remote sensing prediction map;

[0036] Fuse the target remote sensing prediction map and the clustered remote sensing image to obtain a fused remote sensing image, and calculate the intersection over union (IoU) of each pixel type in the fused remote sensing image;

[0037] Calculate the average value of the IoU corresponding to all pixel types to obtain the average IoU corresponding to the pixel type, and compare the average IoU with the average IoU obtained in the previous time;

[0038] If the average IoU is greater than the average IoU obtained in the previous time, then fuse the fused remote sensing image with the first remote sensing prediction map and the second remote sensing prediction map to obtain a new remote sensing prediction map, and fuse the new remote sensing prediction map with the previous clustered remote sensing image to obtain an iterative new fused remote sensing image;

[0039] Obtain the newly formed iterative fused remote sensing image, and use the land cover classification corresponding to the average IoU of the new fused remote sensing image as the target land cover included in the initial remote sensing image.

[0040] On the other hand, the present application also provides a remote sensing image recognition device, which includes:

[0041] An image acquisition module, configured to acquire an initial remote sensing image to be processed;

[0042] An input prediction module, configured to input the initial remote sensing image into a preset first image prediction model and a preset second image prediction model to obtain a first remote sensing prediction map and a second remote sensing prediction map;

[0043] A clustering analysis module, configured to perform clustering analysis on the initial remote sensing image to obtain a clustered remote sensing image;

[0044] A fusion recognition module, configured to fuse and recognize the first remote sensing prediction map, the second remote sensing prediction map and the clustered remote sensing image to obtain the target land cover included in the initial remote sensing image.

[0045] On the other hand, the present application also provides a remote sensing image recognition device, which includes:

[0046] One or more processors;

[0047] A memory; and

[0048] One or more applications, where the one or more applications are stored in the memory and are configured to be executed by the processor to implement the remote sensing image recognition method.

[0049] On the other hand, the present application also provides a computer-readable storage medium, on which a computer program is stored. The computer program is loaded by a processor to execute the steps in the remote sensing image recognition method described above.

[0050] In the technical solution of the present application, an initial remote sensing image to be processed is obtained; the initial remote sensing image is input into a preset first image prediction model and a preset second image prediction model to obtain a first remote sensing prediction map and a second remote sensing prediction map; the initial remote sensing image is subjected to clustering analysis to obtain a clustered remote sensing image; the first remote sensing prediction map, the second remote sensing prediction map and the clustered remote sensing image are fused and recognized to obtain the target ground objects included in the initial remote sensing image. In the embodiments of the present application, two different image prediction models are preset. By combining the two image prediction models and image clustering analysis with the initial remote sensing image, when analyzing the remote sensing image, both the overall image and each pixel point in the image are considered, making the remote sensing image recognition more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those skilled in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0052] Figure 1 is a schematic diagram of the scenario of the remote sensing image recognition method provided by the embodiment of the present application;

[0053] Figure 2 is a schematic flow chart of an embodiment of the remote sensing image recognition method provided by the embodiment of the present application;

[0054] Figure 3 is a schematic diagram of the initial remote sensing image in an embodiment of the remote sensing image recognition method provided by the embodiment of the present application;

[0055] Figure 4 is a schematic flow chart of an embodiment for pre-constructing the first image prediction model and the second image prediction model in the remote sensing image recognition method of the embodiment of the present application;

[0056] Figure 5 is a schematic flow chart of the fusion and recognition of the first remote sensing prediction map, the second remote sensing prediction map and the clustered remote sensing image in the remote sensing image recognition method provided by the embodiment of the present application;

[0057] Figure 6 is a schematic flow chart of the specific process of remote sensing image recognition in the remote sensing image recognition method provided by the embodiment of the present application;

[0058] Figure 7It is a schematic diagram of a specific scenario for one-hot encoding conversion in the remote sensing image recognition method provided in the embodiment of this application;

[0059] Figure 8 It is a classification schematic diagram of the fused remote sensing image in the remote sensing image recognition method provided in the embodiment of this application;

[0060] Figure 9 It is a schematic diagram of the process of an embodiment of iterative recognition in the remote sensing image recognition method in the embodiment of this application;

[0061] Figure 10 It is a schematic diagram of a specific scenario of an embodiment of iterative recognition in the remote sensing image recognition method in the embodiment of this application;

[0062] Figure 11 It is a schematic diagram of the structure of an embodiment of the remote sensing image recognition device provided in the embodiment of this application;

[0063] Figure 12 It is a schematic diagram of the structure of an embodiment of the remote sensing image recognition device provided in the embodiment of this application. Detailed implementation manners

[0064] Next, the technical solutions in the embodiments of this application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope included in this application.

[0065] In the description of this application, it should be understood that the terms "first" and "second" are only used for descriptive purposes, and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of the described features. In the description of this application, "a plurality of" means two or more, unless otherwise specifically defined.

[0066] In this application, the term "exemplary" is used to mean "serving as an example, illustration, or instance". Any embodiment described as "exemplary" in this application is not necessarily to be construed as more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use this application. In the following description, details are set forth for purposes of explanation. It should be understood that those of ordinary skill in the art can recognize that this application can be implemented without these specific details. In other instances, well-known structures and processes are not elaborated in detail so as not to obscure the description of this application with unnecessary details. Therefore, this application is not intended to be limited to the embodiments shown, but rather to be in line with the broadest scope consistent with the principles and features disclosed in this application.

[0067] An embodiment of this application provides a remote sensing image recognition method, apparatus, device, and computer-readable storage medium, which will be described in detail below.

[0068] The remote sensing image recognition method in the embodiment of this application is applied to a remote sensing image recognition apparatus, and the remote sensing image recognition apparatus is disposed in a remote sensing image recognition device. One or more processors, a memory, and one or more application programs are provided in the remote sensing image recognition device, where one or more application programs are stored in the memory and configured to be executed by the processor to implement the remote sensing image recognition method; the remote sensing image recognition device may be a terminal, or the remote sensing image recognition device may also be a server, or a service cluster composed of multiple servers.

[0069] As Figure 1 shown, Figure 1 is a schematic diagram of the scenario of the remote sensing image recognition method in the embodiment of this application. In the remote sensing image recognition scenario in the embodiment of this application, a remote sensing image recognition device 100 is included. A remote sensing image recognition apparatus is integrated in the remote sensing image recognition device 100, and a computer-readable storage medium corresponding to the remote sensing image recognition is run to execute the steps of the remote sensing image recognition.

[0070] It can be understood that Figure 1 the remote sensing image recognition device in the specific application scenario of the remote sensing image recognition method shown, or the apparatus included in the remote sensing image recognition device does not constitute a limitation on the embodiment of this application. That is, the number of devices, the types of devices included in the specific application scenario of the remote sensing image recognition method, or the number of apparatuses, the types of apparatuses included in each device do not affect the overall implementation of the technical solution in the embodiment of this application, and can all be regarded as equivalent replacements or derivatives of the technical solution claimed in the embodiment of this application.

[0071] In the embodiment of the present application, the remote sensing image recognition device 100 is mainly used for: obtaining an initial remote sensing image to be processed; inputting the initial remote sensing image into a preset first image prediction model and a preset second image prediction model to obtain a first remote sensing prediction map and a second remote sensing prediction map; performing clustering analysis on the initial remote sensing image to obtain a clustered remote sensing image; fusing and recognizing the first remote sensing prediction map, the second remote sensing prediction map and the clustered remote sensing image to obtain the target ground objects included in the initial remote sensing image; in the present application, clustering recognition and neural network recognition are respectively adopted for the remote sensing image, and then the clustered remote sensing image formed by clustering recognition is fused and recognized with the prediction image formed by neural network recognition, so that the recognition of ground objects in the remote sensing image is more accurate.

[0072] In the embodiment of the present application, the remote sensing image recognition device 100 may be an independent remote sensing image recognition device, or a remote sensing image recognition device network or a remote sensing image recognition device cluster composed of remote sensing image recognition devices. For example, the remote sensing image recognition device 100 described in the embodiment of the present application includes, but is not limited to, a computer, a network host, a single network remote sensing image recognition device, a set of multiple network remote sensing image recognition devices, or a cloud remote sensing image recognition device composed of multiple remote sensing image recognition devices. Among them, the cloud remote sensing image recognition device is composed of a large number of computers or network remote sensing image recognition devices based on cloud computing.

[0073] Those skilled in the art can understand that Figure 1 the application environment shown in Figure 1 is only one application scenario of the solution of the present application, and does not constitute a limitation on the application scenario of the solution of the present application. Other application environments may also include more or fewer remote sensing image recognition devices than those shown in Figure 1 or the network connection relationship of remote sensing image recognition devices. For example, Figure 1 only shows 1 remote sensing image recognition device. It can be understood that the specific application scenario of this remote sensing image recognition method may also include one or more other remote sensing image recognition devices, which are not specifically limited here; the remote sensing image recognition device 100 may also include a memory.

[0074] In addition, in the specific application scenario of the remote sensing image recognition method of the present application, the remote sensing image recognition device 100 may be provided with a display device, or the remote sensing image recognition device 100 is not provided with a display device and is communicatively connected to an external display device 200. The display device 200 is used to output the results of the execution of the remote sensing image recognition method in the remote sensing image recognition device. The remote sensing image recognition device 100 can access the background database 300 (the background database can be in the local memory of the remote sensing image recognition device, and the background database can also be set in the cloud). The background database 300 stores information related to remote sensing image recognition.

[0075] It should be noted that Figure 1 The scene schematic diagram of the remote sensing image recognition method shown is only an example. The specific application scenarios of the remote sensing image recognition method described in the embodiments of the present application are for more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute a limitation to the technical solutions provided by the embodiments of the present application.

[0076] Based on the specific application scenarios of the above remote sensing image recognition method, embodiments of the remote sensing image recognition method are proposed.

[0077] Referring to Figure 2 , Figure 2 is a schematic flowchart of an embodiment of the remote sensing image recognition method in the embodiments of the present application. The remote sensing image recognition method includes steps 201-204:

[0078] 201. Obtain an initial remote sensing image to be processed.

[0079] The remote sensing image recognition method in this embodiment is applied to a remote sensing image recognition device. The types of the remote sensing image recognition device are not specifically limited. For example, the remote sensing image recognition device can be a terminal or a server.

[0080] The remote sensing image recognition device receives a remote sensing image recognition request. Among them, the triggering method for receiving the remote sensing image recognition request is not specifically limited. That is, the remote sensing image recognition request can be actively triggered by a user. For example, the user selects in the remote sensing image recognition device: "The initial remote sensing image transmitted back by satellite device No. xx", triggering an identification instruction. In addition, the remote sensing image recognition request can also be automatically triggered by the remote sensing image recognition device. For example, when the remote sensing image recognition device detects an updated remote sensing image, it automatically triggers a remote sensing image recognition request.

[0081] After the remote sensing image recognition device receives the remote sensing image recognition request, the remote sensing image recognition device obtains the initial remote sensing image to be processed corresponding to the remote sensing image recognition request. Among them, the quantity and specific form of the initial remote sensing image are not limited. Referring to Figure 3 , Figure 3 is a schematic diagram of the initial remote sensing image in an embodiment of the remote sensing image recognition method provided in the embodiments of the present application.

[0082] The remote sensing image recognition device processes the initial remote sensing image, analyzes each pixel point of the initial remote sensing image to obtain the ground object boundaries included in the initial remote sensing image, and the remote sensing image recognition device analyzes the ground object boundaries to obtain the target ground objects included in the initial remote sensing image. That is, the remote sensing image recognition device combines the existing ground object classifications, and the target ground objects include: forest land, grassland, cultivated land, water area, road, urban construction land, rural construction land, industrial land, structures, and bare land.

[0083] In this embodiment, the method for the remote sensing image recognition device to process the initial remote sensing image is not specifically limited. Specifically, it includes:

[0084] 202. Input the initial remote sensing image into a preset first image prediction model and a preset second image prediction model to obtain a first remote sensing prediction map and a second remote sensing prediction map.

[0085] In the remote sensing image recognition device, there are a preset first image prediction model and a preset second image prediction model. Among them, the first image prediction model and the second image prediction model are algorithms for extracting the feature information of the initial remote sensing image. It can be understood that in this embodiment, the first image prediction model and the second image prediction model are obtained through deep neural network learning. The first image prediction model and the second image prediction model are not the same, and the specific types of the first image prediction model and the second image prediction model in this embodiment are not limited.

[0086] The remote sensing image recognition device inputs the initial remote sensing image into a preset first image prediction model and a preset second image prediction model. The first image prediction model and the second image prediction model respectively analyze and extract features from the initial remote sensing image according to their respective image analysis algorithms to obtain a first remote sensing prediction map and a second remote sensing prediction map.

[0087] 203. Perform clustering analysis on the initial remote sensing image to obtain a clustered remote sensing image.

[0088] The remote sensing image recognition device performs clustering analysis on the initial remote sensing image to obtain a clustered remote sensing image, that is, the remote sensing image recognition device clusters each pixel point in the initial remote sensing image according to the pixel values of the pixel points, deletes some interfering pixel points, and obtains a clustered remote sensing image.

[0089] 204. Fuse and identify the first remote sensing prediction map, the second remote sensing prediction map, and the clustered remote sensing image to obtain the target ground objects included in the initial remote sensing image.

[0090] The remote sensing image recognition device fuses the first remote sensing prediction map and the second remote sensing prediction map once to obtain a target remote sensing image. The remote sensing image recognition device fuses the target remote sensing image and the clustered remote sensing image twice to obtain a fused remote sensing image. The remote sensing image recognition device analyzes the fused remote sensing image to obtain the target ground objects included in the initial remote sensing image.

[0091] In this embodiment, the specific implementation manner for the remote sensing image recognition device to analyze the fused remote sensing image to obtain the target ground objects included in the initial remote sensing image is not limited. That is, the remote sensing image recognition device can analyze the pixel points of the fused remote sensing image to determine the target ground objects included in the initial remote sensing image. Specifically, it includes:

[0092] (1) Fuse the first remote sensing prediction map, the second remote sensing prediction map, and the clustered remote sensing image to obtain a fused remote sensing image;

[0093] (2) Obtain the pixel point types of the fused remote sensing image and the number of types of the pixel point types;

[0094] (3) Input the pixel point types and the number of types into a preset intersection over union calculation formula to obtain the average intersection over union of the image;

[0095] (4) Obtain the land cover classification corresponding to the average intersection over union of the image, and use the land cover corresponding to the land cover classification as the target land cover included in the initial remote sensing image.

[0096] That is, in this embodiment, the remote sensing image recognition device fuses the first remote sensing prediction map, the second remote sensing prediction map, and the clustered remote sensing image to obtain a fused remote sensing image; the remote sensing image recognition device obtains the pixel point types of the fused remote sensing image and the number of types of the pixel point types. The types of pixel points include forest land pixel points, grassland pixel points, cultivated land pixel points, water area pixel points, road pixel points, urban construction land pixel points, rural construction land pixel points, industrial land pixel points, structure pixel points, and bare land pixel points; the number of types is 10.

[0097] The remote sensing image recognition device inputs the pixel point types and the number of types into a preset intersection over union calculation formula to obtain the average intersection over union of the image; wherein, the preset intersection over union calculation formula is:

[0098]

[0099] wherein, mIoU represents the average intersection over union of the image, k is the number of target land cover types, and pij is the number of pixels of type i classified as the j-th class.

[0100] The remote sensing image recognition device obtains the land cover classification corresponding to the average intersection over union of the image, and the remote sensing image recognition device uses the land cover corresponding to the land cover classification as the target land cover included in the initial remote sensing image.

[0101] In this embodiment, the mean Intersection over Union (mIoU) of images is introduced. By introducing the mIoU, the recognition of ground objects can be made more accurate. That is, in this embodiment, an identification algorithm is designed based on the given training samples, validation samples, and test data in the image training sample set to improve the recognition accuracy of remote sensing images. The mean Intersection over Union (mIoU) of images is used to reflect the recognition accuracy of the clustering algorithm (e.g., the k-means algorithm) of the present invention. Its value is the average of the Intersection over Union of all classes. As shown in the following table, corresponding to the score in the table, the calculated mean Intersection over Union of images is compared with the previous score. Suppose the previous score is A1 and the current score is A2. If A2 > A1, it indicates that the k-means algorithm of the present invention can improve the recognition effect.

[0102] For example, after the semantic segmentation post-processing method based on k-means clustering in the embodiment of the present invention, the recognition accuracy can be improved. The specific experimental results are as follows:

[0103] Name mIoU (before use) mIoU (after use) Score 0.3940 0.3951 Farmland 0.5723 0.5717 Forest 0.8782 0.8777 Grassland 0.0071 0.0081 Road 0.2725 0.2808 Urban construction land 0.3869 0.3871 Rural construction land 0.4624 0.4618 Industrial land 0.5079 0.5074 Construction land 0.0778 0.0804 Water area 0.7738 0.7748 Bare land 0.0012 0.0012

[0104] In this embodiment, the remote sensing image recognition device acquires an initial remote sensing image to be processed; inputs the initial remote sensing image into a preset first image prediction model and a preset second image prediction model to obtain a first remote sensing prediction map and a second remote sensing prediction map; performs clustering analysis on the initial remote sensing image to obtain a clustered remote sensing image; fuses and identifies the first remote sensing prediction map, the second remote sensing prediction map, and the clustered remote sensing image to obtain the target ground objects included in the initial remote sensing image. In the embodiment of the present application, two different image prediction models are preset. By combining the two image prediction models and image clustering analysis with the initial remote sensing image, when analyzing the remote sensing image, both the overall image and each pixel point in the image are considered, making the recognition of the remote sensing image more accurate.

[0105] Refer to Figure 4 , Figure 4 which is a schematic flowchart of an embodiment for pre-constructing the first image prediction model and the second image prediction model in the remote sensing image recognition method of the embodiment of the present application.

[0106] In this embodiment, an example of pre-constructing the first image prediction model and the second image prediction model by the remote sensing image recognition device is given, including steps 301 - 303:

[0107] 301. Acquire a large number of remote sensing images, add marks to each frame of sample remote sensing image in the large number of remote sensing images as image training samples, and summarize the image training samples to form an image training sample set.

[0108] The remote sensing image recognition device acquires a large amount of remote sensing images. The remote sensing image recognition device adds labels to each frame of sample remote sensing images in the large amount of remote sensing images. The way of adding labels is not limited. That is, the addition of labels can be manual labeling. For example, the remote sensing image recognition device outputs a sample remote sensing image, and the remote sensing image recognition device collects the classification label input by the user based on the sample remote sensing image. The remote sensing image recognition device associates the classification label with the sample remote sensing image. In addition, machine automatic labeling can also be set. For example, in the remote sensing image recognition device, the pixel points with the predefined pixel value corresponding to green are defined as grassland, etc. The remote sensing image recognition device automatically adds labels to the sample remote sensing images according to the defined pixel rules.

[0109] In this embodiment, the classification labels include cultivated land, forest land, grassland, roads, urban construction land, rural construction land, industrial land, structures, water areas, and bare land.

[0110] The remote sensing image recognition device adds labels to the labeled sample remote sensing images as image training samples, and aggregates the image training samples to form an image training sample set. Among them, the image training sample set refers to all the images used for model construction. The image training samples in the image training sample set can be divided into images for model training, images for model verification, and images for model testing according to specific functions.

[0111] This embodiment gives a specific scenario of the image training sample set. For example, the remote sensing image recognition device samples more than 40,000 sample remote sensing images and corresponding ground object classification label samples, and the image size is 256*256 pixels. The remote sensing image recognition device extracts these samples:

[0112] 16,017 training samples (where each tif picture corresponds to a png annotation picture);

[0113] 3,000 verification samples (where each tif picture corresponds to a png annotation picture);

[0114] 4,366 test samples (where each tif picture corresponds to a png annotation picture);

[0115] The image storage format is tif files, including four bands of R, G, B, and Nir. The image sizes of the training and test sets are both 256*256 pixels.

[0116] 302. Extract image training samples from the image training sample set, and iteratively train the first encoder and the first decoder through the image training samples to obtain a preset first image prediction model.

[0117] 303. Extract image training samples from the image training sample set, and iteratively train the second encoder and the second decoder with the image training samples to obtain a preset second image prediction model.

[0118] The remote sensing image recognition device extracts image training samples from the image training sample set, iteratively trains the first encoder and the first decoder with the image training samples to obtain a training prediction model, and detects and validates the training prediction model with validation samples and test samples. When the training prediction model passes the detection, the remote sensing image recognition device uses the trained training prediction model as a preset first image prediction model.

[0119] The remote sensing image recognition device extracts image training samples from the image training sample set, iteratively trains the second encoder and the second decoder with the image training samples to obtain a training prediction model, and detects and validates the training prediction model with validation samples and test samples. When the training prediction model passes the detection, the remote sensing image recognition device uses the trained training prediction model as a preset second image prediction model.

[0120] For example, the remote sensing image recognition device trains on training data. Common training models include: Unet, Unet++, MAnet, Linknet, FPN, PSPNet, PAN, DeepLabV3, DeepLabV3+. In the embodiments of this application, Unet++ is used as the first decoder, EfficientNet-b7 is used as the first encoder, and Unet++ is used as the second decoder, and ResNet101 is used as the second encoder to train the first image prediction model and the second image prediction model, marked as unetB7model1 and unetRestmodel2.

[0121] In this embodiment, the remote sensing image recognition device pre-constructs two image prediction models. In this way, by processing the initial remote sensing image with different image prediction models, different image feature information of the initial remote sensing image is extracted to obtain two different remote sensing prediction maps. Further, the two remote sensing prediction maps are fused and analyzed to make the remote sensing image recognition more accurate.

[0122] Refer to Figure 5 , Figure 5 which is a schematic flowchart of the fusion and recognition of the first remote sensing prediction map, the second remote sensing prediction map and the clustered remote sensing image in the remote sensing image recognition method provided in the embodiments of this application.

[0123] 401. Perform fusion dilation processing on the first remote sensing prediction map and the second remote sensing prediction map to obtain a target remote sensing prediction map.

[0124] The remote sensing image recognition device performs fusion expansion processing on the first remote sensing prediction image and the second remote sensing prediction image to obtain a target remote sensing prediction image, specifically including:

[0125] (1) extracting first feature data of the first remote sensing prediction map, extracting second feature data of the second remote sensing prediction map, encoding and converting the first feature data and the second feature data, and adding them to obtain encoded feature data;

[0126] (2) Processing the coded feature data according to a preset maximum independent variable function, fusing the processed coded feature data, and obtaining a target remote sensing prediction map.

[0127] That is, the remote sensing image recognition device extracts the first feature data of the first remote sensing prediction image, the remote sensing image recognition device extracts the second feature data of the second remote sensing prediction image, and the remote sensing image recognition device encodes and converts the first feature data and the second feature data and adds them to obtain the encoded feature data. Among them, the encoding method in the remote sensing image recognition device is one-hot encoding. One-hot encoding is also called one-bit effective encoding. Its method is to use an N-bit state register to encode N states. Each state has its own independent register bit, and at any time, only one of them is valid.

[0128] The remote sensing image recognition device presets a maximum independent variable function, wherein the preset maximum independent variable function is also called argmax, which is a function for finding a parameter (set) of a function. The remote sensing image recognition device processes the coded feature data according to the preset maximum independent variable function, and the remote sensing image recognition device fuses the processed coded feature data to obtain a target remote sensing prediction map. Specifically, it includes:

[0129] a. Processing the coded feature data according to a preset maximum independent variable function to obtain a classification label corresponding to the processed coded feature data;

[0130] b. comparing the preset standard structural element of the classification mark with each pixel point in the corresponding area of ​​the classification mark;

[0131] c. If the pixel point is the same as the standard structure element, retain the pixel point;

[0132] d. If the pixel point is different from the standard structure element, dilation processing is performed according to the standard structure element with the pixel point as the center to form a dilated pixel point;

[0133] e. Summarize the retained pixels and the expanded pixels to obtain the target remote sensing prediction map.

[0134] That is, in this embodiment, the remote sensing image recognition device processes the encoded feature data according to a preset maximum independent variable function, and the remote sensing image recognition device obtains the classification label corresponding to the processed encoded feature data; different standard structural elements corresponding to different classification labels are preset in the remote sensing image recognition device. For example, when the classification label is grassland, the corresponding standard structural element is a green pixel point.

[0135] The remote sensing image recognition device compares the standard structural element preset for the classification label with each pixel point in the area corresponding to the classification label; if the pixel point is the same as the standard structural element, the remote sensing image recognition device retains the pixel point; if the pixel point is different from the standard structural element, the remote sensing image recognition device performs dilation processing centered on the pixel point according to the standard structural element to form dilated pixel points; the remote sensing image recognition device summarizes the retained pixel points and the dilated pixel points to obtain a target remote sensing prediction map.

[0136] 402. Fuse and recognize the target remote sensing prediction map and the clustered remote sensing image to obtain the target ground objects included in the initial remote sensing image.

[0137] The remote sensing image recognition device fuses and recognizes the target remote sensing prediction map and the clustered remote sensing image to obtain the target ground objects included in the initial remote sensing image. Specifically, it includes:

[0138] (1). Obtain the first subscript of the first matrix corresponding to the target remote sensing prediction map, and obtain the second subscript of the second matrix corresponding to the clustered remote sensing image;

[0139] (2). Fuse the target remote sensing prediction map and the clustered remote sensing image according to the first subscript and the second subscript to obtain a fused remote sensing image, and recognize the fused remote sensing image to obtain the target ground objects included in the initial remote sensing image.

[0140] That is, the remote sensing image recognition device obtains the first subscript of the first matrix corresponding to the target remote sensing prediction map, and obtains the second subscript of the second matrix corresponding to the clustered remote sensing image; the remote sensing image recognition device fuses the target remote sensing prediction map and the clustered remote sensing image according to the first subscript and the second subscript to obtain a fused remote sensing image, and the remote sensing image recognition device recognizes the fused remote sensing image to obtain the target ground objects included in the initial remote sensing image.

[0141] For easy understanding, specific scenarios and steps for ground object recognition in the remote sensing image recognition method are given in this embodiment:

[0142] Refer to Figure 6 , Figure 6It is a schematic diagram of the specific process of remote sensing image recognition in the remote sensing image recognition method provided in the embodiment of the present application; the remote sensing image recognition device inputs the initial remote sensing image into the trained first image prediction model and the second image prediction model to obtain the predicted first remote sensing prediction map and the second remote sensing prediction map.

[0143] The remote sensing image recognition device performs feature conversion according to one-hot encoding, as Figure 7 shown. Figure 7 It is a schematic diagram of the specific scenario of one-hot encoding conversion in the remote sensing image recognition method provided in the embodiment of the present application. Figure 7 In it, the number 1 is converted to 100, the number 2 is converted to 010, and the number 3 is converted to 001. The remote sensing image recognition device adds the two-dimensional arrays corresponding to the first remote sensing prediction map and the second remote sensing prediction map after feature conversion, and uses argmax to take the subscript of the maximum value. For example, the subscript of the maximum value of 5402300904 is 8 (9 is the maximum value and it is in the 8th position), so argmax(5402300904) = 8.

[0144] Figure 8 It is a classification schematic diagram of the fused remote sensing image in the remote sensing image recognition method provided in the embodiment of the present application; the fused remote sensing image is a 256*256 matrix, where each number is a classification, which are 1-10 respectively. The current prediction map is marked as Myc. The remote sensing image recognition device obtains the target ground objects contained in the initial remote sensing image according to the classification on the fused remote sensing image. In this embodiment, taking the ground object on the fused remote sensing image as a road as an example:

[0145] Step 1: Obtain the position marked as a road (value is 4) in the model prediction map to obtain the area of the road, marked as LU;

[0146] Step 2: The remote sensing image recognition device dilates the area marked as a road in the fused remote sensing image. Among them, the specific method of the dilation operation is: compare each point in the image with the standard structure element marked as a road. If they are exactly the same, keep the point; if they are not the same, use the standard structure element marked as a road as a template to expand the point with this point as the center point. It can be implemented by using the dilate method in the opencv library. After the area LU is dilated, it is marked as PZ.

[0147] Step 3: Perform pixel clustering on the initial remote sensing image. The remote sensing image recognition device classifies pixels with similar color differences into one category. Since each remote sensing image may not include all 10 classification labels, for example, it may only include categories such as roads, forests, and waters. Before performing pixel clustering on the initial remote sensing image, the remote sensing image recognition device needs to first obtain the different values in the prediction map Myc, determine how many categories are included in the current prediction map Myc. For example, if there are only 6 categories, namely 1, 2, 4, 6, 8, and 9 in the prediction map Myc, then perform pixel clustering on the initial remote sensing image according to these categories. The clustering result is marked as JL, and the format of JL is the same as the content format of Myc, which is 256*256, but the labels inside are not 1-10. Here, the content is 0, 1, 2, 3, 4, 5 because there are only 6 categories in the prediction map.

[0148] Step 4: Calculate the labels in the corresponding clustered remote sensing image.

[0149] PZI = (PZ == 4), obtain the subscripts of the matrix equal to 4 in the target remote sensing prediction map.

[0150] JLLUlabel = mode(JL[PZI]), obtain the mode at the PZI position in the clustered remote sensing image.

[0151] JLLI = (JL == JLLUlabel), obtain the matrix subscripts of the label with the value of JLLU in the clustered remote sensing image.

[0152] Myc(PZI & JLLI) = 4, set the position that is both the subscript of PZI and the subscript of JLLI in the prediction map to 4; obtain the final fused remote sensing image.

[0153] Step 5: After performing the operations in Steps 1 to 4 on all the initial remote sensing images, conduct an evaluation. The evaluation metrics are: the mean intersection over union (mIoU) of the images. The mean intersection over union (mIoU) of the images is the average of the intersection over union of each type of image, and it is only used to evaluate the algorithm effect. The specific calculation formula is:

[0154]

[0155] where k is the number of target types, and pij is the number of pixels with the true type i that are classified into the j-th category.

[0156] Refer to Figure 9 , Figure 9 This is a schematic diagram of an embodiment process of iterative recognition in the remote sensing image recognition method of this application example.

[0157] 501, fuse the first remote sensing prediction map and the second remote sensing prediction map to obtain a target remote sensing prediction map.

[0158] The remote sensing image recognition device fuses the first remote sensing prediction map and the second remote sensing prediction map to obtain a target remote sensing prediction map.

[0159] 502. Fuse the target remote sensing prediction map and the clustered remote sensing image to obtain a fused remote sensing image, and calculate the intersection over union (IoU) of each pixel type in the fused remote sensing image.

[0160] The remote sensing image recognition device fuses the target remote sensing prediction map and the clustered remote sensing image to obtain a fused remote sensing image, and the remote sensing image recognition device calculates the intersection over union (IoU) of each pixel type in the fused remote sensing image.

[0161] 503. Calculate the average value of the intersection over union (IoU) corresponding to all pixel types to obtain the average IoU corresponding to all pixel types, and compare the average IoU with the average IoU obtained in the previous time.

[0162] The remote sensing image recognition device calculates the average value of the intersection over union (IoU) corresponding to all pixel types to obtain the average IoU corresponding to all pixel types, that is, sum the intersection over union (IoU) corresponding to each pixel type and then calculate the average value. For the first time, the average IoU is 0.4014, where the intersection over union (IoU) of cultivated land, forest land, grassland, road, urban construction land, rural construction land, industrial land, structure, water area and bare land are respectively: 0.5822, 0.8815, 0.0059, 0.2881, 0.3978, 0.4656, 0.5195, 0.0868, 0.7848, 0.0016 and 0.5822. The remote sensing image recognition device compares the average IoU with the average IoU obtained in the previous time. If the average IoU is larger than the average IoU obtained in the previous time, it means that the accuracy has improved. If the average IoU is smaller than the average IoU obtained in the previous time, a prompt is output.

[0163] 504. If the average IoU is greater than the average IoU obtained in the previous time, fuse the fused remote sensing image with the first remote sensing prediction map and the second remote sensing prediction map to obtain a new remote sensing prediction map, and fuse the new remote sensing prediction map with the previous clustered remote sensing image to obtain an iterative new fused remote sensing image;

[0164] If the average IoU is greater than the average IoU obtained in the previous time, the remote sensing image recognition device iterates the fused remote sensing image to the previous step, performs one - hot and argmax operations on the first remote sensing prediction map, the second remote sensing prediction map, and the fused remote sensing image to obtain a new remote sensing prediction map. The remote sensing image recognition device fuses the new remote sensing prediction map with the original image clustering map to obtain the new fused remote sensing image of this iteration; as Figure 10 shown Figure 10This is a schematic diagram of a specific scenario of iterative recognition in the remote sensing image recognition method in the embodiment of the present application; Figure 10 The remote sensing image recognition device uses the fused remote sensing image as a new clustered remote sensing image to iteratively calculate the mean intersection over union of the image, making the recognition of the remote sensing image more accurate.

[0165] 505. Obtain the newly formed fused remote sensing image through iteration, and use the ground object classification corresponding to the mean intersection over union of the new fused remote sensing image as the target ground object included in the initial remote sensing image.

[0166] The remote sensing image recognition device obtains the ground object classification corresponding to the mean intersection over union, and the remote sensing image recognition device uses the ground object corresponding to the ground object classification as the target ground object included in the initial remote sensing image.

[0167] For example, in this embodiment, the remote sensing image recognition device performs one-hot and argmax operations on the new fused remote sensing image, the first remote sensing prediction map, the second remote sensing prediction map, and the clustered remote sensing image to obtain a prediction image, and then performs dilation, clustering on the original image, and subsequent fusion, intersection over union calculation, etc.; during the experiment, generally when the number of iterations is about 4 times, its recognition accuracy and calculation time are within an acceptable range. In the subsequent table, it is iterated 60 times, and the score value has been increasing, but in actual applications, considering the acceptable data operation time, it generally will not be iterated to 60 times. In this embodiment, through multiple iterations, the recognition of the target ground object can be made more accurate.

[0168] For the convenience of understanding, in this embodiment, the relationship between the mean intersection over union and the number of iterations is given, and the results show that continuously repeating the mean intersection over union mIou will always be more accurate.

[0169] Times The 1st time The 2nd time The 3rd time The 4th time The 5th time The 60th time mIou 0.4014 0.4017 0.4018 0.4021 0.4022 … 0.4028 Farmland 0.5822 0.5816 0.5814 0.5813 0.5811 … 0.5805 Forest 0.8815 0.8812 0.8811 0.8810 0.8810 … 0.8807 Grassland 0.0059 0.0061 0.0063 0.0065 0.0067 … 0.0083 Road 0.2881 0.2892 0.2888 0.2897 0.2901 … 0.2896 Urban construction land 0.3978 0.3982 0.3983 0.3987 0.3989 … 0.3996 Rural construction land 0.4656 0.4657 0.4657 0.4658 0.4658 … 0.4662 Industrial land 0.5195 0.5195 0.5196 0.5198 0.5198 … 0.5191 Construction land 0.0868 0.0885 0.0896 0.0907 0.0913 … 0.0953 Water area 0.7848 0.7854 0.7856 0.7858 0.7861 … 0.789 Bare land 0.0016 0.0018 0.0016 0.0016 0.0015 … 0.0014

[0170] Such as Figure 11 shown, Figure 11 is a schematic structural diagram of an embodiment of a remote sensing image recognition device.

[0171] To better implement the remote sensing image recognition method in the embodiment of the present application, based on the remote sensing image recognition method, the embodiment of the present application also provides a remote sensing image recognition device, and the remote sensing image recognition device includes:

[0172] An image acquisition module 601, configured to acquire an initial remote sensing image to be processed;

[0173] An input prediction module 602, configured to input the initial remote sensing image into a preset first image prediction model and a preset second image prediction model to obtain a first remote sensing prediction map and a second remote sensing prediction map;

[0174] The clustering analysis module 603 is configured to perform clustering analysis on the initial remote sensing image to obtain a clustered remote sensing image;

[0175] The fusion recognition module 604 is configured to fuse and recognize the first remote sensing prediction map, the second remote sensing prediction map, and the clustered remote sensing image to obtain the target ground objects included in the initial remote sensing image.

[0176] In some embodiments of the present application, the remote sensing image recognition device includes:

[0177] Obtain a large number of remote sensing images, add marks to each frame of sample remote sensing images in the large number of remote sensing images as image training samples, and summarize the image training samples to form an image training sample set;

[0178] Extract image training samples from the image training sample set, and iteratively train a first encoder and a first decoder through the image training samples to obtain a preset first image prediction model;

[0179] Extract image training samples from the image training sample set, and iteratively train a second encoder and a second decoder through the image training samples to obtain a preset second image prediction model.

[0180] In some embodiments of the present application, the fusion recognition module 604 includes:

[0181] Perform fusion dilation processing on the first remote sensing prediction map and the second remote sensing prediction map to obtain a target remote sensing prediction map;

[0182] Fuse and recognize the target remote sensing prediction map and the clustered remote sensing image to obtain the target ground objects included in the initial remote sensing image.

[0183] In some embodiments of the present application, when the fusion recognition module 604 performs the fusion dilation processing on the first remote sensing prediction map and the second remote sensing prediction map to obtain a target remote sensing prediction map, it includes:

[0184] Extract first feature data of the first remote sensing prediction map, extract second feature data of the second remote sensing prediction map, perform encoding transformation on the first feature data and the second feature data and then add them to obtain encoded feature data;

[0185] Process the encoded feature data according to a preset maximum independent variable function, and fuse the processed encoded feature data to obtain a target remote sensing prediction map.

[0186] In some embodiments of the present application, the fusion recognition module 604 performs processing the encoded feature data according to a preset maximum independent variable function, fusing the processed encoded feature data, and obtaining a target remote sensing prediction map, including:

[0187] Process the encoded feature data according to a preset maximum independent variable function, and obtain a classification label corresponding to the processed encoded feature data;

[0188] Compare each pixel point in the corresponding area of the classification label with a preset standard structural element of the classification label;

[0189] If the pixel point is the same as the standard structural element, retain the pixel point;

[0190] If the pixel point is different from the standard structural element, perform dilation processing centered on the pixel point according to the standard structural element to form dilated pixel points;

[0191] Summarize the retained pixel points and the dilated pixel points to obtain a target remote sensing prediction map.

[0192] In some embodiments of the present application, the fusion recognition module 604 performs fusing and recognizing the target remote sensing prediction map and the clustered remote sensing image to obtain the target ground objects included in the initial remote sensing image, including:

[0193] Obtain a first subscript of a first matrix corresponding to the target remote sensing prediction map, and obtain a second subscript of a second matrix corresponding to the clustered remote sensing image;

[0194] Fuse the target remote sensing prediction map and the clustered remote sensing image according to the first subscript and the second subscript to obtain a fused remote sensing image, and recognize the fused remote sensing image to obtain the target ground objects included in the initial remote sensing image.

[0195] In some embodiments of the present application, the fusion recognition module 604 includes:

[0196] Fuse the first remote sensing prediction map, the second remote sensing prediction map and the clustered remote sensing image to obtain a fused remote sensing image;

[0197] Obtain the pixel point type of the fused remote sensing image and the number of types of the pixel point type;

[0198] Input the pixel point type and the number of types into a preset intersection over union calculation formula to obtain an average image intersection over union;

[0199] Obtain the ground object classification corresponding to the average image intersection over union, and use the ground object corresponding to the ground object classification as the target ground object included in the initial remote sensing image.

[0200] In some embodiments of the present application, the fusion recognition module 604 includes:

[0201] Fuse the first remote sensing prediction map and the second remote sensing prediction map to obtain a target remote sensing prediction map;

[0202] Fuse the target remote sensing prediction map and the clustered remote sensing image to obtain a fused remote sensing image, and calculate the intersection over union (IoU) of each pixel type in the fused remote sensing image;

[0203] Calculate the average value of the IoU corresponding to all pixel types to obtain the average IoU corresponding to the pixel type, and compare the average IoU with the average IoU obtained in the previous time;

[0204] If the average value of the average IoU is greater than the average IoU obtained in the previous time, fuse the fused remote sensing image with the first remote sensing prediction map and the second remote sensing prediction map to obtain a new remote sensing prediction map, and fuse the new remote sensing prediction map with the previous clustered remote sensing image to obtain an iterative new fused remote sensing image;

[0205] Obtain the newly formed fused remote sensing image through iteration, and use the ground object classification corresponding to the average IoU of the newly formed fused remote sensing image as the target ground object included in the initial remote sensing image.

[0206] In this embodiment, the remote sensing image recognition device obtains an initial remote sensing image to be processed; inputs the initial remote sensing image into a preset first image prediction model and a preset second image prediction model to obtain a first remote sensing prediction map and a second remote sensing prediction map; performs clustering analysis on the initial remote sensing image to obtain a clustered remote sensing image; and fuses and recognizes the first remote sensing prediction map, the second remote sensing prediction map, and the clustered remote sensing image to obtain the target ground object included in the initial remote sensing image. In the embodiments of the present application, the remote sensing image recognition device presets two different image prediction models, and combines the initial remote sensing image through the two image prediction models and image clustering analysis. In this way, when analyzing the remote sensing image, both the overall image and each pixel in the image are considered, making the remote sensing image recognition more accurate.

[0207] The embodiments of the present application further provide a remote sensing image recognition device, as Figure 12 shown, Figure 12 is a schematic structural diagram of an embodiment of the remote sensing image recognition device provided in the embodiments of the present application.

[0208] The remote sensing image recognition device integrates any one of the remote sensing image recognition devices provided in the embodiments of the present application. The remote sensing image recognition device includes:

[0209] One or more processors;

[0210] A memory; and

[0211] One or more applications, wherein the one or more applications are stored in the memory and are configured to execute the steps in the remote sensing image recognition method described in any one of the embodiments of the above remote sensing image recognition method by the processor.

[0212] Specifically: The remote sensing image recognition device may include components such as a processor 701 with one or more processing cores, a memory 702 of one or more computer-readable storage media, a power supply 703, and an input unit 704. Those skilled in the art can understand that Figure 12 the structure of the remote sensing image recognition device shown in

[0213] does not constitute a limitation on the remote sensing image recognition device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Among them:

[0214] The processor 701 is the control center of the remote sensing image recognition device, connecting various parts of the entire remote sensing image recognition device through various interfaces and lines. By running or executing software programs and / or modules stored in the memory 702, and by calling data stored in the memory 702, it executes various functions of the remote sensing image recognition device and processes data, thereby monitoring the entire remote sensing image recognition device. Optionally, the processor 701 may include one or more processing cores; preferably, the processor 701 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above modem processor may not be integrated into the processor 701.

[0215] The remote sensing image recognition device further includes a power supply 703 for supplying power to each component. Preferably, the power supply 703 can be logically connected to the processor 701 through a power management system, so as to manage functions such as charging, discharging, and power consumption management through the power management system. The power supply 703 can also include any components such as one or more DC or AC power supplies, a recharge system, a power failure detection circuit, a power converter or inverter, and a power status indicator.

[0216] The remote sensing image recognition device may further include an input unit 704, and the input unit 704 can be used to receive input digital or character information.

[0217] Although not shown, the remote sensing image recognition device may further include a display unit, etc., which will not be elaborated here. Specifically, in this embodiment, the processor 701 in the remote sensing image recognition device will load the executable files corresponding to the processes of one or more application programs into the memory 702 according to the following instructions, and the processor 701 will run the application programs stored in the memory 702 to implement various functions as follows:

[0218] Obtain an initial remote sensing image to be processed;

[0219] Input the initial remote sensing image into a preset first image prediction model and a preset second image prediction model to obtain a first remote sensing prediction map and a second remote sensing prediction map;

[0220] Perform clustering analysis on the initial remote sensing image to obtain a clustered remote sensing image;

[0221] Fusion-recognize the first remote sensing prediction map, the second remote sensing prediction map, and the clustered remote sensing image to obtain the target ground objects included in the initial remote sensing image.

[0222] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by controlling related hardware through instructions. The instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0223] Therefore, an embodiment of the present application provides a computer-readable storage medium, which may include: a read-only memory (ROM, Read Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disc, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any remote sensing image recognition method provided by the embodiments of the present application. For example, when the computer program is loaded by a processor, the following steps can be executed:

[0224] Obtain an initial remote sensing image to be processed;

[0225] Input the initial remote sensing image into a preset first image prediction model and a preset second image prediction model to obtain a first remote sensing prediction map and a second remote sensing prediction map;

[0226] Perform clustering analysis on the initial remote sensing image to obtain a clustered remote sensing image;

[0227] Fusion-identify the first remote sensing prediction map, the second remote sensing prediction map, and the clustered remote sensing image to obtain the target ground objects included in the initial remote sensing image.

[0228] In the above embodiments, the descriptions of the respective embodiments have their own emphases. For parts not detailed in a certain embodiment, reference may be made to the detailed descriptions of other embodiments above, and details will not be repeated here.

[0229] In specific implementation, the above-mentioned respective units or structures can be implemented as independent entities, or can be combined arbitrarily to be implemented as the same or several entities. The specific implementation of the above-mentioned respective units or structures can refer to the method embodiments above, and details will not be repeated here.

[0230] The specific implementation of the above respective operations can refer to the previous embodiments, and details will not be repeated here.

[0231] The above has introduced in detail a remote sensing image recognition method provided by an embodiment of the present application. Specific examples are used in this article to elaborate on the principle and implementation manner of the present application. The descriptions of the above embodiments are only used to help understand the method and its core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A method for remote sensing image recognition, characterized in that, The method includes: Obtaining an initial remote sensing image to be processed; Inputting the initial remote sensing image into a preset first image prediction model and a preset second image prediction model to obtain a first remote sensing prediction map and a second remote sensing prediction map; wherein, the first image prediction model and the second image prediction model are obtained through deep neural network learning, and the first image prediction model and the second image prediction model are different; Performing clustering analysis on the initial remote sensing image to obtain a clustered remote sensing image; Fusing and identifying the first remote sensing prediction map, the second remote sensing prediction map and the clustered remote sensing image to obtain the target ground objects included in the initial remote sensing image.

2. The method for remote sensing image recognition according to claim 1, characterized in that, Before inputting the initial remote sensing image into a preset first image prediction model and a preset second image prediction model to obtain a first remote sensing prediction map and a second remote sensing prediction map, the method includes: obtaining a large number of remote sensing images, adding marks to each frame of sample remote sensing image in the large number of remote sensing images as image training samples, and summarizing the image training samples to form an image training sample set; Extracting image training samples from the image training sample set, and iteratively training a first encoder and a first decoder through the image training samples to obtain a preset first image prediction model; Extracting image training samples from the image training sample set, and iteratively training a second encoder and a second decoder through the image training samples to obtain a preset second image prediction model.

3. The method for remote sensing image recognition according to claim 1, characterized in that, The fusing and identifying the first remote sensing prediction map, the second remote sensing prediction map and the clustered remote sensing image to obtain the target ground objects included in the initial remote sensing image includes: Performing fusion dilation processing on the first remote sensing prediction map and the second remote sensing prediction map to obtain a target remote sensing prediction map; Fusing and identifying the target remote sensing prediction map and the clustered remote sensing image to obtain the target ground objects included in the initial remote sensing image.

4. The method for remote sensing image recognition according to claim 3, characterized in that, The performing fusion dilation processing on the first remote sensing prediction map and the second remote sensing prediction map to obtain a target remote sensing prediction map includes: Extracting first feature data of the first remote sensing prediction map, extracting second feature data of the second remote sensing prediction map, adding the first feature data and the second feature data after encoding transformation to obtain encoded feature data; Processing the encoded feature data according to a preset maximum independent variable function, and fusing the processed encoded feature data to obtain a target remote sensing prediction map.

5. The method for remote sensing image recognition according to claim 4, characterized in that, The processing the encoded feature data according to a preset maximum independent variable function, and fusing the processed encoded feature data to obtain a target remote sensing prediction map includes: Processing the encoded feature data according to a preset maximum independent variable function, and obtaining a classification label corresponding to the processed encoded feature data; Comparing the classification label with each pixel point in the corresponding area of a preset standard structural element; If the pixel point is the same as the standard structural element, retaining the pixel point; If the pixel point is different from the standard structural element, performing dilation processing with the pixel point as the center according to the standard structural element to form dilated pixel points; Summarize the reserved pixel points and the dilated pixel points to obtain a target remote sensing prediction map.

6. The method for remote sensing image recognition according to claim 3, characterized in that, The fusion recognition of the target remote sensing prediction map and the clustered remote sensing image to obtain the target ground objects included in the initial remote sensing image includes: Obtain the first subscript of the first matrix corresponding to the target remote sensing prediction map, and obtain the second subscript of the second matrix corresponding to the clustered remote sensing image; Fuse the target remote sensing prediction map and the clustered remote sensing image according to the first subscript and the second subscript to obtain a fused remote sensing image, and identify the fused remote sensing image to obtain the target ground objects included in the initial remote sensing image.

7. The method for remote sensing image recognition according to any one of claims 1-6, characterized in that, The fusion recognition of the first remote sensing prediction map, the second remote sensing prediction map and the clustered remote sensing image to obtain the target ground objects included in the initial remote sensing image includes: Fuse the first remote sensing prediction map, the second remote sensing prediction map and the clustered remote sensing image to obtain a fused remote sensing image; Obtain the pixel point types of the fused remote sensing image and the number of types of the pixel point types; Input the pixel point types and the number of types into a preset intersection over union calculation formula to obtain the average intersection over union of the image; Obtain the ground object classification corresponding to the average intersection over union of the image, and use the ground objects corresponding to the ground object classification as the target ground objects included in the initial remote sensing image.

8. The method for remote sensing image recognition according to any one of claims 1-6, characterized in that, The fusion recognition of the first remote sensing prediction map, the second remote sensing prediction map and the clustered remote sensing image to obtain the target ground objects included in the initial remote sensing image includes: Fuse the first remote sensing prediction map and the second remote sensing prediction map to obtain a target remote sensing prediction map; Fuse the target remote sensing prediction map and the clustered remote sensing image to obtain a fused remote sensing image, and calculate the intersection over union of each pixel point type in the fused remote sensing image; Calculate the average value of the intersection over union corresponding to all pixel point types to obtain the average intersection over union corresponding to the pixel point types, and compare the average intersection over union with the average intersection over union obtained in the previous time; If the average intersection over union is greater than the average intersection over union obtained in the previous time, then fuse the fused remote sensing image with the first remote sensing prediction map and the second remote sensing prediction map to obtain a new remote sensing prediction map, and fuse the new remote sensing prediction map with the previous clustered remote sensing image to obtain an iterative new fused remote sensing image; Obtain the newly formed fused remote sensing image through iteration, and use the ground object classification corresponding to the average intersection over union of the newly formed fused remote sensing image as the target ground objects included in the initial remote sensing image.

9. A remote sensing image recognition device, characterized in that, The remote sensing image recognition device includes: An image acquisition module, configured to acquire an initial remote sensing image to be processed; An input prediction module, configured to input the initial remote sensing image into a preset first image prediction model and a preset second image prediction model to obtain a first remote sensing prediction map and a second remote sensing prediction map; wherein, the first image prediction model and the second image prediction model are obtained through deep neural network learning, and the first image prediction model and the second image prediction model are different; A clustering analysis module, configured to perform clustering analysis on the initial remote sensing image to obtain a clustered remote sensing image; The fusion recognition module is used to fuse and recognize the first remote sensing prediction map, the second remote sensing prediction map and the clustered remote sensing image, so as to obtain the target ground objects included in the initial remote sensing image.

10. A remote sensing image recognition device, characterized in that, The remote sensing image recognition device includes: One or more processors; A memory; and One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the remote sensing image recognition method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, A computer program is stored thereon, and the computer program is loaded by the processor to execute the steps in the remote sensing image recognition method according to any one of claims 1 to 8.

Images