Method, device and storage medium for object recognition in remote sensing images

Abstract

The present application discloses a method, device and storage medium for object recognition in remote sensing images. The method for object recognition in remote sensing images includes acquiring remote sensing image data, dividing the remote sensing image data into a plurality of first segmented image data according to a preset method; importing each of the first segmented image data into a preset model respectively, obtaining the second segmented image data corresponding to each of the first segmented image data; respectively calculating the probability value of the existence of the target object at each pixel point in each of the second segmented image data; and determining the object distribution image data corresponding to the remote sensing image data according to the probability value of the existence of the target object at each pixel point in each of the second segmented image data. This scheme realizes the recognition of target objects in remote sensing images. Since the pixels of the output image in the model are weighted according to the distance of the pixel points from the image center in the segmented image before being integrated into the final image, the detection accuracy of the target object is relatively high.

Description

Technical Field

[0001] The present application relates to the field of remote sensing image object recognition, and more specifically, to a method, device and storage medium for object recognition in remote sensing images. Background Art

[0002] As a high-tech technology with decades of history, remote sensing has important value in the fields of national defense, agriculture, town planning, urban construction, housing statistics, etc. With the development of science and technology, the data obtained by remote sensing equipment such as remote sensing satellites, drones, and manned aircraft are becoming more and more accurate, and the amount of data is increasing. In particular, my country's high-resolution satellite program has promoted a large number of sub-meter resolution satellites to the sky, and obtained a large amount of data on the global surface. However, how to automatically interpret and use this data is the most urgent thing at present. Automated target detection of specific types of objects in remote sensing image data is a very critical application field. For example, for urban planning, in the face of massive remote sensing image data, it is necessary to quickly detect the distribution and number of buildings and the corresponding vector information every quarter. This has great practical significance for finding illegal buildings, counting the number and area of ​​houses in a certain area, and providing accurate vector information.

[0003] In recent years, with the development of deep learning, there has been a trend in the field of remote sensing to use deep learning related technologies for target detection. Deep learning is a model that uses matrix multiplication and nonlinear functions to repeatedly iterate input data. The gradient method can be used to find the appropriate model weights so that the model can establish a complex mapping relationship between the input data and the target. At present, the deep learning building edge detection model commonly used in academia generally performs well on a wider range of data sets, but there are still many problems for building recognition. For example, it is difficult to detect the straight edge of a building; the detection effect of buildings in urban areas, mountains, and rural areas varies greatly; it is difficult to resist the interference of building shadows; due to the shooting angle of remote sensing images, the side of the tilted building is difficult to be correctly excluded, etc. In short, the current common deep learning building edge detection model is not very accurate on a wide range of remote sensing images. Summary of the invention

[0004] In order to solve the above problems, the embodiments of the present application provide a method, device and storage medium for object recognition in remote sensing images. In a first aspect, the embodiments of the present application provide a method for object recognition in remote sensing images, the method comprising:

[0005] Acquire remote sensing image data, and divide the remote sensing image data into a plurality of first segmented image data according to a preset method;

[0006] Importing each of the first segmented image data into a preset model respectively to obtain second segmented image data corresponding to each of the first segmented image data;

[0007] Calculating the probability value of the existence of the target object at each pixel point in each of the second segmented image data respectively;

[0008] Determining object distribution image data corresponding to the remote sensing image data according to a probability value of the existence of a target object at each pixel in each of the second segmented image data;

[0009] Among them, any one of the pixel points is located in at least one of the second segmented image data, and the probability value of whether the pixel point exists a target object is obtained by weighted summing up the positions of the pixel point in at least one of the second segmented image data, and the weighting coefficient of each pixel point in any one of the second segmented image data increases as the position of the pixel point in the second segmented image data increases from the edge to the center.

[0010] Optionally, the preset model includes a group of models, the preset model includes a group of models for different building densities, and the models are respectively trained based on a plurality of training data sets divided according to different building densities;

[0011] The step of importing each of the first segmented image data into a preset model to obtain the second segmented image data corresponding to each of the first segmented image data comprises:

[0012] Importing the first segmented image data into the preset model to obtain a set of third segmented image data;

[0013] The third segmented image data of the same group are integrated according to a preset integration method to obtain second segmented image data.

[0014] Optionally, a weighting coefficient of each pixel point in any one of the second segmented image data changes in a normally distributed manner along with the position of the pixel point in the second segmented image data.

[0015] Optionally, the acquiring of remote sensing image data, dividing the remote sensing image data into a plurality of first segmented image data in a preset manner, includes:

[0016] Acquire remote sensing image data, and read all the remote sensing image data into memory;

[0017] The remote sensing image data is divided into a plurality of first segmented image data according to a preset method.

[0018] Optionally, dividing the remote sensing image data into a plurality of first segmented image data in a preset manner includes:

[0019] The remote sensing image data is divided in a preset manner by setting a sliding window with a preset overlapping range to obtain a plurality of first segmented image data.

[0020] Optionally, after determining the object distribution image data corresponding to the remote sensing image data according to the probability value of the existence of the target object at each pixel in each of the second segmented image data, the method further includes:

[0021] Processing the pixel color values ​​of the object distribution image data according to a preset threshold value to obtain black and white image data;

[0022] Among them, when the color value of the pixel point is greater than a preset threshold, the color value of the pixel point is set to the color value corresponding to white; when the color value of the pixel point is less than or equal to the preset threshold, the color value of the pixel point is set to the color value corresponding to black.

[0023] Optionally, before acquiring the remote sensing image data and dividing the remote sensing image data into a plurality of first segmented image data in a preset manner, the method further includes:

[0024] Acquire training image data, and generate training black-and-white image data based on the training image data; the training black-and-white image data is black-and-white image data generated after annotating the training image data;

[0025] Dividing the training image data and the training black-and-white image data based on the preset method to obtain segmented training image data and segmented training black-and-white image data;

[0026] The segmented training image data and the segmented training black-and-white image data are imported into a preset training model to obtain the model.

[0027] Optionally, after dividing the training image data and the training black-and-white image data based on the preset method to obtain the segmented training image data and the segmented training black-and-white image data, the method further includes:

[0028] The parameters of the segmentation training image data and the segmentation training black-and-white image data are changed to obtain several groups of the segmentation training image data and the segmentation training black-and-white image data with different parameters; the parameters include the brightness of the image.

[0029] Optionally, before importing the segmented training image data and the segmented training black-and-white image data into a preset training model to obtain the model, the method further includes:

[0030] Normalizing the segmentation training image data and the segmentation training black-and-white image data;

[0031] The step of importing the segmented training image data and the segmented training black-and-white image data into a preset training model to obtain the model comprises:

[0032] The normalized segmentation training image data and the segmentation training black-and-white image data are imported into a preset training model to obtain the model.

[0033] In a second aspect, an embodiment of the present application provides a device for identifying objects in a remote sensing image, the device comprising:

[0034] An acquisition module, used for acquiring remote sensing image data, and dividing the remote sensing image data into a plurality of first segmented image data according to a preset method;

[0035] An import module, used for importing each of the first segmented image data into a preset model to obtain the second segmented image data corresponding to each of the first segmented image data;

[0036] A calculation module, used for respectively calculating the probability value of the existence of the target object at each pixel point in each of the second segmented image data;

[0037] A determination module, configured to determine the object distribution image data corresponding to the remote sensing image data according to the probability value of the existence of the target object at each pixel point in each of the second segmented image data;

[0038] Among them, any one of the pixel points is located in at least one of the second segmented image data, and the probability value of whether the pixel point exists a target object is obtained by weighted summing up the positions of the pixel point in at least one of the second segmented image data, and the weighting coefficient of each pixel point in any one of the second segmented image data increases as the position of the pixel point in the second segmented image data increases from the edge to the center.

[0039] In a third aspect, an embodiment of the present application provides a device for identifying objects in remote sensing images, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the processor implements the steps of the method provided in the first aspect or any possible implementation of the first aspect.

[0040] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method provided in the first aspect or any possible implementation manner of the first aspect.

[0041] In one or more embodiments of the present application, remote sensing image data is obtained, and the remote sensing image data is divided into a number of first segmented image data according to a preset method; each of the first segmented image data is imported into a preset model to obtain the second segmented image data corresponding to each of the first segmented image data; the probability value of the existence of the target object at each pixel in each of the second segmented image data is calculated; according to the probability value of the existence of the target object at each pixel in each of the second segmented image data, the object distribution image data corresponding to the remote sensing image data is determined. The server divides the remote sensing image into a number of first segmented image data with overlapping areas, and imports the first segmented image data into a preset model to obtain the second segmented image data, and after calculating the probability value of the existence of the target object at each pixel, all the pixels are integrated to determine the object distribution image data. In this way, the recognition of the target object in the remote sensing image is realized. Since the pixels of the output image in the model are weighted according to the distance of the pixel from the image center in the segmented image before being integrated into the final image, the detection result is not easily disturbed by factors such as shadows, and the detection accuracy of the target object is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 A schematic diagram of the architecture of a system for identifying objects in remote sensing images provided in an embodiment of the present application;

[0044] Figure 2 A schematic diagram of a flow chart of a method for identifying objects in remote sensing images provided in an embodiment of the present application;

[0045] Figure 3 A schematic diagram of a certain pixel point provided in an embodiment of the present application being located in four second segmented image data at the same time;

[0046] Figure 4 A schematic diagram of a flow chart of another method for identifying objects in remote sensing images provided in an embodiment of the present application;

[0047] Figure 5 A schematic diagram of dividing a remote sensing image by a sliding window provided in an embodiment of the present application;

[0048] Figure 6 A schematic diagram of a flow chart of another method for identifying objects in remote sensing images provided in an embodiment of the present application;

[0049] Figure 7A schematic diagram of the structure of a device for identifying objects in remote sensing images provided in an embodiment of the present application;

[0050] Figure 8 A schematic diagram of the structure of another device for identifying objects in remote sensing images provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0052] In the following introduction, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance. The following introduction provides multiple embodiments of the present application, and different embodiments can be replaced or combined, so the present application can also be considered to include all possible combinations of the same and / or different embodiments recorded. Therefore, if one embodiment includes features A, B, and C, and another embodiment includes features B and D, then the present application should also be considered to include embodiments containing one or more of all other possible combinations of A, B, C, and D, although the embodiment may not be clearly recorded in the following text.

[0053] The following description provides examples and does not limit the scope, applicability or examples set forth in the claims. Changes may be made to the functions and arrangements of the elements described without departing from the scope of the present application. Various processes or components may be appropriately omitted, substituted or added to each example. For example, the described method may be performed in an order different from the order described, and various steps may be added, omitted or combined. In addition, features described in some examples may be combined in other examples.

[0054] See also Figure 1 , Figure 1 Schematic diagram of the architecture of a remote sensing image object recognition system provided by an embodiment of the present application. Figure 1 As shown, the object recognition system in remote sensing images may include a server 10 , a model 20 , and an image acquisition device 30 .

[0055] The server 10 is responsible for processing tasks such as dividing and integrating remote sensing images. The server 10 may be a server that provides various services, and may be, but not limited to, a hardware server, a virtual server, a cloud server, etc. Specifically, the server 10 may be hardware or software.

[0056] Possibly, when the server 10 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server.

[0057] Possibly, when the server 10 is software, it can be implemented as multiple software or software modules, or it can be implemented as a single software or software module, which is not specifically limited here.

[0058] The model 20 is a server loaded with a deep learning model, which is used to process the remote sensing image sub-images divided into several overlapping areas sent by the server 10, and output the black and white sub-images corresponding to the remote sensing image sub-images to the server 10. The model 20 can be integrated in the server 10.

[0059] Possibly, the color of the pixel points corresponding to the target object in the black-and-white sub-image may be white, and the color of the pixel points corresponding to the non-target object may be black.

[0060] Possibly, model 20 can be a deep learning model or a group of deep learning models. In a group of deep learning models, different deep learning models have different detection error ranges for remote sensing images of different target object densities.

[0061] The image acquisition device 30 is responsible for capturing remote sensing images required for the server 10 to identify buildings, and sending the remote sensing images to the server 10 .

[0062] Possibly, the image acquisition device 30 may be a camera device on a satellite, an unmanned aerial vehicle, a manned aircraft, or the like.

[0063] Understandably, Figure 1 The number of servers 10 in the remote sensing image object recognition system architecture shown is only an example. In a specific implementation, the remote sensing image object recognition system may include any number of servers 10, and the embodiment of the present application does not limit this.

[0064] Exemplarily, after the image acquisition device 30 captures a remote sensing image, the image acquisition device 30 sends the remote sensing image to the server 10. After the remote sensing image is written into the server 10, the server 10 divides the remote sensing image into a plurality of remote sensing image sub-images with overlapping areas, and then sends the remote sensing image sub-images to the model 20. After receiving the remote sensing image sub-image, the model 20 processes the remote sensing image sub-image to obtain a black-and-white sub-image corresponding to the remote sensing image sub-image, and sends the black-and-white sub-image to the server 10. After the server 10 obtains the black-and-white sub-image, it integrates all the black-and-white sub-images according to the weighting coefficient that increases from the edge of the image to the middle of the image to obtain the object distribution image corresponding to the remote sensing image.

[0065] See also Figure 2 , Figure 2 : is a flow chart of a method for object recognition in remote sensing images provided in an embodiment of the present application. In the embodiment of the present application, the method includes:

[0066] S201: Acquire remote sensing image data, and divide the remote sensing image data into a plurality of first segmented image data according to a preset method.

[0067] The first segmented image data in the embodiment of the present application can be understood as a smaller remote sensing image obtained by segmenting a large remote sensing image data.

[0068] In an embodiment of the present application, the execution subject of the method may be a server, and the object that the server needs to identify from the remote sensing image may be a building. Remote sensing images are different from ordinary pictures. The original size of high-precision remote sensing images is generally large, for example, it may be an ultra-large image of 100,000*100,000 pixels. Therefore, after the server obtains the remote sensing image data, it will divide the remote sensing image data into several smaller first segmented image data in a preset manner, so that the server can complete the task with reasonable GPU computing resources.

[0069] Possibly, the remote sensing image data may be high-precision image data taken by satellites, unmanned aerial vehicles, manned aircraft, etc.

[0070] Possibly, the first segmented image data after division may be tens of thousands of small-sized remote sensing image pictures with a size of 512*512*3.

[0071] S202: Import each of the first segmented image data into a preset model to obtain second segmented image data corresponding to each of the first segmented image data.

[0072] In the embodiment of the present application, the second segmented image data may be understood as black-and-white image data corresponding to the first segmented image data obtained after the first segmented image data is imported into a preset model.

[0073] In an embodiment of the present application, after dividing the first segmented image data, the server imports the first segmented image data into a preset model, which is a pre-trained building edge detection model. After the first segmented image data is processed by the model, the second segmented image data will be obtained.

[0074] S203: Calculate the probability value of the existence of the target object at each pixel point in each of the second segmented image data.

[0075] In the embodiment of the present application, after each first segmented image data passes through the algorithm model, the output second segmented image data can be understood as a label of the same size as the first segmented image data (for example, 512*512 size), and the value in this label is still a floating point number continuously distributed from 0 to 1. Since there are overlapping areas between the divided areas when dividing the remote sensing image data, when these labels are reassembled into a large image, the inevitable problem is the processing of the overlapping areas in the process of reassembling.

[0076] For example, Figure 3 As shown, for a remote sensing image of 100000*100000*3, the point at the 12345th row and the 54321th column is located in four second segmented image data after division. One of the second segmented image data considers that the probability value of the presence of a building (i.e., the target object) at this location is 0.4, one of the second segmented image data considers that the probability value of the presence of a building at this location is 0.1, one of the second segmented image data considers that the probability value of the presence of a building at this location is 0.9, and one of the second segmented image data considers that the probability value of the presence of a building at this location is 0.7.

[0077] It should be noted that the default algorithm model generally does not perform better at the edge than at the center, that is, it is believed that the output result of the algorithm model is more accurate at the center. Therefore, the server will assign a weighting coefficient to each pixel in each picture, and the weighting coefficient increases from the edge to the center as the pixel is located in the second segmented image data. For example, in a second segmented image data, the label believes that the probability value of a building at a certain pixel in the second segmented image data is 0.4, but the pixel is at the edge of the second segmented image data. When the pixels are weighted and summed when spliced ​​into the whole picture, the weight assigned to the probability value of 0.4 of the pixel in the second segmented image data will be very small. In a second segmented image data, the label believes that the probability value of a building at a certain pixel in the second segmented image data is 0.1, but the pixel is at the center of the second segmented image data. When the pixels are weighted and summed when spliced ​​into the whole picture, the weight assigned to the probability value of 0.1 of the pixel in the second segmented image data will be very large. The server will perform weighted calculation based on the above weight coefficients to determine the final probability value of each pixel.

[0078] S204: Determine object distribution image data corresponding to the remote sensing image data according to the probability value of the existence of the target object at each pixel point in each of the second segmented image data.

[0079] In an embodiment of the present application, after the server calculates the final probability value of each pixel point, it is equivalent to having uniquely determined the final probability value of the same pixel point after splicing several second segmented image data. The server will determine the object distribution image data, that is, the building distribution image data, based on the probability values ​​of all pixels.

[0080] Possibly, when the server calculates that the final probability value of a certain pixel is higher than 0.5, the server considers that the target object exists at the pixel. When the server calculates that the final probability value of a certain pixel is not higher than 0.5, the server considers that the target object does not exist at the pixel. In the embodiment of the present application, the probability value can be any value between 0 and 1, and the specific numerical value of the probability value is only an example, and the present application does not impose any limitation on the specific data of the probability value.

[0081] Among them, any one of the pixel points is located in at least one of the second segmented image data, and the probability value of whether the pixel point exists a target object is obtained by weighted summing up the positions of the pixel point in at least one of the second segmented image data, and the weighting coefficient of each pixel point in any one of the second segmented image data increases as the position of the pixel point in the second segmented image data increases from the edge to the center.

[0082] In one possible implementation, the weighting coefficient of each pixel point in any one of the second segmented image data changes in a normally distributed manner along with the position of the pixel point in the second segmented image data.

[0083] In the embodiment of the present application, the server specifically determines the weighting coefficient value of each pixel point in the second segmented image data according to the normal distribution.

[0084] Optionally, after determining the object distribution image data corresponding to the remote sensing image data, that is, the final generated black-and-white image of the building distribution, the black-and-white image can be converted into a vector polygon format using a GIS tool. GIS tools are a set of tools specifically for processing geographic information data, which can easily convert the final black-and-white image into a vector polygon.

[0085] Through the above steps, the server divides the remote sensing image into a number of first segmented image data with overlapping areas, and imports the first segmented image data into the preset model to obtain the second segmented image data. After calculating the probability value of the target object at each pixel point, all the pixels are integrated to determine the object distribution image data. In this way, the recognition of the target object in the remote sensing image is realized. Since the pixels of the output image in the model are weighted according to the distance of the pixel points from the image center in the segmented image before being integrated into the final image, the detection result is not easily disturbed by factors such as shadows, and the detection accuracy of the target object is high.

[0086] See also Figure 4 , Figure 4 FIG. 2 is a flow chart of another method for identifying objects in remote sensing images according to an embodiment of the present application. Figure 4 As shown, the method includes:

[0087] S401, acquiring remote sensing image data, and reading all the remote sensing image data into a memory.

[0088] In the embodiment of the present application, in order to speed up the extraction of segmentation maps from remote sensing image data, the server obtains the remote sensing image data and sends the entire high-precision remote sensing image to the server. Figure 1 The entire solution is performed in memory, and since there is no additional file writing or reading process, the speed of the entire solution is greatly improved.

[0089] S402: Divide the remote sensing image data in a preset manner by setting a sliding window with a preset overlapping range to obtain a plurality of first segmented image data.

[0090] In an embodiment of the present application, the server divides the remote sensing image data through a sliding window.

[0091] Possibly, if Figure 5 As shown, the specific way of dividing the remote sensing image data by the sliding window can be sliding to the right or sliding to the lower right. The overlapping range of the sliding window can be 50% or 25%, and this application does not limit this.

[0092] S403: Import each of the first segmented image data into a preset model to obtain second segmented image data corresponding to each of the first segmented image data.

[0093] The specific process is shown in step S202 and will not be repeated here.

[0094] S404: Calculate the probability value of the existence of the target object at each pixel point in each of the second segmented image data.

[0095] The specific process is shown in step S203 and will not be repeated here.

[0096] S405 : Determine object distribution image data corresponding to the remote sensing image data according to the probability value of the existence of the target object at each pixel point in each of the second segmented image data.

[0097] The specific process is shown in step S204 and will not be repeated here.

[0098] S406, processing the pixel color value of the object distribution image data according to a preset threshold value to obtain black and white image data. When the pixel color value is greater than the preset threshold value, the pixel color value is set to the color value corresponding to white; when the pixel color value is less than or equal to the preset threshold value, the pixel color value is set to the color value corresponding to black.

[0099] In the embodiment of the present application, since the remote sensing image will have obvious splicing traces, it may cause different brightness in different areas of the remote sensing image, which may cause the object distribution image data finally generated by the server to have grayscale rather than a completely black and white image. Therefore, the server will process the color value of each pixel in the object distribution image data according to the threshold. When the color value of the pixel is greater than the preset threshold, the color value of the pixel is set to the color value corresponding to white; when the color value of the pixel is less than or equal to the preset threshold, the color value of the pixel is set to the color value corresponding to black.

[0100] Through the above steps, the server will read the remote sensing image into the memory at one time after acquiring it, select the sliding window to divide the remote sensing image into several first segmented image data with overlapping areas, and import the first segmented image data into the preset model to obtain the second segmented image data. After calculating the probability value of the target object at each pixel point, all the pixels are integrated to determine the object distribution image data, and the grayscale of the object distribution image data is adjusted according to the preset threshold to obtain completely black and white black and white image data. In this way, the recognition of the target object in the remote sensing image is realized. Since the pixels of the output image in the model are weighted according to the distance of the pixel points from the image center in the segmented image before being integrated into the final image, the detection result is not easily disturbed by factors such as shadows, and the detection accuracy of the target object is high. The entire solution is carried out in memory. Since there is no additional file writing or reading process, the speed of the entire solution is greatly improved.

[0101] See also Figure 6 , Figure 6 FIG. 2 is a flow chart of another method for identifying objects in remote sensing images according to an embodiment of the present application. Figure 6 As shown, the method includes:

[0102] S601, acquiring training image data, and generating training black-and-white image data based on the training image data; the training black-and-white image data is black-and-white image data generated after annotating the training image data.

[0103] In an embodiment of the present application, in order to construct a data set for a building edge detection model, the server will obtain a small portion of the high-precision remote sensing image data as training image data, and generate training black-and-white image data based on the training image data.

[0104] Specifically, since the training image data can be regarded as a matrix, the training image data can be labeled, with areas with buildings labeled as 1 and areas without buildings labeled as 0. The label of the labeled number, i.e., the training black-and-white image data, can be a black-and-white picture.

[0105] Possibly, the training image data may be labeled manually.

[0106] S602: Divide the training image data and the training black-and-white image data based on the preset method to obtain segmented training image data and segmented training black-and-white image data.

[0107] In the embodiment of the present application, the server divides the training image data and the training black-and-white image data so that the task can be completed with reasonable GPU resources. The segmented training image data is the sample input into the model for training. The segmented training black-and-white image data is the control sample of the result output after the segmented training image data is input into the model.

[0108] Possibly, the server divides the training image data and the training black-and-white image data by a sliding window with a certain overlapping area.

[0109] S603, changing the parameters of the segmentation training image data and the segmentation training black-and-white image data to obtain several groups of the segmentation training image data and the segmentation training black-and-white image data with different parameters; the parameters include image brightness.

[0110] In the embodiment of the present application, since remote sensing images will have obvious splicing traces, this will cause different brightness in different areas, and different color distributions in different areas. In order to maintain data consistency and allow the trained model to have better generalization capabilities, the server will change the parameters of the segmentation training image data with certain random values, and obtain multiple sets of segmentation training image data and segmentation training black and white image data with different parameters, so as to increase the number of samples for model training.

[0111] Possibly, the parameters may also include contrast, saturation.

[0112] Possibly, the server changes the parameters of the segmented training image data by a color random dithering operation, that is, by shifting the hues constituting the image, thereby creating a color crossover effect of differences in adjacent points.

[0113] S604: normalize the segmented training image data and the segmented training black-and-white image data.

[0114] In the embodiment of the present application, in order to avoid data from different regions and different sources being too different, the data needs to be normalized before being input into the model for training, that is, the average value of the data is subtracted and then divided by the variance of the data. This can eliminate the influence of large differences in image data from different regions, so that the training process of the building edge detection model can change smoothly.

[0115] It should be noted that in the prior art, the normalization operation is performed before image segmentation, that is, the whole image is normalized and then the image is segmented into several small images. This method is the best way to process ordinary images. The difference between remote sensing images and ordinary images is that remote sensing images are particularly large in size, and are not as colorful, shaped, and have various 3D angles as ordinary images, that is, general data sets. After experimental verification, for remote sensing images, the entire data is normalized to ensure the neat distribution of the entire data, and each divided image is not normalized separately to make each sample distributed neatly.

[0116] S605: Import the normalized segmented training image data and the segmented training black-and-white image data into a preset training model to obtain the model.

[0117] In the embodiment of the present application, since the building density of buildings in urban areas and buildings in rural areas, woodlands and other areas is different, the training results of the models obtained by training the same training model may be different. Some models have better parameter performance in urban areas and smaller errors, while some models have better parameter performance in mountainous areas and smaller errors. Ensure that the algorithm model performs well in different types of areas such as urban areas, mountain villages, and towns. Finally, a group of models corresponding to different building densities will be obtained. The model group is end-to-end and multi-model comprehensive integration.

[0118] It should be noted that the model is a set of building density models for different building densities, which are obtained by dividing the training data sets into several groups according to different building densities, and then using these data sets to train the training models respectively. For example, using a set of training data sets mainly including rural mountainous areas to train the training model, the obtained building density model performs well in areas with low building density; using a set of training data sets mainly including urban areas to train the training model, the obtained building density model performs well in areas with high building density.

[0119] S606: Acquire remote sensing image data, and divide the remote sensing image data into a plurality of first segmented image data according to a preset method.

[0120] The specific process is as described in step S201 and will not be repeated here.

[0121] S607: Import the first segmented image data into the preset model to obtain a set of third segmented image data.

[0122] The third segmented image data in the embodiment of the present application can be understood as comprising a set of preset models for different building density models, in which a certain building density model is output as a result of black and white image data based on the first segmented image data.

[0123] In the embodiment of the present application, since the preset model includes a set of building edge detection models corresponding to different building densities, after the server imports the first segmented image data into the model, a set of third segmented image data will be obtained. Each building edge detection model outputs a third segmented image data result.

[0124] S608: Integrate the third segmented image data of the same group according to a preset integration method to obtain second segmented image data.

[0125] In the embodiment of the present application, since different building edge detection models perform differently under different building densities, the server will assign a weighted value to the third segmented image data output by each model based on the density of the third segmented image data. For example, if the buildings in the third segmented image data are relatively dense, the weighted value of the third segmented image data output by the model with better output results for dense buildings is larger. The server will perform weighted summation and superposition on all the third segmented image data to obtain the second segmented image data.

[0126] S609: Calculate the probability value of the existence of the target object at each pixel point in each of the second segmented image data.

[0127] The specific process is as described in step S203 and will not be repeated here.

[0128] S610: Determine object distribution image data corresponding to the remote sensing image data according to a probability value of the existence of a target object at each pixel in each of the second segmented image data.

[0129] The specific process is as described in step S204 and will not be repeated here.

[0130] Through the above steps, the server first performs preprocessing such as dividing the training image data used for model training, random parameter jittering, etc., and then normalizes the divided segmented training image data and imports it into the training model for training. After the training is completed, the server divides the remote sensing image into a number of first segmented image data with overlapping areas, and imports the first segmented image data into a preset model to obtain a set of third segmented image data. After integrating the third segmented image data, the second segmented image data is obtained, and after calculating the probability value of the existence of the target object at each pixel point, all the pixels are integrated to determine the object distribution image data. In this way, the recognition of the target object in the remote sensing image is realized. Since the pixels of the output image in the model are weighted according to the distance of the pixel points from the image center in the segmented image before being integrated into the final image, the detection result is not easily disturbed by factors such as shadows, and the detection accuracy of the target object is high.

[0131] The following will be combined with the attached Figure 7 , the object recognition device in the remote sensing image provided by the embodiment of the present application is introduced in detail. It should be noted that the attached Figure 7 The object recognition device in the remote sensing image shown is used to execute the present application Figure 2 , Figure 4 , Figure 6 For the convenience of explanation, only the part related to the embodiment of the present application is shown. For the specific technical details not disclosed, please refer to the present application. Figure 2 , Figure 4 , Figure 6 The embodiment shown.

[0132] See also Figure 7 , Figure 7 Schematic diagram of a device for identifying objects in remote sensing images provided in an embodiment of the present application. Figure 7 As shown, the device comprises:

[0133] The acquisition module 701 is used to acquire remote sensing image data and divide the remote sensing image data into a plurality of first segmented image data according to a preset method;

[0134] An import module 702, used for importing each of the first segmented image data into a preset model to obtain second segmented image data corresponding to each of the first segmented image data;

[0135] A calculation module 703, used to calculate the probability value of the existence of the target object at each pixel point in each of the second segmented image data;

[0136] A determination module 704 is used to determine the object distribution image data corresponding to the remote sensing image data according to the probability value of the existence of the target object at each pixel in each of the second segmented image data;

[0137] Among them, any one of the pixel points is located in at least one of the second segmented image data, and the probability value of whether the pixel point exists a target object is obtained by weighted summing up the positions of the pixel point in at least one of the second segmented image data, and the weighting coefficient of each pixel point in any one of the second segmented image data increases as the position of the pixel point in the second segmented image data increases from the edge to the center.

[0138] In one possible implementation, the weighting coefficient of each pixel point in any one of the second segmented image data changes in a normally distributed manner along with the position of the pixel point in the second segmented image data.

[0139] In one possible implementation, the acquisition module 701 includes:

[0140] An acquisition unit, used for acquiring remote sensing image data and reading all the remote sensing image data into a memory;

[0141] The division unit is used to divide the remote sensing image data into a plurality of first segmented image data according to a preset method.

[0142] In one possible implementation, the division unit is specifically used for:

[0143] The remote sensing image data is divided in a preset manner by setting a sliding window with a preset overlapping range to obtain a plurality of first segmented image data.

[0144] In one possible implementation, the device further comprises:

[0145] A processing module, used for processing the pixel color values ​​of the object distribution image data according to a preset threshold value to obtain black and white image data;

[0146] Among them, when the color value of the pixel point is greater than a preset threshold, the color value of the pixel point is set to the color value corresponding to white; when the color value of the pixel point is less than or equal to the preset threshold, the color value of the pixel point is set to the color value corresponding to black.

[0147] In one possible implementation, the device further comprises:

[0148] A generating module, used to obtain training image data, and generate training black-and-white image data based on the training image data; the training black-and-white image data is black-and-white image data generated after annotating the training image data;

[0149] A division module, used for dividing the training image data and the training black-and-white image data based on the preset method to obtain segmented training image data and segmented training black-and-white image data;

[0150] The training module is used to import the segmented training image data and the segmented training black-and-white image data into a preset training model to obtain the model.

[0151] In one possible implementation, the device further comprises:

[0152] The modification module is used to change the parameters of the segmentation training image data and the segmentation training black-and-white image data to obtain several groups of the segmentation training image data and the segmentation training black-and-white image data with different parameters; the parameters include the brightness of the image.

[0153] In one possible implementation, the device further comprises:

[0154] The normalization module is used to perform normalization processing on the segmentation training image data and the segmentation training black-and-white image data.

[0155] In one possible implementation, the import module 702 is specifically used for:

[0156] Importing the first segmented image data into the preset model to obtain a set of third segmented image data;

[0157] The third segmented image data of the same group are integrated according to a preset integration method to obtain second segmented image data.

[0158] Those skilled in the art can clearly understand that the technical solutions of the embodiments of the present application can be implemented with the help of software and / or hardware. The "unit" and "module" in this specification refer to software and / or hardware that can independently complete or cooperate with other components to complete specific functions, where the hardware can be, for example, a field programmable gate array (Field-Programmable Gate Array, FPGA), an integrated circuit (Integrated Circuit, IC), etc.

[0159] Each processing unit and / or module of the embodiments of the present application may be implemented by an analog circuit that implements the functions described in the embodiments of the present application, or may be implemented by software that executes the functions described in the embodiments of the present application.

[0160] See also Figure 8 , which shows a schematic diagram of the structure of a remote sensing image object recognition device involved in an embodiment of the present application, the remote sensing image object recognition device can be used to implement Figure 2 , Figure 4 , Figure 6 The method in the embodiment shown. Figure 8As shown, the device 800 for identifying objects in remote sensing images may include: at least one central processor 801 , at least one network interface 804 , a user interface 803 , a memory 805 , and at least one communication bus 802 .

[0161] The communication bus 802 is used to realize the connection and communication between these components.

[0162] The user interface 803 may include a display screen (Display) and a camera (Camera), and the optional user interface 803 may also include a standard wired interface and a wireless interface.

[0163] The network interface 804 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0164] The central processing unit 801 may include one or more processing cores. The central processing unit 801 uses various interfaces and lines to connect the various parts of the entire remote sensing image object recognition device 800, and executes various functions and processes data of the remote sensing image object recognition device 800 by running or executing instructions, programs, code sets or instruction sets stored in the memory 805, and calling data stored in the memory 805. Optionally, the central processing unit 801 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-ProgrammableGate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The central processing unit 801 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; and the modem is used to process wireless communication. It is understandable that the above-mentioned modem may not be integrated into the central processing unit 801, but may be implemented by a separate chip.

[0165] Among them, the memory 805 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 805 includes a non-transitory computer-readable storage medium. The memory 805 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 805 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned method embodiments, etc.; the data storage area may store data involved in the above-mentioned method embodiments, etc. The memory 805 may optionally be at least one storage device located away from the aforementioned central processor 801. As Figure 8 As shown, the memory 805 as a computer storage medium may include an operating system, a network communication module, a user interface module, and program instructions.

[0166] exist Figure 8 In the device 800 for identifying objects in remote sensing images, the user interface 803 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 801 can be used to call the application program for identifying objects in remote sensing images stored in the memory 805, and specifically perform the following operations:

[0167] Acquire remote sensing image data, and divide the remote sensing image data into a plurality of first segmented image data according to a preset method;

[0168] Importing each of the first segmented image data into a preset model respectively to obtain second segmented image data corresponding to each of the first segmented image data;

[0169] Calculating the probability value of the existence of the target object at each pixel point in each of the second segmented image data respectively;

[0170] Determining object distribution image data corresponding to the remote sensing image data according to a probability value of the existence of a target object at each pixel in each of the second segmented image data;

[0171] Among them, any one of the pixel points is located in at least one of the second segmented image data, and the probability value of whether the pixel point exists a target object is obtained by weighted summing up the positions of the pixel point in at least one of the second segmented image data, and the weighting coefficient of each pixel point in any one of the second segmented image data increases as the position of the pixel point in the second segmented image data increases from the edge to the center.

[0172] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of the above method are implemented. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a micro drive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.

[0173] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0174] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0175] In the several embodiments provided in the present application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0176] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0177] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0178] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, disk or optical disk and other media that can store program code.

[0179] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by entering a program to instruct related hardware. The program can be stored in a computer-readable memory, and the memory can include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0180] The above is only an exemplary embodiment of the present disclosure, and the scope of the present disclosure cannot be limited thereto. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the specification and practicing the disclosure here, those skilled in the art will easily think of the implementation scheme of the present disclosure. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field not recorded in the present disclosure. The description and examples are regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A method for object recognition in remote sensing images, characterized in that: include: Acquire training image data, and generate training black-and-white image data based on the training image data; the training black-and-white image data is black-and-white image data generated after annotating the training image data; Dividing the training image data and the training black-and-white image data based on a preset method to obtain segmented training image data and segmented training black-and-white image data; Normalizing the segmentation training image data and the segmentation training black-and-white image data; Importing the normalized segmented training image data and the segmented training black-and-white image data into a preset training model to obtain a preset model, wherein the preset model is a trained building edge detection model; Acquire remote sensing image data, and divide the remote sensing image data into a plurality of first segmented image data according to the preset method; Importing each of the first segmented image data into the preset model respectively to obtain the second segmented image data corresponding to each of the first segmented image data; Calculating the probability value of the existence of the target object at each pixel point in each of the second segmented image data respectively; Determining object distribution image data corresponding to the remote sensing image data according to a probability value of the existence of a target object at each pixel in each of the second segmented image data; Among them, any one of the pixel points is located in at least one of the second segmented image data, and the probability value of whether the pixel point exists a target object is obtained by weighted summing up the positions of the pixel point in at least one of the second segmented image data, and the weighting coefficient of each pixel point in any one of the second segmented image data increases as the position of the pixel point in the second segmented image data increases from the edge to the center.

2. The method according to claim 1, characterized in that The preset model includes a group of models for different building densities, and the models are obtained by training based on a number of training data sets divided according to different building densities; The step of importing each of the first segmented image data into the preset model to obtain the second segmented image data corresponding to each of the first segmented image data comprises: Importing the first segmented image data into the preset model to obtain a set of third segmented image data; The third segmented image data of the same group are integrated according to a preset integration method to obtain second segmented image data.

3. The method according to claim 1, characterized in that: The weighting coefficient of each pixel point in any one of the second segmented image data changes in a normal distribution along with the position of the pixel point in the second segmented image data.

4. The method according to claim 1, characterized in that The acquiring of remote sensing image data, and dividing the remote sensing image data into a plurality of first segmented image data according to the preset method, comprises: Acquire remote sensing image data, and read all the remote sensing image data into memory; The remote sensing image data is divided into a plurality of first segmented image data according to the preset method.

5. The method according to claim 4, characterized in that The step of dividing the remote sensing image data into a plurality of first segmented image data according to the preset method includes: The remote sensing image data is divided according to the preset manner by setting a sliding window with a preset overlapping range to obtain a plurality of first segmented image data.

6. The method according to claim 1, characterized in that After determining the object distribution image data corresponding to the remote sensing image data according to the probability value of the existence of the target object at each pixel in each of the second segmented image data, the method further includes: Processing the pixel color values ​​of the object distribution image data according to a preset threshold value to obtain black and white image data; Among them, when the color value of the pixel point is greater than a preset threshold, the color value of the pixel point is set to the color value corresponding to white; when the color value of the pixel point is less than or equal to the preset threshold, the color value of the pixel point is set to the color value corresponding to black.

7. The method according to claim 1, characterized in that After the training image data and the training black-and-white image data are divided based on a preset method to obtain the segmented training image data and the segmented training black-and-white image data, the method further includes: The parameters of the segmentation training image data and the segmentation training black-and-white image data are changed to obtain several groups of the segmentation training image data and the segmentation training black-and-white image data with different parameters; the parameters include the brightness of the image.

8. A device for identifying objects in remote sensing images, characterized in that: include: A generating module, used to obtain training image data, and generate training black-and-white image data based on the training image data; the training black-and-white image data is black-and-white image data generated after annotating the training image data; A division module, used for dividing the training image data and the training black-and-white image data based on a preset method to obtain segmented training image data and segmented training black-and-white image data; A normalization module, used for normalizing the segmentation training image data and the segmentation training black-and-white image data; A training module, used for importing the normalized segmented training image data and the segmented training black-and-white image data into a preset training model to obtain a preset model, wherein the preset model is a trained building edge detection model; An acquisition module, used for acquiring remote sensing image data, and dividing the remote sensing image data into a plurality of first segmented image data according to the preset method; An import module, used for importing each of the first segmented image data into the preset model to obtain the second segmented image data corresponding to each of the first segmented image data; A calculation module, used for respectively calculating the probability value of the existence of the target object at each pixel point in each of the second segmented image data; A determination module, configured to determine the object distribution image data corresponding to the remote sensing image data according to the probability value of the existence of the target object at each pixel point in each of the second segmented image data; Among them, any one of the pixel points is located in at least one of the second segmented image data, and the probability value of whether the pixel point exists a target object is obtained by weighted summing up the positions of the pixel point in at least one of the second segmented image data, and the weighting coefficient of each pixel point in any one of the second segmented image data increases as the position of the pixel point in the second segmented image data increases from the edge to the center.

9. A device for identifying objects in remote sensing images, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images