Method, apparatus, device and storage medium for identifying ground objects in large-scale remote sensing images

Abstract

The present invention discloses a method for identifying ground objects in large-scale remote sensing images. The present invention performs overlapping cropping on large-scale remote sensing images to obtain picture slices, performs instance segmentation on the picture slices to obtain overlapping building instances, determines the picture slices to which the overlapping building instances belong, determines the overlapping set and independent targets within the neighborhood range according to the indexes of the picture slices to which they belong, screens the overlapping set to obtain a list of overlapping sets, and calculates the mask overlapping rate of each building instance in each set with other building instances in the set according to the screened list of overlapping sets; adds the instances with the mask overlapping rate higher than the threshold to the instance object set, uses the instances with the mask overlapping rate lower than the threshold as independent targets, generates building mask information according to the instance object set, and obtains the mask information of all building instances according to the building mask information and the mask information of the independent targets. It solves the problem that objects are cut into multiple parts after overlapping cropping.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a method, device, equipment and storage medium for identifying ground objects in large-scale remote sensing images. Background Art

[0002] Extracting specific ground objects from large-area remote sensing images is a main task in remote sensing image analysis. To solve the problem that directly inputting large-scale remote sensing images into a network model in the existing solution will cause video memory overflow, the currently adopted technical solution is to crop the large-scale remote sensing image into a series of smaller images and then input them into the network for prediction, and then splice the prediction results into a final result image according to the cropping order. However, this processing method will easily split the objects at the cropping edges. Or, the cropping and prediction are carried out in an overlapping manner, and then the overlap rate is calculated between the circumscribed rectangles of the instance segmentation masks globally. If the overlap rate exceeds a certain threshold, they will be merged into one object. However, this solution can only be applied to objects arranged horizontally, and the processing effect on non-horizontally arranged objects is not good.

[0003] The above content is only used to assist in understanding the technical solution of the present invention, and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of the present invention is to provide a method, device, equipment and storage medium for identifying ground objects in large-scale remote sensing images, aiming to solve the technical problem that the existing technology cannot adapt to the prediction of remote sensing images of non-horizontally arranged objects.

[0005] To achieve the above purpose, the present invention provides a method for identifying ground objects in large-scale remote sensing images, and the method includes the following steps:

[0006] Overlap-crop the large-scale remote sensing image to obtain picture slices;

[0007] Perform instance segmentation on the picture slices to obtain overlapping building instances;

[0008] Determine the picture slices to which the overlapping building instances belong, and determine the overlapping set and independent targets within the neighborhood range according to the indexes of the picture slices to which they belong;

[0009] Screen the overlapping set to obtain an overlapping set list. According to the screened overlapping set list, calculate the mask overlap rate between each building instance in each set and other building instances in the set; add the instances with the mask overlap rate higher than the threshold to the instance object set, and use the instances with the mask overlap rate lower than the threshold as independent targets;

[0010] Generate building mask information based on the set of instance objects, and obtain all building instance mask information based on the building mask information and the mask information of the independent target.

[0011] Optionally, the instance segmentation of the picture slice to obtain overlapping building instances includes:

[0012] Obtain the mask and rectangular box coordinates of the building instance according to the picture slice;

[0013] Create a first mask for the building instance, the first mask includes the coordinate offset relative to the large-scale remote sensing image, the rectangular box coordinates, and the index information of the current picture slice, and the storage size of the first mask is the same as the size of the picture slice;

[0014] Create a second mask based on the large-scale remote sensing image, superimpose the first mask and the second mask to obtain a third mask, and obtain overlapping building instances according to the third mask. The sizes of the second mask and the third mask are the same as the size of the large-scale remote sensing image.

[0015] Optionally, determining the picture slice to which the overlapping building instance belongs, and determining the overlapping set and independent target within the neighborhood range according to the index of the picture slice to which it belongs, includes:

[0016] Determine the index values of other picture slices within the neighborhood range of the index of the picture slice to which it belongs according to the index of the picture slice to which it belongs;

[0017] Determine other building instances within the neighborhood range according to the index values of the other picture slices;

[0018] Traverse the building instances to obtain the rectangular boxes of the building instances;

[0019] Detect the rectangular boxes of the building instances. When there is an overlap in the rectangular boxes of the building instances, add the index of the building instance to the overlap set;

[0020] When there is no overlap in the rectangular boxes of the building instances, determine the building instance as an independent target.

[0021] Optionally, after adding the index of the building instance to the overlap set when there is an overlap in the rectangular boxes of the building instances, it further includes:

[0022] Traverse the overlap set to obtain the merge flag bits of the building instances in the overlap set;

[0023] When there is an intersection between the building instance in the overlap set and other building instances in the overlap set, update the flag bit of the building instance in the overlap set to True;

[0024] Merging the building instances whose flags are True, and detecting the merged building instances;

[0025] If the merged building instance has an intersection with other building instances in the overlapping set, the flag bit of the merged building instance is updated to True, and the building instances with the flag bit being True are merged, and the steps of detecting the merged building instance are performed;

[0026] When there is no intersection between the building instance in the overlapping set and other building instances in the overlapping set, the building instance in the overlapping set is added to the result list.

[0027] Optionally, before determining, according to the index of the picture slice to which it belongs, index values ​​of other picture slices within a neighborhood range of the index of the picture slice to which it belongs, the method further includes:

[0028] Obtaining the number of slices in the horizontal direction and the number of slices in the vertical direction according to the size of the large remote sensing image and the overlapping cropping size;

[0029] A matrix is ​​created according to the number of slices in the horizontal direction and the number of slices in the vertical direction, and the number of rows and the number of columns of the matrix are consistent with the number of slices in the horizontal direction and the number of slices in the vertical direction.

[0030] Optionally, determining index values ​​of other picture slices within a neighborhood range of the index of the picture slice according to the index of the picture slice to which it belongs includes:

[0031] Storing the index of the image slice in an index list;

[0032] Determine the validity of the index values ​​in the four directions of up, down, left, and right of the index of the image slice, and add the valid index to the index list;

[0033] Determine the validity of the index values ​​of the four corners of the index of the picture slice, and add the valid index to the index list;

[0034] The index values ​​of other picture slices within the neighborhood are obtained according to the index values ​​recorded in the index list.

[0035] Optionally, generating building mask information according to the set of instance objects, and obtaining all building instance mask information according to the building mask information and mask information of the independent target, includes:

[0036] Obtaining a coordinate offset from the large-scale remote sensing image according to the building mask information and the mask information of the independent target;

[0037] Add the building mask information and the mask information of the independent target to the large-scale remote sensing image according to the coordinate offset to obtain the mask information of all building instances.

[0038] In addition, to achieve the above object, the present invention also provides a device for identifying ground objects in a large-scale remote sensing image. The device for identifying ground objects in a large-scale remote sensing image includes:

[0039] An image slicing module, configured to perform overlapping cropping on the large-scale remote sensing image to obtain image slices;

[0040] An instance segmentation module, configured to perform instance segmentation on the image slices to obtain overlapping building instances;

[0041] A slice overlapping module, configured to determine the image slices to which the overlapping building instances belong, and determine an overlapping set and independent targets within the neighborhood range according to the indexes of the image slices to which they belong;

[0042] An instance screening module, configured to screen the overlapping set to obtain a list of overlapping sets, and calculate the mask overlap rate of each building instance in each set with other building instances in the set according to the screened list of overlapping sets; add the instances with the mask overlap rate higher than the threshold to the instance object set, and use the instances with the mask overlap rate lower than the threshold as independent targets;

[0043] A mask fusion module, configured to generate building mask information according to the instance object set, and obtain the mask information of all building instances according to the building mask information and the mask information of the independent targets.

[0044] In addition, to achieve the above object, the present invention also provides a device for identifying ground objects in a large-scale remote sensing image. The device for identifying ground objects in a large-scale remote sensing image includes: a memory, a processor, and a program for identifying ground objects in a large-scale remote sensing image stored on the memory and executable on the processor. The program for identifying ground objects in a large-scale remote sensing image is configured to implement the steps of the method for identifying ground objects in a large-scale remote sensing image as described above.

[0045] In addition, to achieve the above object, the present invention also provides a storage medium. A program for identifying ground objects in a large-scale remote sensing image is stored on the storage medium. When the program for identifying ground objects in a large-scale remote sensing image is executed by a processor, the steps of the method for identifying ground objects in a large-scale remote sensing image as described above are implemented.

[0046] The present invention obtains picture slices by overlapping and cropping a large-scale remote sensing image, performs instance segmentation on the picture slices to obtain overlapping building instances, determines the picture slices to which the overlapping building instances belong, determines the overlapping set and independent targets within the neighborhood range according to the indexes of the picture slices to which they belong, adds the instances with a coincidence rate higher than the threshold to the instance object set according to the coincidence rate of each building instance in the overlapping set with other building instances in the overlapping set, takes the instances with a coincidence rate lower than the threshold as independent targets, generates building mask information according to the instance object set, and obtains the mask information of all building instances according to the building mask information and the mask information of the independent targets. It realizes the merging of the original masks that are cut into more than two parts at the cropping edge, restores the true mask of the object at the cropping edge, and solves the problem that the object is cut into multiple parts after overlapping cropping. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a schematic structural diagram of a ground object recognition device for a large-scale remote sensing image in the hardware operating environment involved in the embodiment solution of the present invention;

[0048] Figure 2 It is a schematic flowchart of the first embodiment of the ground object recognition method for a large-scale remote sensing image of the present invention;

[0049] Figure 3 It is a flowchart of the fusion of the ground object recognition results of a large-scale remote sensing image in an embodiment of the ground object recognition method for a large-scale remote sensing image of the present invention;

[0050] Figure 4 It is a schematic diagram of the image overlapping cropping in an embodiment of the ground object recognition method for a large-scale remote sensing image of the present invention;

[0051] Figure 5 It is the original mask image of the instance segmentation algorithm in an embodiment of the ground object recognition method for a large-scale remote sensing image of the present invention;

[0052] Figure 6 It is a schematic diagram of the 8-neighborhood of a picture slice in an embodiment of the ground object recognition method for a large-scale remote sensing image of the present invention;

[0053] Figure 7 It is a comparison diagram before and after the fusion of the original mask in an embodiment of the ground object recognition method for a large-scale remote sensing image of the present invention;

[0054] Figure 8 It is a schematic flowchart of the second embodiment of the ground object recognition method for a large-scale remote sensing image of the present invention;

[0055] Figure 9 It is a schematic diagram of the overlapping rectangular frame in an embodiment of the ground object recognition method for a large-scale remote sensing image of the present invention;

[0056] Figure 10This is the structural block diagram of the first embodiment of the ground feature recognition device for large-scale remote sensing images of the present invention.

[0057] The realization of the purpose of the present invention, functional characteristics and advantages will be further described with reference to the embodiments and the accompanying drawings. Specific Embodiments

[0058] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0059] Refer to Figure 1 , Figure 1 This is the structural schematic diagram of the ground feature recognition device for large-scale remote sensing images of the hardware operating environment involved in the embodiment solution of the present invention.

[0060] As Figure 1 shown, the ground feature recognition device for large-scale remote sensing images may include: a processor 1001, such as a Central Processing Unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display) and an input unit such as a keyboard (Keyboard). Optionally, the user interface 1003 may further include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be a high-speed Random Access Memory (RAM) memory or a stable Non-Volatile Memory (NVM), such as a disk memory. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0061] Those skilled in the art can understand that Figure 1 the structure shown in

[0062] does not constitute a limitation on the ground feature recognition device for large-scale remote sensing images, and may include more or fewer components than shown, or combine some components, or have different component arrangements. Figure 1 shown, the memory 1005, as a storage medium, may include an operating system, a network communication module, a user interface module, and a ground feature recognition program for large-scale remote sensing images.

[0063] In Figure 1In the object recognition device for large-scale remote sensing images shown, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the object recognition device for large-scale remote sensing images of the present invention can be arranged in the object recognition device for large-scale remote sensing images. The object recognition device for large-scale remote sensing images calls the object recognition program for large-scale remote sensing images stored in the memory 1005 through the processor 1001 and executes the object recognition method for large-scale remote sensing images provided in the embodiments of the present invention.

[0064] Embodiments of the present invention provide an object recognition method for large-scale remote sensing images. Refer to Figure 2 , Figure 2 which is a schematic flowchart of the first embodiment of the object recognition method for large-scale remote sensing images of the present invention.

[0065] In this embodiment, the object recognition method for large-scale remote sensing images includes the following steps:

[0066] Step S10: Overlap and crop the large-scale remote sensing image to obtain picture slices.

[0067] It should be noted that the execution subject of this embodiment is the object recognition device for large-scale remote sensing images. Among them, the object recognition device for large-scale remote sensing images has functions such as data processing, data communication, and program operation. The object recognition device for large-scale remote sensing images can be an integrated controller, a control computer, etc. Of course, it can also be other devices with similar functions. This embodiment does not limit this.

[0068] It should be understood that the picture slice is a local map of the large-scale remote sensing image, and the picture slice is obtained by overlapping and cropping the large-scale remote sensing image.

[0069] In specific implementation, refer to Figure 3 , Figure 3 which is a flowchart of the fusion of object recognition results for large-scale remote sensing images. First, read the large-scale remote sensing image, and then crop the large-scale remote sensing image into pictures with a size of 640*640. The overlapping area size is 32. The resolution size of the sliced pictures and the overlapping area size are set according to the actual situation. This embodiment does not limit this, and record the coordinate offset of the upper left corner of each picture slice relative to the upper left corner of the large-scale remote sensing image. The coordinate offset can be expressed as (star_x, star_y). Refer to Figure 4 , Figure 4 which is a schematic diagram of image overlapping cropping. For the parts that do not meet the 640*640 size at the right edge and bottom edge of the image seen only during the cropping process, the image can be extended from the rightmost and bottommost to the inside of the image, that is, extended to the top and left to a 640*640 size area. At this time, the overlapping area of the obtained picture slice is greater than 32.

[0070] Step S20: Perform instance segmentation on the sliced images to obtain overlapping building instances.

[0071] It should be noted that a building instance refers to a remote sensing image obtained by remote sensing of a building, and an overlapping building instance refers to a building instance in the overlapping area during overlapping cropping.

[0072] In a specific implementation, refer to Figure 5 , Figure 5 as the original mask image of the instance segmentation algorithm. Use YOLOv8 to predict the 640*640 sliced images that are cropped, obtain the coordinates of the mask and the rectangular box for each building instance, create a 640*640 mask for each building instance, with the position value of 1 where there is a building and 0 in other positions; save information such as the coordinate offset relative to the large-scale remote sensing image, the coordinates of the rectangular box, and the index of the current sliced image; then create a mask with all 0s and the same size as the original large-scale remote sensing image, and then overlay the mask of each building instance on this mask. Where there is an overlap of buildings after overlaying, the pixel value will increase, and the pixel value of the non-overlapping but building-containing area remains 1. After obtaining the results of the entire remote sensing image, traverse the mask of each building instance in the entire remote sensing image, calculate the maximum pixel value, and obtain the overlapping building instances based on the building instances with a maximum value greater than 1.

[0073] Further, the performing instance segmentation on the sliced images to obtain overlapping building instances includes:

[0074] Obtain the mask and rectangular box coordinates of the building instance according to the sliced images;

[0075] Create a first mask for the building instance, where the first mask includes the coordinate offset relative to the large-scale remote sensing image, the rectangular box coordinates, and the index information of the current image slice, and the storage size of the first mask is the same as the size of the sliced image;

[0076] Create a second mask based on the large-scale remote sensing image, overlay the first mask and the second mask to obtain a third mask, and obtain the overlapping building instances according to the third mask. The size of the second mask and the third mask is the same as the size of the large-scale remote sensing image.

[0077] It should be noted that the first mask refers to the mask created for each building instance, the size of the first mask is the same as the size of the sliced image, the second mask refers to a mask with all 0s created based on the read large-scale remote sensing image and with the same size as the large-scale remote sensing image, and the third mask refers to the fused mask obtained after overlaying the first mask and the second mask.

[0078] In specific implementation, YOLOv8 is used to predict the 640*640 image slices that are cropped, and the mask mask and the coordinates [x0, y0, x1, y1] of the rectangular box of each building instance are obtained. Among them, x0 and y0 represent the coordinates of the upper left corner of the rectangular box, and x1 and y1 represent the coordinates of the lower right corner of the rectangular box. A 640*640 mask is created for each building instance in the image slice. In the building instance mask, the position value of the building is 1, and other positions are 0. And the coordinate offset (start_x, start_y) relative to the large-scale remote sensing image, the coordinates [x0, y0, x1, y1] of the rectangular box, the index idx of the current slice image, etc. are saved. Then a mask with all 0s and the same size as the large-scale remote sensing image is created, and then the mask of each building instance is superimposed on this mask. Where there is an overlap of buildings after superposition, the pixel value will increase and become 2, 3..., where there is no overlap but there is a building, the pixel value is still 1, and where there is no building, the pixel value is 0.

[0079] Step S30: Determine the image slice to which the overlapping building instance belongs, and determine the set of overlapping building rectangles and independent targets within the neighborhood range according to the index of the image slice to which it belongs.

[0080] It should be noted that the neighborhood range refers to an area within a certain range around the index of the current image slice. For example, the indexes of the 8 image slices around the index of the current image slice.

[0081] It should be explained that the set of overlapping building rectangles is a set used to record overlapping building rectangles, and an independent target refers to a building instance that does not overlap with other building rectangles.

[0082] In specific implementation, according to the index of the image slice to which the current building instance belongs, calculate the indexes of all other image slices within its 8-neighborhood range, referring to Figure 6 , Figure 6 is a schematic diagram of the 8-neighborhood of the image slice. Find all other building instances within this range according to the index, traverse the rectangular boxes of each building instance, calculate whether there is an overlap with the rectangular boxes of other building instances within this range, and if so, add the index of the building instance to the same set. After traversing, a list of sets is obtained, and the sets with intersections are merged until there are no intersections between all sets after merging. The building instances corresponding to the rectangular boxes that do not overlap are used as independent targets.

[0083] Step S40: Screen the overlapping set to obtain a list of overlapping sets. According to the screened list of overlapping sets, calculate the mask overlap rate of each building instance in each set with other building instances in the set; add the instances with the mask overlap rate higher than the threshold to the instance object set, and use the instances with the mask overlap rate lower than the threshold as independent targets.

[0084] In a specific implementation, in the process of judging whether building instances overlap according to pixel values, if the judgment result is that there are overlapping building instances, then when obtaining the overlapping building instances, a set list of all target boxes with overlap can be obtained, and the entire set list can be traversed to merge the sets with intersections inside until there is no intersection between any two sets. After the merging of the rectangular boxes is completed, after obtaining the set list, calculate the overlap rate between the mask of the building instances in the set and the masks of other building instances. When the mask overlap rate between two buildings is higher than the set overlap threshold, a list set composed of building instances with a high mask overlap rate in the set can be obtained, and the masks of all building instances in the set are merged to generate a new building instance mask. Generating a new building instance mask includes calculating the rectangular size of each building instance in the set, creating a mask full of 0s according to the rectangular size, and projecting the building instance masks in the set onto the created mask to generate a new building instance mask, and calculating the coordinate offset of the new building instance mask based on the original large-scale building remote sensing image. When the mask overlap rate between two buildings is lower than the set overlap threshold, the current building instance mask is used as an independent building target mask.

[0085] Step S50: Generate building mask information according to the instance object set, and obtain all building instance mask information according to the building mask information and the mask information of the independent targets.

[0086] In a specific implementation, refer to Figure 7 , Figure 7 is the comparison diagram before and after the original mask fusion. Merge the obtained building mask information of the independent targets and the newly generated building mask information to obtain the final all building instance mask information.

[0087] In this embodiment, image slices are obtained by overlapping and cropping large-scale remote sensing images. Instance segmentation is performed on the image slices to obtain overlapping building instances. The image slices to which the overlapping building instances belong are determined, and the overlapping set and independent targets within the neighborhood range are determined according to the indexes of the image slices to which they belong. According to the coincidence rate of each building instance in the overlapping set with other building instances in the overlapping set, the instances with a coincidence rate higher than the threshold are added to the instance object set, and the instances with a coincidence rate lower than the threshold are used as independent targets. Building mask information is generated according to the instance object set, and all building instance mask information is obtained according to the building mask information and the mask information of the independent targets. It realizes the merging of the original masks that are cut into more than two parts at the cropping edge, restores the true mask of the object at the cropping edge, and solves the problem that the object is cut into multiple parts after overlapping cropping.

[0088] Reference Figure 8 , Figure 8 is a schematic flowchart of the second embodiment of a method for identifying ground objects in a large-scale remote sensing image according to the present invention.

[0089] Based on the above first embodiment, in step S30 of the method for identifying ground objects in a large-scale remote sensing image in this embodiment, it further includes:

[0090] Step S301: Determine the index values of other image slices within the neighborhood range of the index of the image slice to which it belongs according to the index of the image slice to which it belongs.

[0091] Step S302: Determine other building instances within the neighborhood range according to the index values of the other image slices.

[0092] Step S303: Traverse the building instances to obtain the rectangular frames of the building instances.

[0093] Step S304: Detect the rectangular frames of the building instances. When there is an overlap in the rectangular frames of the building instances, add the index of the building instance to the overlapping set.

[0094] Step S305: When there is no overlap in the rectangular frames of the building instances, determine the building instance as an independent target.

[0095] In specific implementation, each image slice has a corresponding index value, and each image slice can contain several building instances. Therefore, it is possible to determine the image slice to which the current instance belongs and determine the corresponding index value according to the image slice to which it belongs. After obtaining the corresponding index value of the image slice to which it belongs, according to Figure 6 the 8-neighborhood shown, obtain the index values of other image slices within the neighborhood range. After obtaining the 8-neighborhood image slice indexes, find all other building instances within this range, traverse the rectangular frames of each building instance, and refer to Figure 9 ,Figure 9 It is a schematic diagram of overlapping rectangular frames. Calculate whether the rectangular frames of other building instances within this range overlap with it. If there is an overlap, add the index of the building instance to the same set. Assume that the coordinates of the current rectangular frame are [x0, y0, x1, y1], and the coordinates of another rectangular frame are [x2, y2, x3, y3]. If max(x0, x2) ≤ min(x1, x3) && max(y0, y2) ≤ min(y1, y3), it indicates that the two rectangular frames overlap. If the above relationship is not satisfied, it means that the two rectangular frames do not overlap, and the building instance is determined as an independent target.

[0096] Further, after adding the index of the building instance to the overlap set when there is an overlap in the rectangular frame of the building instance, it further includes:

[0097] Traverse the overlap set to obtain the merge flag bits of the building instances in the overlap set;

[0098] When there is an intersection between the building instance in the overlap set and other building instances in the overlap set, update the flag bit of the building instance in the overlap set to True;

[0099] Merge the building instances with the flag bit being True, and detect the merged building instances;

[0100] If there is an intersection between the merged building instance and other building instances in the overlap set, update the flag bit of the merged building instance to True, and execute the steps of merging the building instances with the flag bit being True and detecting the merged building instances;

[0101] When there is no intersection between the building instance in the overlap set and other building instances in the overlap set, add the building instance in the overlap set to the result list.

[0102] In a specific implementation, an empty list result can be created to store the final result. The variable merged represents a flag indicating whether merging is to be performed. After creation, the flag is initialized to False. The overlapping sets are traversed, and during the traversal, it can be determined whether there is an intersection between the building instances in the set and other building instances in the set. If there is an intersection, the corresponding building instance's flag is updated to True, and the building instances with the flag set to True are merged to obtain the merged building instances. After the merging is completed, the merged building instances are subjected to the same detection to determine whether there is still an intersection with other building instances, and this process is continuously looped until there is no intersection between the merged building instances and other building instances. At this time, the flag of the merged building instances is changed to False, and the merged building instances are added to result. If during the judgment process, it is directly obtained that there is no intersection between the current building instance and other building instances in the set, then the current building instance can also be directly added to result.

[0103] Further, before determining the index values of other picture slices within the neighborhood range of the index of the picture slice to which it belongs according to the index of the picture slice to which it belongs, it further includes:

[0104] Obtaining the number of slices in the horizontal direction and the number of slices in the vertical direction based on the size of the large-scale remote sensing image and the overlapping cropping size;

[0105] Creating a matrix based on the number of slices in the horizontal direction and the number of slices in the vertical direction, where the number of rows and columns of the matrix is the same as the number of slices in the horizontal direction and the number of slices in the vertical direction.

[0106] In a specific implementation, calculate the number of slices in the horizontal direction and the number of slices in the vertical direction after cropping. The formula is:

[0107]

[0108] Where m and n respectively represent the number of slices in the horizontal direction and the number of slices in the vertical direction, W and H respectively represent the width and height of the original remote sensing image, w and h respectively represent the width and height of the sliced image. In this embodiment, w = h = 640, overlap_w and overlap_h respectively represent the overlapping pixel values in the horizontal direction and the vertical direction. In this embodiment, overlap_w = overlap_h = 32, and math_ceil represents rounding up.

[0109] After obtaining the number of slices in the horizontal and vertical directions, create a matrix Z with m rows and n columns, where Z ∈ [0, m*n - 1], based on the number of slices m and n in the horizontal and vertical directions. Assume the index of the current building instance mask is cur_indx, and find its index (i, j) in Z according to the value of cur_indx. Create an empty list neighbors to store the 8-neighborhood indices of (i, j). First, check whether the indices in the four directions of up, down, left, and right are valid. If valid, add them to neighbors. Then, check whether the four corner indices are valid. If valid, add them to neighbors. Finally, find the corresponding values in Z according to all the indices in neighbors, which are the 8-neighborhood values corresponding to cur_indx.

[0110] In this embodiment, by classifying the building mask into overlapping building instances and independent building instances according to whether there is overlap. Since there is no image transformation for independent building instances, the overlapping building instances are affected during cropping. Therefore, when fusing overlapping building instances, the possible positions of the overlapping areas are determined to be within the 8-neighborhood range of the corresponding image slices. This not only ensures finding all overlapping building instances but also greatly reduces the computational complexity, realizing the merging of the original mask that is cut into more than two parts at the cropping edge, restoring the true mask of the object at the cropping edge, and well solving the problem that the object is cut into multiple parts after overlapping cropping.

[0111] In addition, an embodiment of the present invention further provides a storage medium, on which a ground object recognition program for large-scale remote sensing images is stored. When the ground object recognition program for large-scale remote sensing images is executed by a processor, the steps of the ground object recognition method for large-scale remote sensing images as described above are implemented.

[0112] Refer to Figure 10 , Figure 10 which is the structural block diagram of the first embodiment of the ground object recognition device for large-scale remote sensing images of the present invention.

[0113] As Figure 10 shown, the ground object recognition device for large-scale remote sensing images proposed by an embodiment of the present invention includes:

[0114] An image slicing module 10, configured to perform overlapping cropping on a large-scale remote sensing image to obtain picture slices;

[0115] An instance segmentation module 20, configured to perform instance segmentation on the picture slices to obtain overlapping building instances;

[0116] A slice overlapping module 30, configured to determine the picture slices to which the overlapping building instances belong, and determine the overlapping set and independent targets within the neighborhood range according to the indices of the picture slices to which they belong;

[0117] The instance screening module 40 is used to screen the overlapping set to obtain a list of overlapping sets. According to the screened list of overlapping sets, calculate the mask overlap rate between each building instance in each set and other building instances in the set; add the instances with the mask overlap rate higher than the threshold to the instance object set, and use the instances with the mask overlap rate lower than the threshold as independent targets.

[0118] The mask fusion module 50 is used to generate building mask information according to the instance object set, and obtain the mask information of all building instances based on the building mask information and the mask information of the independent targets.

[0119] In this embodiment, the large-scale remote sensing image is overlappingly cropped to obtain picture slices, the picture slices are instance-segmented to obtain overlapping building instances, the picture slices to which the overlapping building instances belong are determined, the overlapping sets and independent targets within the neighborhood range are determined according to the indexes of the picture slices to which they belong, and according to the coincidence rate between each building instance in the overlapping set and other building instances in the overlapping set, the instances with the coincidence rate higher than the threshold are added to the instance object set, the instances with the coincidence rate lower than the threshold are used as independent targets, building mask information is generated according to the instance object set, and the mask information of all building instances is obtained based on the building mask information and the mask information of the independent targets. It realizes the merging of the original masks that are cut into more than two parts at the cropping edge, restores the true masks of the objects at the cropping edge, and solves the problem that the objects are cut into multiple parts after overlapping cropping.

[0120] In one embodiment, the instance segmentation module 20 is further used to obtain the mask and rectangular frame coordinates of the building instance according to the picture slice.

[0121] Create a first mask for the building instance. The first mask includes the coordinate offset relative to the large-scale remote sensing image, the rectangular frame coordinates, and the index information of the current picture slice. The storage size of the first mask is the same as the size of the picture slice.

[0122] Create a second mask based on the large-scale remote sensing image, superimpose the first mask and the second mask to obtain a third mask, and obtain the overlapping building instances according to the third mask. The sizes of the second mask and the third mask are the same as the size of the large-scale remote sensing image.

[0123] In one embodiment, the slice overlapping module 30 is further configured to determine the index values of other picture slices within the neighborhood range of the index of the picture slice to which it belongs according to the index of the picture slice to which it belongs; determine other building instances within the neighborhood range according to the index values of the other picture slices; traverse the building instances to obtain the rectangular frames of the building instances; detect the rectangular frames of the building instances, and when there is an overlap in the rectangular frames of the building instances, add the index of the building instance to the overlap set; when there is no overlap in the rectangular frames of the building instances, determine the building instance as an independent target.

[0124] In one embodiment, the slice overlapping module 30 is further configured to traverse the overlap set to obtain the merge flag bits of the building instances in the overlap set; when there is an intersection between the building instances in the overlap set and other building instances in the overlap set, update the flag bit of the building instance in the overlap set to True; merge the building instances with the flag bit being True, and detect the merged building instances; when there is an intersection between the merged building instances and other building instances in the overlap set, update the flag bit of the merged building instance to True, and execute the steps of merging the building instances with the flag bit being True and detecting the merged building instances; when there is no intersection between the building instances in the overlap set and other building instances in the overlap set, add the building instances in the overlap set to the result list.

[0125] In one embodiment, the slice overlapping module 30 is further configured to obtain the number of slices in the horizontal direction and the number of slices in the vertical direction according to the large-scale remote sensing image size and the overlapping cropping size; create a matrix according to the number of slices in the horizontal direction and the number of slices in the vertical direction, where the number of rows and columns of the matrix is the same as the number of slices in the horizontal direction and the number of slices in the vertical direction.

[0126] In one embodiment, the slice overlapping module 30 is further configured to store the index of the picture slice to which it belongs in an index list; judge the validity of the index values in the four directions of up, down, left, and right of the index of the picture slice to which it belongs, and add the valid indexes to the index list; judge the validity of the index values at the four corners of the index of the picture slice to which it belongs, and add the valid indexes to the index list; obtain the index values of other picture slices within the neighborhood range according to the index values recorded in the index list.

[0127] In one embodiment, the mask fusion module 50 is further configured to obtain the coordinate offset from the large-scale remote sensing image according to the building mask information and the mask information of the independent target; and add the building mask information and the mask information of the independent target to the large-scale remote sensing image according to the coordinate offset to obtain all the building instance mask information.

[0128] It should be understood that the above is only an example and does not constitute any limitation to the technical solution of the present invention. In specific applications, those skilled in the art can set according to needs, and the present invention does not make any limitation thereto.

[0129] It should be understood that although the steps in the flowcharts in the embodiments of the present application are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise clearly stated in this article, the execution of these steps is not strictly limited in order, and they can be executed in other orders. Moreover, at least a part of the steps in the figure may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

[0130] It should be noted that the above-described work process is only illustrative and does not limit the protection scope of the present invention. In actual applications, those skilled in the art can select some or all of them according to actual needs to achieve the purpose of the solution of this embodiment, and no limitation is imposed here.

[0131] In addition, it should be noted that in this article, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or system. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or system including that element.

[0132] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments.

[0133] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as a read-only memory (ROM) / RAM, magnetic disk, optical disk), and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0134] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. A method for identifying ground objects in large-scale remote sensing images, characterized in that, The method for identifying ground objects in large-scale remote sensing images includes: Overlapping and cropping the large-scale remote sensing image to obtain picture slices; Obtaining the mask and rectangular frame coordinates of building instances based on the picture slices; Creating a first mask for the building instance, where the first mask includes coordinate offsets relative to the large-scale remote sensing image, rectangular frame coordinates, and index information of the current picture slice, and the storage size of the first mask is the same as the size of the picture slice; Creating a second mask based on the large-scale remote sensing image, superimposing the first mask and the second mask to obtain a third mask, and obtaining overlapping building instances based on the third mask. The sizes of the second mask and the third mask are the same as the size of the large-scale remote sensing image; Determining the picture slices to which the overlapping building instances belong, and determining the overlapping set and independent targets within the neighborhood range based on the indexes of the belonging picture slices; Filtering the overlapping set to obtain an overlapping set list. According to the filtered overlapping set list, calculating the mask overlapping rate of each building instance in each set with other building instances in the set; adding the instances with the mask overlapping rate higher than the threshold to the instance object set, and taking the instances with the mask overlapping rate lower than the threshold as independent targets; Generating building mask information based on the instance object set, and obtaining the mask information of all building instances based on the building mask information and the mask information of the independent targets.

2. The method according to claim 1, characterized in that, The step of determining the picture slices to which the overlapping building instances belong and determining the overlapping set and independent targets within the neighborhood range based on the indexes of the belonging picture slices includes: Determining the index values of other picture slices within the neighborhood range of the index of the belonging picture slice according to the index of the belonging picture slice; Determining other building instances within the neighborhood range according to the index values of the other picture slices; Traversing the building instances to obtain the rectangular frames of the building instances; Detecting the rectangular frames of the building instances. When there is an overlap in the rectangular frames of the building instances, adding the index of the building instance to the overlapping set; When there is no overlap in the rectangular frames of the building instances, determining the building instance as an independent target.

3. The method according to claim 2, wherein After adding the index of the building instance to the overlapping set when there is an overlap in the rectangular frames of the building instances, it further includes: Traversing the overlapping set to obtain the merge flag bits of the building instances in the overlapping set; When there is an intersection between the building instance in the overlapping set and other building instances in the overlapping set, updating the flag bit of the building instance in the overlapping set to True; Merging the building instances with the flag bit being True, and detecting the merged building instances; When there is an intersection between the merged building instance and other building instances in the overlapping set, updating the flag bit of the merged building instance to True, and executing the steps of merging the building instances with the flag bit being True and detecting the merged building instances; When there is no intersection between the building instance in the overlapping set and other building instances in the overlapping set, add the building instance in the overlapping set to the result list.

4. The method according to claim 2, wherein Before determining the index values of other image slices within the neighborhood range of the index of the image slice to which it belongs according to the index of the image slice to which it belongs, it further includes: Obtain the number of slices in the horizontal direction and the number of slices in the vertical direction based on the size of the large-scale remote sensing image and the overlapping cropping size; Create a matrix based on the number of slices in the horizontal direction and the number of slices in the vertical direction, where the number of rows and columns of the matrix is the same as the number of slices in the horizontal direction and the number of slices in the vertical direction.

5. The method according to claim 2, characterized in that, Determining the index values of other image slices within the neighborhood range of the index of the image slice to which it belongs according to the index of the image slice to which it belongs includes: Store the index of the image slice to which it belongs in an index list; Judge the validity of the index values in the four directions of up, down, left, and right of the index of the image slice to which it belongs, and add the valid indexes to the index list; Judge the validity of the index values at the four corners of the index of the image slice to which it belongs, and add the valid indexes to the index list; Obtain the index values of other image slices within the neighborhood range according to the index values recorded in the index list.

6. The method according to any one of claims 1 to 5, characterized in that Generating building mask information based on the instance object set, and obtaining all building instance mask information based on the building mask information and the mask information of the independent target includes: Obtain the coordinate offset from the building mask information and the mask information of the independent target to the large-scale remote sensing image; Add the building mask information and the mask information of the independent target to the large-scale remote sensing image according to the coordinate offset to obtain all building instance mask information.

7. An apparatus for identifying ground objects in large-scale remote sensing images, characterized in that, The device for identifying ground objects in the large-scale remote sensing image includes: An image slicing module for overlappingly cropping the large-scale remote sensing image to obtain image slices; An instance segmentation module for obtaining the mask and rectangular box coordinates of the building instance according to the image slice; creating a first mask for the building instance, the first mask including the coordinate offset relative to the large-scale remote sensing image, the rectangular box coordinates, and the index information of the current image slice, and the storage size of the first mask is the same as the size of the image slice; creating a second mask based on the large-scale remote sensing image, and superimposing the first mask and the second mask to obtain a third mask, and obtaining overlapping building instances according to the third mask, and the size of the second mask and the third mask is the same as the size of the large-scale remote sensing image; A slice overlapping module for determining the image slice to which it belongs according to the overlapping building instance, and determining the overlapping set and the independent target within the neighborhood range according to the index of the image slice to which it belongs; An instance screening module, configured to screen the overlapping set to obtain a list of overlapping sets, and calculate the mask overlap rate between each building instance in each set and other building instances in the set according to the screened list of overlapping sets; add the instances with the mask overlap rate higher than the threshold to the instance object set, and use the instances with the mask overlap rate lower than the threshold as independent targets. A mask fusion module, configured to generate building mask information according to the instance object set, and obtain the mask information of all building instances based on the building mask information and the mask information of the independent targets.

8. An object recognition device for large-scale remote sensing images, characterized in that, The device includes: a memory, a processor, and a ground object recognition program for large-scale remote sensing images stored on the memory and operable on the processor, and the ground object recognition program for large-scale remote sensing images is configured to implement the steps of the ground object recognition method for large-scale remote sensing images according to any one of claims 1 to 6.

9. A storage medium, characterized in that, A ground object recognition program for large-scale remote sensing images is stored on the storage medium, and when the ground object recognition program for large-scale remote sensing images is executed by a processor, it implements the steps of the ground object recognition method for large-scale remote sensing images according to any one of claims 1 to 6.

Images