Remote sensing image detection method, device, electronic equipment and storage medium

Abstract

The present invention proposes a remote sensing image detection method, device, electronic device and storage medium, which belongs to the field of image processing and solves the problems of slow detection speed and low accuracy of existing remote sensing image target detection. The method includes: cropping the remote sensing image to obtain multiple images to be tested; for each image to be tested, using a pre-trained target detection model to obtain a target frame in the image to be tested; splicing the images to be tested with target detection results according to the cropping position corresponding to the image to be tested to obtain a tested image; deduplicating the target frame in the tested image to obtain the detection result of the remote sensing image. The present invention reduces the amount of data when the model processes a single image, shortens the detection time, and eliminates more external interference through model detection and target frame deduplication, thereby improving the detection speed and accuracy.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a remote sensing image detection method, device, electronic equipment and storage medium. Background Art

[0002] Remote sensing image target detection is to obtain the location and category information of the target object on the remote sensing image obtained by remote sensing technology, so as to carry out detection tasks such as geospatial intelligence acquisition, urban planning, precision agriculture, geological survey, etc. With the rapid development of remote sensing technology, the quantity and quality of remote sensing image data have exploded, and it is becoming increasingly difficult to accurately and quickly detect the target objects on remote sensing images.

[0003] Currently, manual detection and screening are usually used to identify the contours of target objects in remote sensing images. However, the data size of remote sensing images is large, with one remote sensing image being at least several GB, or even dozens of GB. The detection speed is slow and the detection accuracy is low. Summary of the invention

[0004] In view of this, an object of the present invention is to provide a remote sensing image detection method, device, electronic device and storage medium to more quickly and accurately perform target detection of remote sensing images.

[0005] In order to achieve the above purpose, the technical solution adopted by the embodiment of the present invention is as follows:

[0006] In a first aspect, an embodiment of the present invention provides a remote sensing image detection method, the method comprising:

[0007] Acquire a remote sensing image to be detected, and crop the remote sensing image to obtain multiple images to be detected;

[0008] For each of the images to be tested, a pre-trained target detection model is used to process the image to be tested to obtain a target detection result of the image to be tested; wherein the target detection result includes a target frame;

[0009] According to the cropping position corresponding to the image to be tested, the images to be tested with the target detection result are spliced ​​to obtain a tested image;

[0010] The target frame in the measured image is deduplicated to obtain a detection result of the remote sensing image.

[0011] Optionally, the step of cropping the remote sensing image to obtain a plurality of images to be measured includes:

[0012] Cropping the remote sensing image according to a preset cropping size to obtain a plurality of preliminary cropped images;

[0013] For each of the preliminary cut images, based on the position of the preliminary cut image, the boundary of the preliminary cut image is expanded to obtain the image to be tested, so that there is an overlapping area between two adjacent images to be tested; wherein the size of the overlapping area is a preset size expansion complete value.

[0014] Optionally, when the cropping order of the remote sensing image is from left to right and from top to bottom, the step of performing boundary expansion on the preliminary cropped image based on the position of the preliminary cropped image to obtain the image to be measured includes:

[0015] When the initially cut image is not the first image in the horizontal cutting direction, the left boundary line of the initially cut image is used as the left expansion starting line, a first expansion area with the left expansion starting line as the right boundary line is cut out from the remote sensing image, and the first expansion area is added to the initially cut image;

[0016] When the initially cut image is not the first one in the vertical cropping direction, the upper boundary line of the initially cut image is used as the upper expansion starting line, and a second expansion area with the upper expansion starting line as the lower boundary line is cropped from the remote sensing image, and the second expansion area is added to the initially cut image.

[0017] Optionally, the target detection result further includes a category of a target frame, and the step of removing duplicates of the target frame in the detected image to obtain the detection result of the remote sensing image includes:

[0018] Classifying the target frames of the same category in the measured image into the same category group to obtain multiple category groups;

[0019] For each of the category groups, deduplicate target frames of the category group;

[0020] The remaining target frames in each of the category groups are used as the detection results of the remote sensing image.

[0021] Optionally, the target detection result also includes the size and center point of the target box, and the step of performing target box deduplication on the category group includes:

[0022] Calculating the area of ​​each target frame of the category group according to the size of the target frame, and sorting the target frames of the category group in descending order of area;

[0023] The target frame ranked first is used as a duplicate-free frame;

[0024] Taking each of the target frames that are ranked after the deduplication frame as a secondary frame of the deduplication frame, and calculating an intersection-and-union ratio between the deduplication frame and each of the secondary frames according to the center point and the size;

[0025] Deleting the secondary bitboxes whose intersection-over-union ratio is greater than a preset repetition threshold;

[0026] Determine whether the deduplication frame is the last target frame of the category group;

[0027] If yes, then end the deduplication of the category group;

[0028] If not, the target frame whose sequence is after the duplicate removal frame is used as the next duplicate removal frame, and the process returns to the step of using each target frame whose sequence is after the duplicate removal frame as the next frame of the duplicate removal frame.

[0029] Optionally, the step of calculating the intersection-and-union ratio between the deduplication frame and each of the secondary frames according to the center point and the size includes:

[0030] For each of the secondary frames, an overlapping area between the deduplication frame and the secondary frame is determined according to the center point and the size, and a ratio between an area of ​​the overlapping area and an area of ​​the deduplication frame is used as an intersection-over-union ratio between the deduplication frame and the secondary frame.

[0031] Optionally, before the step of acquiring the remote sensing image to be detected and cropping the remote sensing image to obtain a plurality of images to be detected, the method further includes:

[0032] Acquire multiple original remote sensing images, and crop each of the original remote sensing images into multiple sample images;

[0033] For each of the sample images, annotating a target frame position label and a category label of the sample image according to the contour position and category of the target object in the sample image;

[0034] The labeled sample images are used to train the initial model to obtain a mature target detection model.

[0035] In a second aspect, an embodiment of the present invention provides a remote sensing image detection device, including a cropping module, a detection module, a splicing module and a deduplication module;

[0036] The cropping module is used to obtain a remote sensing image to be detected, and crop the remote sensing image to obtain a plurality of images to be detected;

[0037] The detection module is used to process each of the images to be tested using a pre-trained target detection model to obtain a target detection result of the image to be tested; wherein the target detection result includes a target frame;

[0038] The stitching module is used to stitch the images to be tested with the target detection results according to the cropping positions corresponding to the images to be tested, so as to obtain a tested image;

[0039] The deduplication module is used to deduplicate the target frame in the measured image to obtain the detection result of the remote sensing image.

[0040] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a processor and a memory, wherein the memory stores a computer program that can be executed by the processor, and the processor can execute the computer program to implement the remote sensing image detection method as described in the first aspect.

[0041] In a fourth aspect, an embodiment of the present invention provides a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the remote sensing image detection method as described in the first aspect.

[0042] In a fifth aspect, an embodiment of the present invention provides a computer program product, which includes computer instructions, wherein when the computer instructions are executed by a processor, the remote sensing image detection method as described in the first aspect is implemented.

[0043] The remote sensing image detection method, device, electronic device and storage medium provided by the embodiments of the present invention have at least the following beneficial effects:

[0044] By cropping the remote sensing image into multiple smaller images to be tested and using the target detection model to detect the images to be tested, the amount of data required by the model to process a single image is reduced, which greatly shortens the detection time and improves the detection speed. At the same time, the target detection model is used for target detection, and the images to be tested that have been tested by the model are spliced ​​back to the tested images of the same size as the original remote sensing images. The target frames of the tested images are deduplicated, eliminating more external interference and making the obtained detection results more accurate.

[0045] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0047] Figure 1 A block diagram of a remote sensing image detection system provided by an embodiment of the present invention is shown.

[0048] Figure 2 A block diagram of an electronic device provided by an embodiment of the present invention is shown.

[0049] Figure 3 One of the flow charts of the remote sensing image detection method provided by an embodiment of the present invention is shown.

[0050] Figure 4 Shows Figure 3 Schematic diagram of the process flow of some sub-steps of step 11.

[0051] Figure 5 The second flowchart of the remote sensing image detection method provided by the embodiment of the present invention is shown.

[0052] Figure 6 Shows Figure 3 Schematic diagram of the process flow of some sub-steps of step 15.

[0053] Figure 7 Shows Figure 6 A flowchart of some sub-steps of step 152 in FIG.

[0054] Figure 8 A block diagram of a remote sensing image detection device provided by an embodiment of the present invention is shown.

[0055] Explanation of the accompanying drawings: 10 - remote sensing image detection system; 110 - remote sensing equipment; 120 - detection equipment; 130 - client; 20 - electronic equipment; 210 - memory; 220 - processor; 230 - communication module; 30 - remote sensing image detection device; 310 - cropping module; 320 - detection module; 330 - stitching module; 340 - deduplication module. DETAILED DESCRIPTION

[0056] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0057] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

[0058] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0059] The remote sensing image detection method provided by the embodiment of the present invention can be applied to Figure 1 In the remote sensing image detection system 10 shown, the remote sensing image detection system 10 may include a remote sensing device 110, a detection device 120, and a client 130. The remote sensing device 110 may be connected to the detection device 120 by wireless communication, satellite communication, or network communication, and the detection device 120 may be connected to the client 130 by network, wired, or wireless communication.

[0060] A well-trained target detection model may be pre-deployed in the detection device 120 .

[0061] The remote sensing device 110 is used to perform remote sensing detection on the target area, obtain remote sensing images, and send the remote sensing images to the detection device 120.

[0062] The detection device 120 is used to obtain the detection result of the remote sensing image by using the remote sensing image detection method provided by the embodiment of the present invention, and send the detection result or the warning information corresponding to the detection result to the client 130.

[0063] The client 130 is used to receive and display the detection result or the warning information corresponding to the detection result.

[0064] Among them, the remote sensing device 110 can be but is not limited to: artificial satellites, drones, lidars, aerial cameras, etc., the detection equipment 120 can be but is not limited to: servers, server clusters, personal computers, laptop computers, and the client 130 can be but is not limited to: personal computers, laptop computers, tablet computers, mobile terminals, smart terminals, wearable portable devices, VR devices, etc.

[0065] Please refer to Figure 2 , is a block diagram of an electronic device 20, the electronic device 20 can be Figure 1The detection device 120 in the remote sensing image detection system 10. The electronic device 20 includes a memory 210, a processor 220 and a communication module 230. The memory 210, the processor 220 and the communication module 230 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0066] The memory 210 is used to store programs or data. The memory 210 may be, but is not limited to, a random access memory 210 (RAM), a read only memory 210 (ROM), a programmable read-only memory 210 (PROM), an erasable programmable read-only memory 210 (EPROM), an electrically erasable read-only memory 210 (Electric Erasable Programmable Read-Only Memory 210 (EEPROM), etc.

[0067] The processor 220 is used to read / write data or programs stored in the memory 210 and perform corresponding functions, such as executing a computer program stored in the memory 210 to implement the remote sensing image detection method provided in the embodiment of the present invention.

[0068] The communication module 230 is used to establish a communication connection between the electronic device 20 and other communication terminals through the network, and to send and receive data through the network, for example, to receive remote sensing images sent by the remote sensing device 110, and to send the detection results of the remote sensing images or warning information corresponding to the detection results to the client 130.

[0069] It should be understood that Figure 2 The structure shown is only a schematic diagram of the structure of the electronic device 20. The electronic device 20 may also include Figure 2 More or fewer components as shown, or with Figure 2 Different configurations are shown. Figure 2 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0070] Please refer to Figure 3 , which is one of the flowcharts of the remote sensing image detection method provided by the embodiment of the present invention, includes steps 11 to 17. In this embodiment, the execution subject of steps 11 to 17 can be Figure 1The detection device 120 in the detection device 120, for example, the processor 220 in the detection device 120 executes the computer program stored in the memory 210 to implement steps 11 to 17.

[0071] Step 11, obtaining a remote sensing image to be detected, and cropping the remote sensing image to obtain a plurality of images to be detected.

[0072] Step 13: for each image to be tested, a pre-trained target detection model is used to process the image to be tested to obtain a target detection result of the image to be tested.

[0073] Among them, the target detection result includes a target frame, which can be understood as the outline frame of the target object to be identified. The target object can be but is not limited to: objects that need to be identified in various scenes such as people, animals, ships, water levels, houses, etc.

[0074] Step 15, according to the cropping position corresponding to the image to be tested, the images to be tested with the target detection results are spliced ​​to obtain the tested image.

[0075] Step 17, deduplicate the target frame in the measured image to obtain the detection result of the remote sensing image.

[0076] For example, in combination Figure 1 As shown, a remote sensing device 110 such as a drone or an artificial satellite performs remote sensing detection on a target area on the earth, obtains a remote sensing image, and sends the remote sensing image to a detection device 120. After receiving the remote sensing image to be detected, the detection device 120 cuts the remote sensing image into multiple images to be tested, wherein there may be no overlapping areas between the images to be tested, or there may be overlapping areas (i.e., areas with the same content). Thus, the detection device 120 calls a pre-trained target detection model, performs target detection on the images to be tested, and obtains the target detection result of each image to be tested, that is, obtains the target frame of the target object in each image to be tested.

[0077] After determining the target detection results of all images to be tested, the detection device 120 can splice the images to be tested with the target detection results according to the corresponding positions of the images to be tested in the remote sensing image to obtain the tested image. At this time, the tested image contains the target frame of each target object and the target frame information (such as the category, width, height and center point of the target frame). Then, the detection device 120 removes the duplication of the target frame in the tested image to obtain the detection result of the remote sensing image, that is, the final target frame and the category, width, height and center point of the target frame.

[0078] The detection device 120 can directly send the detection result to the client 130, and the remote sensing image and the detection result can be displayed jointly in the client 130, that is, the position, size and category of the target frame of each target object in the remote sensing image. It is also possible to determine whether there is a dangerous object (or a hidden danger object) based on the category of the target frame. If it is determined that there is a dangerous object (or a hidden danger object) or a risk event, a warning message is generated, and the warning message and the detection result are sent to the client 130 together. The client 130 jointly displays the warning message, the detection result and the remote sensing image, and issues an alarm.

[0079] The detection device 120 can also determine whether the distribution of the target boxes meets the preset risk event characteristics based on the position of each target box (i.e., the center point and width and height). The risk events include but are not limited to highway collapse, high water level in reservoirs, landslides, mud-rock flows, etc. If it is determined that the distribution of the target boxes meets the preset risk event characteristics, a warning message is generated, and the warning message and the detection results are sent to the client 130 together. The client 130 activates the risk event response mechanism to issue an alarm, and jointly displays the warning message, the detection results, and the remote sensing image.

[0080] Compared with the traditional detection method, the above steps 11 to 17 of the remote sensing image detection method provided by the embodiment of the present invention, by cutting the remote sensing image into multiple smaller images to be tested and using the target detection model to detect the images to be tested, reduce the amount of data when the model processes the image, greatly shorten the detection time, and thus improve the detection speed. At the same time, the target detection model is used to perform target detection, and the images to be tested that have been detected by the model are spliced ​​back to the tested images of the same size as the original remote sensing images, and the target frames of the tested images are deduplicated, eliminating more external interference, so that the obtained detection results are more accurate.

[0081] Among them, in step 11, a variety of methods can be used to crop the remote sensing image. For example, continuous cropping can be performed without repetition, that is, there is no repeated content in multiple images to be tested and the detection of the images to be tested exists in an adjacent continuous relationship. Repeated cropping can also be performed, that is, there are repeated areas between adjacent images to be tested. It is also possible to crop only the part with a data volume greater than a preset data volume threshold, which is not limited in this embodiment.

[0082] In order to make the target object in each image to be tested as complete as possible and improve the detection accuracy, the concept of repeated cropping is introduced in the cropping of remote sensing images. Figure 4 , step 11 may include steps 111 to 112.

[0083] Step 111 , cropping the remote sensing image according to a preset cropping size to obtain a plurality of preliminary cropped images.

[0084] Step 112: for each preliminary cut image, based on the position of the preliminary cut image, the boundary of the preliminary cut image is expanded to obtain an image to be tested, so that there is an overlapping area between two adjacent images to be tested.

[0085] The size of the overlapping area is a preset size expansion value, which can be 5cm×5cm or 8cm×7cm, which is not limited in this embodiment. In addition, the overlapping area is the content of the remote sensing image and has a content continuity relationship with the preliminary cut image.

[0086] In step 111, the cropping can be performed according to a preset cropping size and in a variety of cropping orders, for example, from the middle to the periphery, from right to left and from top to bottom, or from left to right and from top to bottom, which is not limited in this embodiment. It should be noted that when the size of the last remaining image in each cropping direction is less than the cropping size, the remaining image is used as a preliminary cropped image.

[0087] In order to reduce the amount of data processing during target detection and further improve the detection speed, the concept of different contents of each overlapping area is introduced in the boundary extension of the preliminary cut image in step 112, that is, the concept of unique content of each overlapping area.

[0088] For example, when the cropping order of the remote sensing image is from left to right and from top to bottom, in step 112, a single expansion can be performed based on the position of the initially cropped image in the cropping direction: when the initially cropped image is not the first one in the horizontal cropping direction, the left boundary line of the initially cropped image is used as the left expansion starting point line, and a first expansion area with the left expansion starting point line as the right boundary line is cropped from the remote sensing image, and the first expansion area is added to the initially cropped image; when the initially cropped image is not the first one in the vertical cropping direction, the upper boundary line of the initially cropped image is used as the upper expansion starting point line, and a second expansion area with the upper expansion starting line as the lower boundary line is cropped from the remote sensing image, and the second expansion area is added to the initially cropped image; when the initially cropped image is the first one in the horizontal cropping direction, the initially cropped image is not expanded to the left; when the initially cropped image is the first one in the vertical cropping direction, the initially cropped image is not expanded upward.

[0089] Finally, the preliminary cut image expanded with the first expansion area and / or the second expansion area, and the preliminary cut image without the first expansion area and the second expansion area are both used as the image to be tested.

[0090] It should be understood that the first expansion area may also be a right expansion area (i.e., an image area with the right boundary line of the initially cut image as the left boundary line), and the second expansion area may also be a lower expansion area (i.e., an image area with the lower boundary line of the initially cut image as the upper boundary line), or other expansion methods may be used. The above expansion method is only an example and is not limited in this embodiment.

[0091] In this way, each image to be tested has at most two expansion areas (which can also be understood as two overlapping areas), and each expansion area only appears once in the expansion area, so that the outline of the target object in the image to be tested is as complete as possible. At the same time, the amount of data that needs to be processed by the target detection model is reduced as much as possible, and the detection time is shortened, thereby making the detection accuracy higher and the detection speed improved.

[0092] In order to improve the integrity of the target object in each image to be tested, a pre-set object boundary recognition model or object boundary recognition rule can also be used to identify the object boundary line of two adjacent (including adjacent left and right and adjacent up and down) preliminary cut images to determine the boundary line between different objects. Therefore, when it is determined that there is an incomplete object in the preliminary cut image based on the boundary line between the objects, the preliminary cut image is used as the image to be expanded, and the segmented area of ​​the incomplete object is extracted from the adjacent preliminary cut image based on the object boundary line, and the segmented area is expanded to the image to be expanded to obtain the image to be tested. When there is no incomplete object in the preliminary cut image, the preliminary cut image is used as the image to be tested.

[0093] After expanding all the preliminary images and obtaining the image to be tested, in step 13, the pre-deployed target detection model is called to perform target detection on the input image to be tested, and the image to be tested with the target detection result is output. The target detection result may include multiple different target boxes, and also include the center point, size (size includes the width and height of the target box), category and category confidence of each target box. It should be noted that different target boxes represent different target objects.

[0094] Among them, the target detection model used in step 13 can be a model trained with any basic model, for example, it can be a YOLO series model such as a YOLOv5n model, a YOLOv5x model, or a Faster R-CNN model, or an Anchor-Free model, which is not limited in this embodiment.

[0095] When training the model, you can choose models of different sizes according to the needs of the usage scenario. For example, in the context of single target object detection, if you want to achieve detection efficiency, you can choose the YOLOv5n model. In the complex context of multi-target object (i.e., multiple different target objects) detection, in order to improve detection accuracy, you can choose the YOLOv5x model.

[0096] After selecting the model, refer to Figure 5 , the target detection model can be trained by following steps 21 to 25.

[0097] Step 21, obtaining multiple original remote sensing images, and cropping each original remote sensing image into multiple sample images.

[0098] The regions and / or scenes of the multiple original remote sensing images are different.

[0099] Step 23: for each sample image, annotate the target frame position label and category label of the sample image according to the contour position and category of the target object in the sample image.

[0100] Step 25, using the labeled sample images to train the initial model to obtain a mature target detection model.

[0101] When the initial model is a model of the YOLO system, any labeling tool (such as labelImg) can be used to label the sample image. In step 25, the label of the sample image can be first converted from the xml file into the YOLO format, and then the initial model can be trained using any model training method or optimization method such as gradient descent method, supervised learning, etc. to obtain a mature target detection model. Thus, the target detection model is deployed to the detection device 120, and the detection device 120 can execute step 13 to obtain the target detection result of the image to be tested.

[0102] Furthermore, in step 15, each pre-cut image corresponding to the image to be tested has a unique center point, and the images to be tested with the target detection results are spliced ​​according to the position of the center point of each image to be tested to obtain a tested image. The images in the overlapping area are directly superimposed and not deleted.

[0103] On the basis of the above, due to overlapping areas, external interference, and the detection capability of the target detection model, repeated target frames are likely to exist in the measured image. In order to filter out repeated target frames in the measured image to improve the accuracy of the detection result, in step 15, multiple methods can be used to deduplicate the target frames in the measured image.

[0104] For example, the target boxes may be sorted by confidence, and then the IOU between the target boxes may be calculated, and only one of the target boxes whose IOU value reaches a preset threshold may be retained. Alternatively, multiple overlapping target boxes may be determined, and only one of the multiple target boxes may be retained.

[0105] In order to avoid filtering out overlapping target frames of different target objects in deduplication and improve detection accuracy, the concept of deduplication by category is introduced in step 15. For example, referring to Figure 6 , step 15 may include steps 151 to 153.

[0106] Step 151 , classifying the target frames of the same category in the measured image into the same category group to obtain multiple category groups.

[0107] Step 152: for each category group, deduplicate the target boxes of the category group.

[0108] Step 153, taking the remaining target frames in each category group as the detection results of the remote sensing image.

[0109] Among them, in step 153, multiple methods can be used to deduplicate target frames in the category group. For example, the overlapping area or the intersection and union ratio can be calculated in descending order of the confidence of the target frames in the category group to deduplicate. A pre-trained machine learning model can also be used for deduplication. Deduplication can also be performed according to preset rules, which are not specifically limited in this embodiment.

[0110] In order to make the remaining target frame after deduplication contain the complete outline of the target object and improve the completeness performance of target detection, the idea of ​​calculating the intersection and union ratio in order of area is introduced in the deduplication of target frames of category groups in step 152. Figure 7 , step 152 may include steps 1521 to 1527.

[0111] Step 1521, calculate the area of ​​each target frame of the category group according to the size of the target frame, and sort the target frames of the category group in order from large to small in area.

[0112] Step 1522: The target frame ranked first is used as a deduplicated frame.

[0113] Step 1523: Take each target frame that is ranked after the deduplication frame as a secondary frame of the deduplication frame, and calculate the intersection-and-union ratio between the deduplication frame and each secondary frame according to the center point and size.

[0114] Step 1524: delete the secondary bitboxes whose intersection-over-union ratio is greater than a preset duplication threshold.

[0115] Step 1525, determine whether the deduplication frame is the last target frame of the category group. If yes, execute step 1526, if not, execute step 1527.

[0116] Step 1526, end the deduplication of the category group.

[0117] Step 1527, taking the target frame whose sequence is after the duplicate removal frame as the next duplicate removal frame, and returning to step 1523 after step 1527.

[0118] In step 1523, the area of ​​the overlapping region between the deduplication frame and the secondary frame may be determined, and the ratio of the area of ​​the overlapping region to the area of ​​the deduplication frame may be used as the intersection-over-union ratio.

[0119] For each secondary frame, the overlap area between the deduplication frame and the secondary frame can be determined according to the center point and size, and then the ratio between the area of ​​the overlap area and the area of ​​the deduplication frame is calculated to obtain the intersection-union ratio between the deduplication frame and the secondary frame.

[0120] The method for calculating the area can be flexibly set, for example, a geometric method, an analytical method, or a digital topographic map measurement method can be used, which is not limited in this embodiment.

[0121] In addition, in step 1524, if the intersection-over-union ratio is greater than the preset duplication threshold, it means that the deduplication frame and the secondary frame are the contour frame of the same target object.

[0122] Through the above steps 151 to 153, and steps 1521 to 1527, the target frames are arranged according to their areas, the intersection-and-union ratios between the target frames are calculated from large to small, and the secondary frames whose intersection-and-union ratios are greater than the duplication threshold are filtered out, that is, the target frames with smaller contours are filtered out, so that the target frames remaining after deduplication are the largest target frames, so as to retain the complete contours of the target objects as much as possible, thereby greatly improving the integrity and accuracy of target detection.

[0123] Based on the same inventive concept as the above remote sensing image detection method, Figure 8 The embodiment of the present invention further provides a remote sensing image detection device 30, including a cropping module 310, a detection module 320, a splicing module 330 and a deduplication module 340. The remote sensing image detection device 30 provided in this embodiment can be applied to Figure 1 The detection device 120 in.

[0124] The cropping module 310 is used to obtain a remote sensing image to be detected, and crop the remote sensing image to obtain multiple images to be detected.

[0125] The detection module 320 is used to process each image to be tested using a pre-trained target detection model to obtain a target detection result of the image to be tested, wherein the target detection result includes a target frame.

[0126] The stitching module 330 is used to stitch the images to be tested with the target detection results according to the cropping positions corresponding to the images to be tested to obtain the tested images.

[0127] The deduplication module 340 is used to deduplicate the target frame in the detected image to obtain the detection result of the remote sensing image.

[0128] The remote sensing image detection device 30 may further include a model training module.

[0129] The model training module is used to obtain multiple original remote sensing images and crop each original remote sensing image into multiple sample images; for each sample image, the target frame position label and category label of the sample image are annotated according to the contour position and category of the target object in the sample image; the annotated sample images are used to train the initial model to obtain a mature target detection model.

[0130] In the above remote sensing image detection device 30, the remote sensing image is cropped into multiple smaller images to be tested through the synergistic effect of the cropping module 310, the detection module 320, the splicing module 330 and the deduplication module 340, and the target detection model is used to detect the images to be tested, which reduces the amount of data when the model processes the image, greatly shortens the detection time, and thus improves the detection speed. At the same time, the target detection model is used to perform target detection, and the images to be tested that have been detected by the model are spliced ​​back to the tested images of the same size as the original remote sensing images, and the target frames of the tested images are deduplicated, eliminating more external interference, so that the obtained detection results are more accurate.

[0131] For the specific implementation and effect of the remote sensing image detection device 30, please refer to the description of the implementation of the remote sensing image detection method above, such as, for the specific implementation and effect of the cropping module 310, please refer to the description of the relevant content of step 11 above, for the specific implementation and effect of the detection module 320, please refer to the description of the relevant content of step 13 above, for the specific implementation and effect of the stitching module 330, please refer to the description of the relevant content of step 15 above, for the specific implementation and effect of the deduplication module 340, please refer to the description of the relevant content of step 17 above, and no further details will be given here.

[0132] In addition, each module in the remote sensing image detection device 30 can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor 220 in the electronic device 20 in the form of hardware, or can be stored in the memory 210 of the electronic device 20 in the form of software, so that the processor 220 can call and execute the operations corresponding to each module to implement the remote sensing image detection method provided above.

[0133] The remote sensing image detection device 30 provided by the present invention can be implemented in the form of a computer program. The computer program can be Figure 2 The memory 210 of the electronic device 20 can store various program modules constituting the remote sensing image detection device 30, for example, Figure 8 The cropping module 310, the detection module 320, the splicing module 330 and the deduplication module 340 are shown. The computer program composed of the various program modules enables the processor 220 to execute the steps of the remote sensing image detection method described in this specification.

[0134] For example, Figure 2 The electronic device 20 shown can be Figure 8 The cropping module 310 in the remote sensing image detection device 30 shown in the figure performs step 11. The electronic device 20 can perform step 13 through the detection module 320. The electronic device 20 can perform step 15 through the stitching module 330. The electronic device 20 can perform step 17 through the deduplication module 340.

[0135] Based on the same inventive concept as the above-mentioned remote sensing image detection method, an embodiment of the present invention also provides an electronic device 20, including a processor 220 and a memory 210, the memory 210 stores a computer program that can be executed by the processor 220, and the processor 220 can execute the computer program to implement the remote sensing image detection method proposed in the embodiment of the present invention.

[0136] The embodiment of the present invention further provides a storage medium on which a computer program is stored. When the computer program is executed by the processor 220, the remote sensing image detection method proposed in the embodiment of the present invention is implemented.

[0137] In addition, an embodiment of the present invention further provides a computer program product, which includes computer instructions, wherein when the computer instructions are executed by the processor 220, the remote sensing image detection method proposed in the embodiment of the present invention is implemented.

[0138] In several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0139] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0140] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory 210 (ROM, Read-Only Memory), random access memory 210 (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

[0141] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A remote sensing image detection method, characterized in that: The method comprises: Acquire a remote sensing image to be detected, and crop the remote sensing image to obtain multiple images to be detected; For each of the images to be tested, a pre-trained target detection model is used to process the image to be tested to obtain a target detection result of the image to be tested; wherein the target detection result includes a target frame; According to the cropping position corresponding to the image to be tested, the images to be tested with the target detection result are spliced ​​to obtain a tested image; Deduplication of the target frame in the measured image to obtain a detection result of the remote sensing image; The step of cropping the remote sensing image to obtain a plurality of images to be measured includes: Cropping the remote sensing image according to a preset cropping size to obtain a plurality of preliminary cropped images; For each of the preliminary cut images, based on the position of the preliminary cut image, the boundary of the preliminary cut image is extended to obtain the image to be tested, so that there is an overlapping area between two adjacent images to be tested; wherein the size of the overlapping area is a preset size expansion complete value; When the cropping order of the remote sensing image is from left to right and from top to bottom, the step of performing boundary expansion on the preliminary cropped image based on the position of the preliminary cropped image to obtain the image to be measured comprises: When the initially cut image is not the first image in the horizontal cutting direction, the left boundary line of the initially cut image is used as the left expansion starting line, a first expansion area with the left expansion starting line as the right boundary line is cut out from the remote sensing image, and the first expansion area is added to the initially cut image; When the initially cut image is not the first image in the vertical cutting direction, taking the upper boundary line of the initially cut image as the upper expansion starting line, cutting out a second expansion area with the upper expansion starting line as the lower boundary line from the remote sensing image, and adding the second expansion area to the initially cut image; When the initially cut image is the first image in the horizontal cutting direction, the initially cut image is not expanded to the left; when the initially cut image is the first image in the vertical cutting direction, the initially cut image is not expanded to the top; The preliminary cut image expanded with the first extended area and / or the second extended area, and the preliminary cut image without the first extended area and the second extended area are both used as images to be tested; wherein each of the images to be tested has at most two extended areas, and each extended area appears only once in the image to be tested.

2. The remote sensing image detection method according to claim 1, characterized in that: The target detection result also includes the category of the target frame. The step of removing duplicates of the target frame in the measured image to obtain the detection result of the remote sensing image includes: Classifying the target frames of the same category in the measured image into the same category group to obtain multiple category groups; For each of the category groups, deduplicate target frames of the category group; The remaining target frames in each of the category groups are used as the detection results of the remote sensing image.

3. The remote sensing image detection method according to claim 2, characterized in that: The target detection result also includes the size and center point of the target frame, and the step of removing duplicate target frames from the category group includes: Calculating the area of ​​each target frame of the category group according to the size of the target frame, and sorting the target frames of the category group in descending order of area; The target frame ranked first is used as a duplicate-free frame; Taking each of the target frames that are ranked after the deduplication frame as a secondary frame of the deduplication frame, and calculating an intersection-and-union ratio between the deduplication frame and each of the secondary frames according to the center point and the size; Deleting the secondary bitboxes whose intersection-over-union ratio is greater than a preset repetition threshold; Determine whether the deduplication frame is the last target frame of the category group; If yes, then end the deduplication of the category group; If not, the target frame whose sequence is after the duplicate removal frame is used as the next duplicate removal frame, and the process returns to the step of using each target frame whose sequence is after the duplicate removal frame as the next frame of the duplicate removal frame.

4. The remote sensing image detection method according to claim 3, characterized in that: The step of calculating the intersection-and-union ratio between the deduplication frame and each of the secondary frames according to the center point and the size comprises: For each of the secondary frames, an overlapping area between the deduplication frame and the secondary frame is determined according to the center point and the size, and a ratio between an area of ​​the overlapping area and an area of ​​the deduplication frame is used as an intersection-over-union ratio between the deduplication frame and the secondary frame.

5. The remote sensing image detection method according to claim 2, characterized in that: Before the step of acquiring the remote sensing image to be detected and cropping the remote sensing image to obtain a plurality of images to be detected, the method further includes: Acquire multiple original remote sensing images, and crop each of the original remote sensing images into multiple sample images; For each of the sample images, annotating a target frame position label and a category label of the sample image according to the contour position and category of the target object in the sample image; The labeled sample images are used to train the initial model to obtain a mature target detection model.

6. A remote sensing image detection device, characterized in that: Including cropping module, detection module, splicing module and deduplication module; The cropping module is used to obtain a remote sensing image to be detected, and crop the remote sensing image to obtain a plurality of images to be detected; The detection module is used to process each of the images to be tested using a pre-trained target detection model to obtain a target detection result of the image to be tested; wherein the target detection result includes a target frame; The stitching module is used to stitch the images to be tested with the target detection results according to the cropping positions corresponding to the images to be tested, so as to obtain a tested image; The deduplication module is used to dedupe the target frame in the measured image to obtain the detection result of the remote sensing image; The cutting module is used for: Cropping the remote sensing image according to a preset cropping size to obtain a plurality of preliminary cropped images; For each of the preliminary cut images, based on the position of the preliminary cut image, the boundary of the preliminary cut image is extended to obtain the image to be tested, so that there is an overlapping area between two adjacent images to be tested; wherein the size of the overlapping area is a preset size expansion complete value; When the cropping order of the remote sensing image is from left to right and from top to bottom, the step of performing boundary expansion on the preliminary cropped image based on the position of the preliminary cropped image to obtain the image to be measured comprises: When the initially cut image is not the first image in the horizontal cutting direction, the left boundary line of the initially cut image is used as the left expansion starting line, a first expansion area with the left expansion starting line as the right boundary line is cut out from the remote sensing image, and the first expansion area is added to the initially cut image; When the initially cut image is not the first image in the vertical cutting direction, taking the upper boundary line of the initially cut image as the upper expansion starting line, cutting out a second expansion area with the upper expansion starting line as the lower boundary line from the remote sensing image, and adding the second expansion area to the initially cut image; When the initially cut image is the first image in the horizontal cutting direction, the initially cut image is not expanded to the left; when the initially cut image is the first image in the vertical cutting direction, the initially cut image is not expanded to the top; The preliminary cut image expanded with the first extended area and / or the second extended area, and the preliminary cut image without the first extended area and the second extended area are both used as images to be tested; wherein each of the images to be tested has at most two extended areas, and each extended area appears only once in the image to be tested.

7. An electronic device, characterized in that: It comprises a processor and a memory, wherein the memory stores a computer program that can be executed by the processor, and the processor can execute the computer program to implement the remote sensing image detection method as described in any one of claims 1 to 5.

8. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the remote sensing image detection method according to any one of claims 1 to 5 is implemented.

Images