Target detection method and device, electronic device and storage medium

Abstract

The invention relates to a target detection method and device, an electronic device and a storage medium, and the method comprises the steps: obtaining a to-be-detected remote sensing image; segmenting the remote sensing image to be detected to obtain a plurality of segmented sub-images; performing classification based on a judgment result of whether the target object exists in each segmented sub-image, performing image filtering on each segmented sub-image according to a classification result, and retaining a target sub-image in which the target object exists; inputting the plurality of target sub-images into the trained target detection model, outputting a target activation thermodynamic diagram, carrying out region filtering based on the target activation thermodynamic diagram, and determining a target thermodynamic region in the target sub-images; and performing target detection on the target thermal region to generate a target detection result. According to the invention, the problem that the accuracy and efficiency of target detection are difficult to consider for remote sensing images with hundred million-level resolution is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, and in particular to a target detection method and device, an electronic device, and a storage medium. BACKGROUND

[0002] Remote sensing image target detection refers to automatically identifying and locating targets of interest from remote sensing images using computer vision and image processing techniques. These targets can be natural or man-made objects such as buildings, roads, vehicles, and aircraft.

[0003] With the breakthrough development of imaging technology and sensor performance, the resolution of aerial images obtained by unmanned aerial vehicles and satellites continues to improve (e.g., the resolution of high-resolution satellite images can reach 29200x27620 pixels). In this context, remote sensing image target detection technology has become a research hotspot in academia and industry. However, the target detection of ultra-high resolution remote sensing images still faces technical bottlenecks: on the one hand, high-precision calculations need to be performed on the entire image, resulting in low inference efficiency; on the other hand, the target occupies a very small proportion in the ultra-high resolution remote sensing image, and traditional methods often reduce accuracy to improve speed, resulting in the inability to meet the dual demands of accuracy and real-time performance for remote sensing image target detection.

[0004] Currently, there is no effective solution to the problem of balancing the accuracy and efficiency of target detection for billion-pixel resolution remote sensing images in related technologies. SUMMARY

[0005] Embodiments of the present application provide a target detection method, device, electronic device, and storage medium to solve the problem of balancing the accuracy and efficiency of target detection for billion-pixel resolution remote sensing images in related technologies.

[0006] In a first aspect, embodiments of the present application provide a target detection method, which includes:

[0007] obtaining a remote sensing image to be detected;

[0008] segmenting the remote sensing image to be detected to obtain multiple segmented sub-images; classifying based on the judgment result of whether a target object exists in each segmented sub-image, performing image filtering on each segmented sub-image according to the classification result, and retaining a target sub-image in which the target object is detected to exist;

[0009] inputting the multiple target sub-images into a trained target detection model respectively, outputting a target activation heat map, and performing region filtering based on the target activation heat map to determine a target heat region in the target sub-image;

[0010] performing target detection on the target heat region to generate a target detection result.

[0011] In some embodiments, the classification is based on the determination of whether the target object exists in each of the segmented sub-images, the image filtering is performed on each of the segmented sub-images according to the classification result, and the target sub-image in which the target object is detected to exist is retained.

[0012] The segmented sub-images are sequentially input into the trained binary classification filter, and the classification processing is performed on the detection result of whether the target object exists in the segmented sub-images, and the classification result is output.

[0013] The image filtering processing is performed on the segmented sub-image in which the classification result indicates that the target object does not exist, and the segmented sub-image in which the classification result indicates that the target object exists is retained as the target sub-image.

[0014] In some embodiments, the classification is based on the determination of whether the target object exists in each of the segmented sub-images, the image filtering is performed on each of the segmented sub-images according to the classification result, and the target sub-image in which the target object is detected to exist is retained.

[0015] The segmented sub-image is input into the convolution layer of the binary classification filter for feature extraction processing to obtain a deep feature map, and the deep feature map is input into the pooling layer of the binary classification filter for step-by-step compression processing to obtain a compressed feature map.

[0016] The binary classification filter is used to perform feature conversion processing on the compressed feature map to output a target existence probability value, and the classification result is determined based on the target existence probability value.

[0017] In some embodiments, the target sub-image is input into the trained target detection model to output a target activation heat map, including:

[0018] The target sub-image is input into the feature extraction layer of the target detection model for feature extraction processing to output a reduced dimension feature map.

[0019] The reduced dimension feature map is input into the target activation network of the target detection model for convolution processing to output the target activation heat map.

[0020] In some embodiments, the method further includes:

[0021] The target sub-image is divided into a plurality of image grids according to a preset division parameter.

[0022] input the image grid into the target detection model for processing, and output the target activation heat map; wherein the value of each position in the target activation heat map corresponds to the target nature score of each image grid.

[0023] In some embodiments, the region filtering based on the target activation heat map determines a target heat region in the target sub-image, including:

[0024] calculating the heat value of each region in the target activation heat map;

[0025] comparing the heat value with a preset heat threshold value, and determining the region with a heat value greater than the heat threshold value as the target heat region.

[0026] In some embodiments, the target detection on the target heat region generates a target detection result, including:

[0027] calculating the overlap degree value between each target heat region in each target sub-image;

[0028] based on the overlap degree value, performing deduplication processing on the target heat region to obtain a non-redundant region in each target sub-image;

[0029] performing target detection on the non-redundant region to generate the target detection result.

[0030] In a second aspect, the embodiments of the present application provide a target detection device, including:

[0031] an acquisition module configured to acquire a to-be-detected remote sensing image;

[0032] an image filtering module configured to perform segmentation processing on the to-be-detected remote sensing image to obtain a plurality of segmented sub-images; perform classification based on the judgment result of whether a target object exists in each segmented sub-image, perform image filtering on each segmented sub-image according to the classification result, and retain a target sub-image in which the target object is detected to exist;

[0033] a region filtering module configured to input each target sub-image into a trained target detection model, output a target activation heat map, and perform region filtering based on the target activation heat map to determine a target heat region in the target sub-image;

[0034] a detection module configured to perform target detection on the target heat region to generate a target detection result.

[0035] In a third aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the target detection method according to the first aspect when executing the computer program.

[0036] In a fourth aspect, a storage medium is provided, which stores a computer program executable by a processor to implement the target detection method according to the first aspect.

[0037] Compared with the related art, the target detection method, device, electronic device, and storage medium provided by the embodiments of the present application have the following advantages. The target detection method, device, electronic device, and storage medium provided by the embodiments of the present application acquire a to-be-detected remote sensing image; perform segmentation processing on the to-be-detected remote sensing image to obtain a plurality of segmented sub-images; classify based on a judgment result of whether a target object exists in each segmented sub-image, perform image filtering on each segmented sub-image according to a classification result, and retain a target sub-image in which the target object is detected to exist; input the plurality of target sub-images into a trained target detection model respectively, output a target activation heat map, and perform region filtering based on the target activation heat map to determine a target heat region in the target sub-image; and perform target detection on the target heat region to generate a target detection result.

[0038] Based on this, a multi-level filtering mechanism of image-level coarse screening and region-level fine filtering is introduced, which respectively implements image filtering on non-target image regions and region filtering on background regions, can effectively avoid redundant calculation on non-target regions, helps to compress the detection range of remote sensing images with an order of 100 billion resolution, significantly improves the processing efficiency, dynamically allocates computing resources, and simultaneously performs high-precision detection in a target dense region, which can improve the efficiency while ensuring the accuracy of target detection, thereby effectively solving the problem that the accuracy and efficiency of target detection cannot be considered for remote sensing images, especially remote sensing images with an order of 100 billion resolution.

[0039] Details of one or more embodiments of the present application are presented in the following drawings and description to make other features, objects, and advantages of the present application more apparent. BRIEF DESCRIPTION OF DRAWINGS

[0040] The accompanying drawings illustrated herein are used to provide further understanding of the present application, constitute a part of the present application, and the illustrative embodiments of the present application and their descriptions are used to explain the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0041] Figure 1 is a hardware structure block diagram of a terminal according to a target detection method according to an embodiment of the present application;

[0042] Figure 2 is a flowchart of a target detection method according to an embodiment of the present application;

[0043] Figure 3 is a flowchart of a PAB processing flow according to an embodiment of the present application;

[0044] Figure 4 is a schematic diagram of a target detection method according to an embodiment of the present application;

[0045] Figure 5 is a structural block diagram of a target detection apparatus according to an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be described and illustrated below in conjunction with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort belong to the scope of the present application. In addition, it can be understood that, although the efforts made in this development process can be complex and lengthy, some changes in design, manufacture or production and the like made by those of ordinary skill in the art related to the content disclosed in the present application on the basis of the technical content disclosed in the present application are only routine technical means and should not be understood as insufficient disclosure of the content disclosed in the present application.

[0047] In the present application, the phrase "embodiments" means that the specific features, structures or characteristics described in conjunction with the embodiments can be included in at least one embodiment of the present application. The appearance of this phrase at various places in the specification does not necessarily mean the same embodiment, nor is it an independent or alternative embodiment to other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in the present application can be combined with other embodiments without conflict.

[0048] Unless otherwise defined, technical terms and scientific terms used in the present application shall have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. Unless otherwise defined, the terms "one" and "a" or "an" used in the present application shall not be construed as being limited to a single element, but instead are used to represent one or more elements. The terms "including," "comprising," "having," and variations thereof in the present application are meant to encompass the elements listed thereafter and variations thereof, unless otherwise specified. For example, a process, method, system, product, or apparatus that comprises a list of steps or modules (units) is not limited to those steps or modules which are recited, but can also include additional steps or modules that are not expressly listed or can also include additional steps or modules that are inherent to such process, method, product, or apparatus. The terms "connected," "coupled," and variations thereof in the present application are not limited to direct connections, but can also include indirect connections or intermediate components that facilitate the connection between other components. The term "plurality" in the present application means two or more. The term "and / or" in the present application describes an associated relationship between associated objects, which means that there can be three relationships, for example, "A and / or B" can mean that A exists alone, A and B exist together, and B exists alone. The terms "first," "second," "third," and the like in the present application are merely used to distinguish similar objects, and do not represent a specific order of the objects.

[0049] The method embodiments provided by the present embodiment can be executed in a terminal, a computer, or a similar computing device. Taking a terminal as an example, Figure 1 is a hardware structure diagram of a terminal according to a target detection method of an embodiment of the present application. As shown in Figure 1 , the terminal can include one or more (only one is shown in Figure 1 ) processor 102 (the processor 102 can include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the above terminal can also include a transmission device 106 for communication function and an input and output device 108. Those skilled in the art can understand that Figure 1 the structure shown is only schematic, and does not limit the structure of the above terminal. For example, the terminal can include more or fewer components than those shown in Figure 1 , or have a different configuration from that shown in Figure 1 .

[0050] The memory 104 can be used to store computer programs, such as software programs of application software and modules, such as a computer program corresponding to the target detection method in the embodiments of the present application. The processor 102 performs various functional applications and data processing by running the computer programs stored in the memory 104, that is, implements the above method. The memory 104 can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 104 can further include a memory remotely arranged with respect to the processor 102, which can be connected to the terminal through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0051] The transmission device 106 is used to receive or send data via a network. Specific examples of the above network can include a wireless network provided by a communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC) which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, RF) module used to communicate with the Internet in a wireless manner.

[0052] The embodiments of the present application provide a target detection method, Figure 2 is a flowchart of a target detection method according to the embodiments of the present application, as shown in the figure, the flow includes the following steps: Figure 2

[0053] In step S210, a remote sensing image to be detected is acquired.

[0054] The remote sensing image to be detected specifically refers to an image of an order of 10 billion resolution acquired from a remote sensing data source (such as a satellite, a drone, etc.) and to be detected, for example, an image of an order of 29200x27620 pixels.

[0055] In the related art, the remote sensing image target detection method is usually for an image of an order of 1024x1024, and there is little research on an image of an order of 10 billion resolution. It should also be understood that for an image of an order of 10 billion resolution, usually the region containing the target accounts for less than 50%, and using a traditional backbone network to scan the entire high-resolution image will consume a large amount of computing resources and time, and an additional network model needs to be constructed to obtain the aggregation information, which wastes the inference time. Therefore, the embodiments of the present application optimize the target detection method in the subsequent steps for an image of an order of 10 billion resolution. ​

[0056] Step S220, the remote sensing image to be detected is segmented to obtain multiple segmented sub-images; based on the judgment result of whether there is a target object in each segmented sub-image, classification is performed, and image filtering is performed on each segmented sub-image according to the classification result, and a target sub-image in which a target object is detected is retained.

[0057] In this step, first, the original remote sensing image to be detected is segmented into multiple smaller sub-images, and the segmentation size or the number of segmented images can be set in advance according to actual conditions; for example, a super high-resolution image is cut according to a size of 1024*1024 per block, and the size of the overlapping part is 200 (that is, there is an overlapping area of 200 pixels in width between adjacent two sub-images). The purpose of segmentation is to facilitate subsequent processing, especially when the original image is too large or contains multiple independent regions; the segmented sub-images can be more easily processed and analyzed.

[0058] Next, image filtering is performed on the multiple segmented sub-images, and only the sub-images containing target objects are retained; wherein the target object refers to a specific object or feature in the remote sensing image that is of interest and is expected to be detected, which can be a ship, a vehicle, etc., and at present, it is mainly based on the target in the data set of Dota2. Specifically, for a high-resolution remote sensing image, some target objects only occupy a small part of the entire image, so after the image is segmented into multiple segmented sub-images, it can be judged in turn whether each segmented sub-image contains a target object and classified. The classification method can be a feature comparison method, using an image feature extraction method to extract features from a template image of the target object, for each segmented sub-image, extracting its features, and comparing the features of the sub-image with the features of the target object to calculate the similarity; if the similarity exceeds a certain threshold, it is considered that the sub-image contains a target object. Alternatively, to further improve processing efficiency, a classifier detection method can also be used to determine the classification result of each segmented sub-image. Subsequently, the sub-images classified as not containing a target object are filtered out, and finally, the sub-images determined to contain a target object are retained.

[0059] Step S230, the multiple target sub-images are respectively input into the trained target detection model, and a target activation heat map is output, and a region filtering is performed based on the target activation heat map to determine a target heat region in the target sub-image.

[0060] Specifically, the heat map reflects the confidence of each pixel point on the model predicting the target object. The model learns to identify the features of the target object during the training process, so the region related to the target object will show a higher activation value (or confidence) in the heat map, that is, from the target activation heat map, the high-confidence (such as higher than a certain confidence threshold) region can be extracted as the target heat region. In contrast, the background region and other irrelevant regions will show a lower activation value in the heat map because they do not contain the features of the target object, and thus need to be filtered; the purpose of region filtering is to extract the region most likely to contain the target object from the target sub-image. The final target heat region is the region filtered and optimized through the above steps, and more accurate target detection is performed in the target heat region to generate the target detection result (such as the bounding box and class information).

[0061] In step S240, target detection is performed on the target heat region to generate a target detection result.

[0062] In the above steps S230 to S240, the target sub-image is input into the trained target detection model, which is generally the head part of the target detection model, and is usually used to detect the target object in the image and output the bounding box and class information of the target.

[0063] Through the above target detection method, the multiple segmented sub-images segmented from the ultra-high resolution remote sensing image are filtered by image filtering, only the target sub-image determined to contain the target object is retained, and the retained target sub-image is loaded into the target detection model to filter the irrelevant image regions such as the background region, thereby introducing a multi-level filtering mechanism of image-level coarse screening and region-level fine filtering, respectively implementing image filtering of non-target image regions and region filtering of background regions, which can effectively avoid redundant calculation on non-target regions, help to compress the detection range of the billion-level resolution remote sensing image, significantly improve the processing efficiency, dynamically allocate computing resources, and at the same time perform high-precision detection in the target dense region, which can improve the efficiency while ensuring the accuracy of target detection, thus effectively solving the problem of difficult to balance the accuracy and efficiency of target detection for remote sensing images, especially billion-level resolution remote sensing images.

[0064] In some embodiments, the above classification based on the determination result of whether the target object exists in each segmented sub-image, and the image filtering of each segmented sub-image according to the classification result, and retaining the target sub-image in which the target object is detected to exist, can further include the following steps:

[0065] The segmented sub-images are sequentially input into the trained binary classification filter, and the detection results of whether there is a target object in the segmented sub-images are classified and processed, and the classification results are output; the segmented sub-images indicated by the classification results as not having a target object are subjected to image filtering processing, and the segmented sub-images indicated by the classification results as having a target object are retained as target sub-images.

[0066] The core objective of the binary classification filter network is to efficiently handle the binary classification task of large-sized images (1024x1024), which is to determine whether there is a target object in the input sub-image. The binary classification filter usually outputs a binary result: there is a target object or there is no target object. Its design revolves around "fast dimension reduction" and "lightweight feature extraction". Through multi-level pooling compression of spatial information and global semantic aggregation, the final output is the target existence probability. Specifically, each segmented sub-image is sequentially input into the trained binary classification filter. For each segmented sub-image, the binary classification filter outputs a classification result indicating whether there is a target object in the sub-image. According to this classification result, corresponding classification processing is performed. For segmented sub-images indicated by the classification results as not having a target object, image filtering processing is performed; this means that these sub-images will be discarded because they do not contain the target object of interest. For segmented sub-images indicated by the classification results as having a target object, they are retained as "target sub-images". These sub-images will be used for subsequent target detection or other processing steps.

[0067] Through the above embodiments, the binary classification filter is used to determine whether the segmented sub-image contains a target object, so that the computational load is smaller and the processing speed is faster when processing images, especially in application scenarios that require processing a large number of images.

[0068] In some embodiments, the above step of sequentially inputting the segmented sub-images into the trained binary classification filter and classifying and processing the detection results of whether there is a target object in the segmented sub-images to output the classification results can further include the following steps:

[0069] The segmented sub-images are input into the convolutional layer of the binary classification filter for feature extraction processing to obtain deep feature maps, and the deep feature maps are input into the pooling layer of the binary classification filter for step-by-step compression processing to obtain compressed feature maps; the binary classification filter is used to perform feature conversion processing on the compressed feature maps to output a target existence probability value, and the classification result is determined based on the target existence probability value.

[0070] Specifically, the segmented sub-images (with a size of 1024x1024) are input into the trained binary classification filter. The binary classification filter network adopts a "convolution-pooling" cascade structure, including the following three-stage convolution module processing:

[0071] In the first stage, the input sub-image is convoluted using a 3x3 convolution kernel (padding = 1, for maintaining the spatial size) to maintain the spatial size and output a feature map with 32 channels. A ReLU activation function is applied for nonlinear transformation. Then, a 4x4 max-pooling layer (stride = 4) is used to pool the feature map, compressing the feature map size from 1024x1024 to 256x256.

[0072] In the second stage, the 256x256 feature map is convoluted again using a 3x3 convolution kernel (padding = 1) to output a feature map with 64 channels. Then, a ReLU activation function is applied, followed by a 4x4 max-pooling layer (stride = 4) to pool the feature map and compress the feature map size to 64x64.

[0073] In the third stage, the 64x64 feature map is convoluted using a 3x3 convolution kernel (padding = 1) to output a feature map with 128 channels. Then, a ReLU activation function is applied, followed by a 4x4 max-pooling layer (stride = 4) to pool the feature map and compress the feature map size to 16x16.

[0074] Through the above three-stage processing, the deep feature map size of the input sub-image is gradually compressed from 1024x1024 to 16x16, while the channel number increases to 128. This preserves key features while significantly reducing computational complexity, reducing the computational complexity to 1 / 4096 of the original input.

[0075] Next, global feature compression is performed. The 16x16x128 feature tensor is input into an adaptive global average pooling layer, which is compressed into a 1x1x128 global description vector. This step replaces the flattening operation before the traditional fully connected layer, significantly reducing the parameter amount and improving the adaptability to large-sized images. Finally, the 1x1x128 global description vector is input into a fully connected layer (128→128 dimensions) for nonlinear mapping, and a Sigmoid activation function is applied to convert the high-dimensional features into target existence probability values in the 0-1 interval, forming a binary classification decision system. For example, according to the comparison result of the target existence probability value and the preset threshold (such as 0.5), the classification result of whether there is a target object in the segmented sub-image is determined. If the target existence probability value is greater than the threshold, it is determined that there is a target object; otherwise, it is determined that there is no target object.

[0076] Through the above embodiments, the binary classification filter provided above is used to gradually reduce the feature map size layer by layer using the "convolution-pooling" cascade structure, and the flattening operation before the fully connected layer is replaced by global average pooling, thereby significantly reducing the parameter amount and helping to improve the adaptability to large-sized images.

[0077] In some embodiments, the above-mentioned inputting the plurality of target sub-images into the trained target detection model respectively to output target activation heat maps can further include the following steps:

[0078] The target sub-image is input into the feature extraction layer in the target detection model for feature extraction processing to output a reduced dimension feature map; and the reduced dimension feature map is input into the target activation network in the target detection model for convolution processing to output a target activation heat map.

[0079] The feature extraction layer is an important component of the target detection model, which is usually composed of multiple convolution layers, pooling layers, etc. The convolution layer is responsible for extracting local features in the image by sliding the convolution kernel on the image; and the pooling layer is responsible for reducing the dimension of the feature map while retaining important features. After the target sub-image is input into the feature extraction layer, a series of convolution and pooling operations are performed to gradually extract high-level features in the image, and a reduced dimension feature map is output; this is the result of the processing of the feature extraction layer, which has a lower dimension than the original input image but contains more high-level features that are helpful for target detection.

[0080] On the other hand, the target activation network is another key part of the target detection model, which is usually composed of additional convolution layers, activation functions, etc.; its role is to further process the reduced dimension feature map to highlight the position of the target object in the image. This processing process is that after the reduced dimension feature map is input into the target activation network, a series of convolution operations are performed to output a target activation heat map. Each pixel value in the heat map represents the probability or activation degree of the position belonging to the target object. Generally, the areas with higher values in the heat map correspond to the positions where the target object may exist.

[0081] Through the above embodiments, by inputting the target sub-image into the feature extraction layer of the target detection model for feature extraction processing, the model can efficiently capture and condense the key features in the image, output a reduced dimension feature map, which not only reduces the computational burden but also improves the compactness and effectiveness of feature representation; then, the reduced dimension feature map is input into the target activation network for convolution processing, and the generated target activation heat map directly reveals the potential position of the target object in the image, providing accurate guidance for subsequent target positioning and recognition, thereby improving the accuracy and efficiency of target detection as a whole, enhancing the practicality and reliability of the model in actual application.

[0082] In some embodiments, the above-mentioned target detection method further includes the following steps:

[0083] The target sub-image is divided into a plurality of image grids according to a preset division parameter; the image grid is input into the target detection model for processing, and a target activation heat map is output; wherein in the target activation heat map, the value of each position corresponds to the target score of each image grid.

[0084] The target sub-image (e.g., a raw image block with a size of 1024x1024) is received, and is uniformly divided into a grid of S rows and S columns according to a preset division parameter SxS. For example, when S=16, the image is divided into 16x16=256 grid units, each with a size of 64x64 pixels (1024 / 16=64). The division parameter S can be dynamically adjusted according to ablation experiment results or actual scene requirements to optimize the target detection performance.

[0085] The target detection model described above can use ResNet50 as a backbone for feature extraction to enhance the deep feature extraction capability by using a residual connection mechanism. Specifically, referring to FIG. 3, the feature map C5 output by the last stage of ResNet50 is selected as the input, and the step length of the feature map relative to the input image is 32 pixels, i.e., the spatial dimension is 32x32 (1024 / 32=32). A 3x3 convolution kernel and a convolution layer with a step length of 2 (256 filters) are used to downsample the C5 feature map to realize step convolution dimension reduction, generating a downsampled feature map with a size of 16x16x256, and the spatial dimension is compressed from 32x32 to 16x16, while the number of channels is expanded to 256 to enhance the feature expression capability. Figure 3 The network architecture of the target detection model described above also includes a PAB network; the PAB is a fully convolutional network composed of two 1x1 convolution layers, and its core function is to quantify the target confidence of each image grid. As shown in FIG. 4, the first layer of the PAB network (512 filters) expands the channel dimension of the downsampled feature map to generate a feature map with a size of 16x16x512, further fusing multi-scale feature information. The second layer of the PAB network (1 filter) compresses the channel of the enhanced feature map to generate a target activation heat map M with a size of 16x16x1. The value of each position in the heat map M corresponds to the target score of an image grid, and the higher the score, the greater the probability that the grid contains a target.

[0086] Figure 3 Based on the target activation heat map M, the target detection model concentrates computing resources on grid regions with high target scores, such as by using non-maximum suppression (NMS) to filter candidate boxes, or using an adaptive anchor box generation mechanism to focus on high response regions. Based on the above method, more refined bounding box regression or classification tasks are performed on grid regions with high target scores to improve detection accuracy and reduce redundant calculations.

[0087] In some embodiments, the above-mentioned region filtering based on the target activation heat map to determine the target heat region in the target sub-image can further include the following steps:

[0088] In some embodiments, the above-mentioned region filtering based on the target activation heat map to determine the target heat region in the target sub-image can further include the following steps: ​

[0089] The heat value of each region in the target activation heat map is calculated, and the heat value is compared with a preset heat threshold value, and a region with a heat value greater than the heat threshold value is determined as a target heat region.

[0090] Specifically, the target activation heat map M is divided into SxS regions corresponding to the image grid, and each region corresponds to a target score set of an image grid. For each region, its heat value is calculated. The heat value can be determined in one of the following ways: taking the maximum value of the target score of all pixels in the region as the heat value of the region; calculating the average value of the target score of all pixels in the region as the heat value of the region; or, according to the pixel position or target score distribution, different weights are given, and the weighted sum is calculated as the heat value of the region.

[0091] The region heat value is then compared with a preset heat threshold value. The preset heat threshold value is set according to task requirements or experimental results, and is used to screen regions that may contain targets. If the region heat value > T, the region is determined to be a target heat region, indicating that the region has a high probability of containing a target. If the region heat value ≤ T, the region is determined to be a background region, indicating that the region has a low probability of containing a target. All regions that satisfy the heat value > T are retained as target heat regions for subsequent refined detection or analysis.

[0092] Through the above embodiments, by filtering through the heat threshold value, the interference of the background region on the target detection is effectively excluded, and the detection accuracy is improved; at the same time, the heat threshold value can be dynamically adjusted according to different scenes or task requirements, so that the model adapts to complex and variable detection environments, which is conducive to enhancing the robustness of the model.

[0093] In some embodiments, the above target detection on the target heat region to generate the target detection result can further include the following steps:

[0094] The overlap degree value between each target heat region in each target sub-image is calculated; based on the overlap degree value, the target heat region is de-duplicated to obtain a non-redundant region in each target sub-image; and target detection is performed on the non-redundant region to generate a target detection result.

[0095] Specifically, for each target heat region in each target sub-image, its corresponding Bounding Box coordinates are extracted. For any two target heat regions A and B in the same target sub-image, the IoU is calculated as shown in the following formula:

[0096] ;

[0097] In the above formula, Area(A∩B) is the area of ​​the intersection of the two regions, and Area(A∪B) is the area of ​​the union of the two regions. The IoU value between all thermal regions of the target in each target sub-image is calculated, thus obtaining the aforementioned overlap value.

[0098] In this embodiment, the Local Non-Maximum Suppression (Local NMS) algorithm is used to remove duplicates from the target thermal region. The specific steps are as follows: All regions are sorted in descending order based on their targetability score (or confidence level) to generate a region list L. Initialization result list. Extract the region r with the highest score from list L. max Add the result to list R. Iterate through the remaining region in list L and calculate its sum with r. max The IoU value. If the IoU value is greater than the preset threshold T NMS (Usually taken as 0.3-0.5), then remove that region from L (considered a redundant region). Repeat the above process until list L is empty. The regions in the final output list R are the deduplicated, non-redundant regions.

[0099] For the deduplicated, non-redundant regions, perform more refined target detection tasks, including but not limited to: bounding box regression, optimizing the bounding box coordinates to more accurately enclose the target; target classification to determine the specific category of the target (such as vehicles, buildings, etc.), and so on, and compile and generate target detection results for ultra-high resolution remote sensing images.

[0100] By employing the above method, for large-size images (such as aerial / satellite images), the principle of spatial locality is utilized to perform NMS processing only on local regions where targets may exist, further reducing computational complexity. Specifically, the search range of NMS can be dynamically adjusted based on the distribution density of the target's thermal region, thus realizing a large-scale detection computation optimization strategy based on local non-maximum suppression. It should also be noted that in large-scale image detection tasks, traditional Global Non-Maximum Suppression (GNMS) suffers from significant computational complexity issues. Taking a scenario with 1224 images and 10 candidate boxes per image as an example, the total number of candidate boxes that Global NMS needs to process is:

[0101] ;

[0102] At this point, the computational complexity of GNMS is:

[0103] ;

[0104] However, when using the Local Non-Maximum Suppression (Local NMS) strategy, the computational complexity will exhibit a linear growth characteristic:

[0105] ;

[0106] It can be seen that based on the above local non-maximum suppression algorithm, the divide-and-conquer computing paradigm is realized, the global detection task is decoupled into independent subtasks, and the order of magnitude of the computational complexity is reduced (k is the number of blocks), as shown in the following formula:

[0107] ;

[0108] The application will be described in detail below with specific embodiments. Figure 4 is a schematic diagram of a target detection method according to an embodiment of the application, as shown in Figure 4 , the method comprises the following steps:

[0109] Step S401, segmentation. A high-resolution remote sensing image is first cut according to the size of each block of 1024x1024, and the overlapping part is 200.

[0110] Step S402, BF: classify whether there is a target, and filter out the segmented sub-image without a target.

[0111] Step S403, PAB: continue to block the segmented picture, and pay attention to the region with a target. In this step, the retained target sub-image is input to the backbone network of the target detection model, the image features are output, and the target detection is performed; at the same time, the feature map is converted into a heat map, and the region where the target exists is extracted.

[0112] Step S404, local nms: adjust from global nms processing to local nms for region deduplication processing, and generate the final target detection result in combination with the target detection of the above steps.

[0113] Taking the detection requirement of a super high-resolution satellite image of 29,200x27,620 pixels as an example, the embodiment constructs a three-level processing pipeline of “blocking-filtering-detection”: first, the original image is cut into blocks of 1024x1024 (overlapping area of 200 pixels, a total of 1,224 sub-blocks are generated), and 690 effective regions are retained through a lightweight two-classification convolutional network (3-layer convolution+global average pooling structure, single block inference time is 3.2s); then, the C5 features are extracted by using the ResNet50 backbone network, the 16x16 heat map is generated by using the target activation network (PAN) to screen the high-probability region, and the directional detection head and the local NMS strategy are combined, so that the NMS processing time of the whole super high-resolution picture is reduced from 143s to 8.7s.

[0114] It is noted that the steps shown in the above flow or the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from here.

[0115] The embodiments also provide a target detection apparatus for implementing the above embodiments and preferred embodiments, which have been described and will not be repeated. As used below, the terms "module", "unit", "sub-unit" and the like can be a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware or a combination of software and hardware implementation is also possible and contemplated.

[0116] Figure 5 is a structural block diagram of a target detection apparatus according to an embodiment of the present application, as shown in Figure 5 The apparatus includes an acquisition module 51, an image filtering module 52, a region filtering module 53 and a detection module 54.

[0117] The acquisition module 51 is configured to acquire a remote sensing image to be detected. The image filtering module 52 is configured to perform segmentation processing on the remote sensing image to be detected to obtain a plurality of segmented sub-images; perform classification based on a judgment result of whether a target object exists in each segmented sub-image; perform image filtering on each segmented sub-image according to a classification result; and retain a target sub-image in which a target object is detected to exist. The region filtering module 53 is configured to input the plurality of target sub-images into a trained target detection model respectively, output a target activation heat map, and perform region filtering based on the target activation heat map to determine a target heat region in the target sub-image. The detection module 54 is configured to perform target detection on the target heat region to generate a target detection result.

[0118] In some embodiments, the image filtering module 52 is further configured to input each segmented sub-image into a trained binary classification filter in sequence, perform classification processing on a detection result of whether a target object exists in the segmented sub-image, and output a classification result. The image filtering module 52 performs image filtering processing on the segmented sub-image indicated by the classification result to indicate that no target object exists, and retains the segmented sub-image indicated by the classification result to indicate that a target object exists as a target sub-image.

[0119] In some embodiments, the image filtering module 52 is further configured to input the segmented sub-image into a convolution layer of the binary classification filter to perform feature extraction processing, to obtain a deep feature map, and to input the deep feature map into a pooling layer of the binary classification filter to perform step-by-step compression processing, to obtain a compressed feature map. The image filtering module 52 performs feature conversion processing on the compressed feature map using the binary classification filter, outputs a target existence probability value, and determines a classification result based on the target existence probability value.

[0120] In some embodiments, the region filtering module 53 is further configured to input the target sub-image into a feature extraction layer of the target detection model to perform feature extraction processing, to output a reduced dimension feature map, and to input the reduced dimension feature map into a target activation network of the target detection model to perform convolution processing, to output a target activation heat map.

[0121] In some embodiments, the region filtering module 53 is further configured to divide the target sub-image into a plurality of image grids according to a preset division parameter, to input the image grids into the target detection model for processing, and to output a target activation heat map. In the target activation heat map, the value of each position corresponds to a target score of each image grid.

[0122] In some embodiments, the region filtering module 53 is further configured to calculate a heat value of each region in the target activation heat map, and to compare the heat value with a preset heat threshold value, to determine a target heat region in which the heat value is greater than the heat threshold value.

[0123] In some embodiments, the detection module 54 is further configured to calculate an overlap degree value between each target heat region in each target sub-image, to perform deduplication processing on the target heat region based on the overlap degree value, to obtain a non-redundant region in each target sub-image, and to perform target detection on the non-redundant region, to generate a target detection result.

[0124] It should be noted that each of the above modules can be a functional module or a program module, and can be implemented by software or hardware. For the modules implemented by hardware, each of the above modules can be located in the same processor, or each of the above modules can be located in different processors in any combination. Specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation manners, which will not be described herein again.

[0125] The embodiment further provides an electronic device including a memory and a processor, the memory storing a computer program, and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0126] Optionally, the electronic device described above can further include a transmission device connected with the processor and an input and output device connected with the processor.

[0127] Optionally, in the embodiment, the processor can be configured to execute the following steps by means of a computer program:

[0128] S1, obtaining a remote sensing image to be detected.

[0129] S2, performing segmentation processing on the remote sensing image to be detected to obtain a plurality of segmented sub-images; classifying based on a judgment result of whether a target object exists in each segmented sub-image, performing image filtering on each segmented sub-image according to a classification result, and retaining a target sub-image in which a target object is detected to exist.

[0130] S3, inputting the plurality of target sub-images into the trained target detection model respectively, outputting a target activation heat map, and performing region filtering based on the target activation heat map to determine a target heat region in the target sub-image.

[0131] S4, performing target detection on the target heat region to generate a target detection result.

[0132] It should be noted that the specific examples in the embodiment can refer to the examples described in the above embodiments and optional implementation manners, and the embodiment will not be described here.

[0133] In addition, in combination with the target detection method in the above embodiments, the application embodiment can provide a storage medium for implementation. The storage medium has a computer program stored thereon; the computer program is executed by a processor to implement any one of the target detection methods in the above embodiments.

[0134] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties.

[0135] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0136] Those skilled in the art should understand that any combination of the technical features of the above-mentioned embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above-mentioned embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0137] The above-mentioned embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of protection of the present application. Therefore, the scope of protection of the patent of the present application should be subject to the appended claims.

Claims

1. A target detection method characterized by, The method comprises: acquiring a remote sensing image to be detected; segmenting the remote sensing image to be detected to obtain a plurality of segmented sub-images; classifying based on a judgment result of whether a target object exists in each segmented sub-image; performing image filtering on each segmented sub-image according to the classification result, and retaining a target sub-image in which the target object is detected to exist; inputting the plurality of target sub-images into a trained target detection model respectively, outputting a target activation heat map, and performing region filtering based on the target activation heat map to determine a target heat region in the target sub-image; performing target detection on the target heat region to generate a target detection result.

2. The object detection method of claim 1, wherein, The classification based on the judgment result of whether a target object exists in each segmented sub-image, the image filtering on each segmented sub-image according to the classification result, and the retaining of a target sub-image in which the target object is detected to exist comprise: inputting each segmented sub-image into a trained binary classification filter in turn, and performing classification processing on the detection result of whether a target object exists in the segmented sub-image, and outputting the classification result; performing image filtering processing on the segmented sub-image in which the classification result indicates that the target object does not exist, and retaining the segmented sub-image in which the classification result indicates that the target object exists as the target sub-image.

3. The object detection method of claim 2, wherein, The inputting of each segmented sub-image into a trained binary classification filter in turn and the performing of classification processing on the detection result of whether a target object exists in the segmented sub-image to output the classification result comprise: inputting the segmented sub-image into a convolution layer of the binary classification filter for feature extraction processing to obtain a deep feature map, and inputting the deep feature map into a pooling layer of the binary classification filter for step-by-step compression processing to obtain a compressed feature map; using the binary classification filter, performing feature conversion processing on the compressed feature map to output a target existence probability value, and determining the classification result based on the target existence probability value.

4. The object detection method of claim 1, wherein, The inputting of the plurality of target sub-images into a trained target detection model respectively to output a target activation heat map comprises: inputting the target sub-image into a feature extraction layer in the target detection model for feature extraction processing to output a reduced dimension feature map; inputting the reduced dimension feature map into a target activation network in the target detection model for convolution processing to output the target activation heat map.

5. The object detection method of claim 4, wherein, The method further comprises: dividing the target sub-image into a plurality of image grids according to a preset division parameter; inputting the image grid into the target detection model for processing, and outputting the target activation heat map; wherein the value of each position in the target activation heat map corresponds to a target score of each image grid.

6. The object detection method of claim 1, wherein, The region filtering based on the target activation heat map to determine the target heat region in the target sub-image comprises: calculating a heat value of each region in the target activation heat map; comparing the heat value with a preset heat threshold value, and determining a region in which the heat value is greater than the heat threshold value as the target heat region.

7. The target detection method according to any one of claims 1 to 6, characterized in that, The target detection is performed on the target thermal regions to generate a target detection result, including: calculating an overlap degree value between each target thermal region in each target sub-image; performing deduplication processing on the target thermal regions based on the overlap degree value to obtain a non-redundant region in each target sub-image; performing target detection on the non-redundant region to generate the target detection result.

8. A target detection apparatus characterized by comprising: comprising: an acquisition module configured to acquire a remote sensing image to be detected; an image filtering module configured to perform segmentation processing on the remote sensing image to be detected to obtain a plurality of segmented sub-images; performing classification based on a judgment result of whether a target object exists in each segmented sub-image, performing image filtering on each segmented sub-image according to a classification result, and retaining a target sub-image in which the target object is detected to exist; a region filtering module configured to input the plurality of target sub-images into a trained target detection model, output a target activation heat map, and perform region filtering based on the target activation heat map to determine a target thermal region in the target sub-image; a detection module configured to perform target detection on the target thermal region to generate a target detection result. 9.An electronic device comprising a memory and a processor, the electronic device characterized by, The memory stores a computer program, and the processor is configured to run the computer program to execute the target detection method in any one of claims 1 to 7.

10. A storage medium, characterized by The storage medium stores a computer program, wherein the computer program is configured to execute the target detection method in any one of claims 1 to 7 when running.

Images