A real-time target detection method and system in an ultrahigh-resolution image scene

Abstract

The application provides a real-time target detection method in an ultrahigh-resolution image scene, comprising: processing an original image to generate a high-resolution image and a low-resolution image; processing the low-resolution image to obtain a reserved area; cutting the high-resolution image according to the reserved area to generate a plurality of regional subgraphs; and detecting the regional subgraphs to obtain a detection result. The application can quickly detect the area requiring further fine-grained detection from an ultrahigh-resolution image through resolution processing, multi-scale density regression, area determination and area generation processing. In addition, due to the scale normalization, target detection and result merging processing, the final ultrahigh-resolution detection result can be generated. In the target detection stage, the plurality of regional subgraphs can be processed in parallel, which can greatly improve the inference time. Since part of the object-free area can be discarded in the density estimation stage, the required storage resources and calculation time can be saved.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and more specifically, to a real-time target detection method and system for ultra-high resolution image scenes. Background Technology

[0002] With the continuous maturation of CMOS (Complementary Metal Oxide Semiconductor) technology, cameras have become ubiquitous in our lives. For example, cameras are now standard equipment in smartphones and in public surveillance systems. To help people process massive amounts of image and video data faster and better, more and more image processing algorithms are being applied to everyday life, such as real-time facial recognition systems, real-time beautification systems, and autonomous driving systems. However, to truly apply these methods in real-world scenarios, most image processing methods face the challenge of "real-time" processing. Current major image processing methods are based on megapixel images and have already met current real-time requirements.

[0003] However, with the rapid development of computer vision and artificial intelligence technologies, computational imaging technologies have already broken through the resolution barrier from the megapixel level to the billion-pixel level, and the trend towards billion-pixel consumer-grade cameras is irreversible. While ultra-large pixels bring richer information, they also bring many new problems, such as: occupying more computing storage space and requiring longer processing latency. These problems are caused by the denser pixels, especially for real-time video stream data. If traditional computer vision processing methods are applied to ultra-high-resolution images, their processing speed is far from keeping up with the video stream's delivery speed, leading to increasingly longer latency in real-time video processing and making normal applications impossible. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the purpose of this invention is to provide a real-time target detection method and system for ultra-high resolution image scenes.

[0005] According to one aspect of the present invention, a real-time target detection method for ultra-high resolution image scenes is provided, comprising:

[0006] The original image is processed to generate high-resolution and low-resolution images;

[0007] The low-resolution image is processed to obtain the preserved region;

[0008] The high-resolution image is cropped based on the preserved region to generate multiple region sub-images that require fine-grained detection.

[0009] The region sub-image is detected to obtain the detection results.

[0010] Preferably, the process of processing the low-resolution image to obtain the preserved region includes:

[0011] Density estimation is performed on the low-resolution image to generate multiple density maps at different scales;

[0012] Each density map is processed to obtain the reserved region for each density map.

[0013] Preferably, the step of performing density estimation on the low-resolution image to generate multiple density maps at different scales includes:

[0014] Multiple scales are set according to the size of the target being detected;

[0015] The low-resolution image is used to generate multiple density maps based on the various scales.

[0016] Preferably, the processing of each density map to obtain the reserved region for each density map includes:

[0017] Each density map is divided into grids, and the probability of an object existing in each grid is calculated to determine whether to retain that region.

[0018] Preferably, the probability of the object's existence is calculated by integration within each grid, and a global threshold is used to determine whether to retain the region. If the probability is greater than the global threshold, the region will be retained; if the probability is less than the global threshold, the region will be discarded.

[0019] Preferably, each segmented grid is larger than the size of the target at that scale;

[0020] The larger the density map, the sparser the grid and the larger the area of ​​each grid region.

[0021] Preferably, the region sub-image is detected to obtain detection results, including:

[0022] Adjustments are made to the multiple regional sub-maps.

[0023] Perform an inspection on each adjusted region submap.

[0024] The detection results of each region sub-map are merged.

[0025] Preferably, adjusting the plurality of regional sub-maps includes:

[0026] The smallest scale is used as the standard scale, and the region submaps at other scales are scaled to the same size as the standard scale, keeping all region submaps the same size, and keeping the objects to be detected in the image at the same scale.

[0027] Preferably, the merging of detection results for each region sub-map includes:

[0028] The detection results of different sub-image regions are mapped back to the original image to obtain the detection results of the original image.

[0029] According to a second aspect of the present invention, a real-time target detection system for ultra-high resolution image scenes is provided, comprising:

[0030] A resolution processing module that processes the original image to generate high-resolution and low-resolution images;

[0031] A region determination module processes the low-resolution image to obtain a preserved region;

[0032] The scaling module crops the high-resolution image based on the preserved region, generating multiple sub-images of regions requiring fine-grained detection.

[0033] The target detection module detects the region sub-image and obtains the detection results.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] The real-time target detection method and system in ultra-high resolution image scenes in this embodiment of the invention can quickly detect regions that need further fine-grained detection in ultra-high resolution images through resolution processing, multi-scale density regression, region determination and region generation processing. It discards some object-free regions in the density estimation stage, thus saving the storage resources and computing time required for computation.

[0036] The real-time target detection method and system for ultra-high resolution image scenes in this invention, through processing such as scale normalization, target detection modulus, and result merging, can generate a final detection result with an ultra-high resolution ratio. Furthermore, it can perform parallel processing on sub-images of multiple regions during the target detection stage, significantly reducing inference time and improving detection efficiency. Attached Figure Description

[0037] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0038] Figure 1 This is a flowchart of a real-time target detection method in ultra-high resolution image scenes according to an embodiment of the present invention;

[0039] Figure 2This is a structural block diagram of a real-time target detection system in an ultra-high resolution image scene according to another embodiment of the present invention. Detailed Implementation

[0040] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0041] The rapid development of deep learning has brought new possibilities to various fields of computer vision and artificial intelligence. Object detection, as a fundamental computer vision image processing method, will impact changes in production and lifestyles across many sectors. However, good results in object detection typically rely on relatively clear object outlines. Crowd density estimation, on the other hand, does not aim for precise object location information but rather to estimate potential object locations even at low resolution.

[0042] Based on the above background, the present invention provides an embodiment, see [link to embodiment]. Figure 1 A real-time target detection method for ultra-high resolution image scenes, comprising:

[0043] S100 processes the original image to generate high-resolution and low-resolution images;

[0044] S200, the low-resolution image obtained in S100 is processed to obtain the preserved region;

[0045] The high-resolution images obtained in S300 and S100 are cropped based on the preserved region obtained in S200 to generate multiple sub-images of regions that require fine-grained detection.

[0046] S400: Detect the region sub-map obtained in S300 to obtain the detection result.

[0047] This embodiment first operates on a low-resolution image, a coarse-grained operation, which identifies the remaining regions, indicating where objects might exist. Then, a cropping process is performed on the high-resolution image to obtain a sub-image, retaining all pixels at the highest resolution. Further detection on this sub-image is called fine-grained detection. Fine-grained detection utilizes the richest pixel information for analysis, obtaining the specific location and semantic information of objects. Compared to coarse detection at low resolution, it has higher confidence and uses more usable pixel information. This embodiment, through resolution processing, multi-scale density regression, region determination, and region generation, enables rapid detection of regions requiring further fine-grained detection in ultra-high-resolution images, improving detection efficiency.

[0048] In a preferred embodiment of the invention, step S100 is implemented to process the original-size image into images of different resolutions. The low-resolution images can be used for subsequent coarse estimation, while the high-resolution images can be used for subsequent detailed detection.

[0049] In another embodiment of the present invention, S200 is implemented based on the low-resolution image generated in S100. Specifically, it includes:

[0050] S201, performs density estimation on low-resolution images to generate density maps at different scales;

[0051] S202, process each density map obtained in S201 to obtain the reserved region for each density map.

[0052] In a preferred embodiment, the different scales in S201 include: micro-scale, small-scale, medium-scale, and large-scale; generally, micro-scale: object area less than 800x800 pixels; small-scale: object area between 800x800 and 1600x1600 pixels; medium-scale: object area between 1600x1600 and 3200x3200 pixels; large-scale: object area greater than 3200x3200 pixels. Each scale regresses targets of different sizes; during regression execution, the ranges of the above four scales are set according to the size of the target to be detected. Four density maps are generated based on the above four scales.

[0053] In a preferred embodiment, in S202, each density map is divided into grids. Naturally, at each scale, the grid size is larger than the largest object at that scale; the larger the scale of the density map, the sparser the grid, and the larger the area of ​​each grid region. After division, the probability of an object's existence within each grid is calculated to determine whether to retain that region. Specifically, the probability of an object's existence within each grid is calculated through integration, and a global threshold is used to determine whether to retain the region. If the probability is greater than the global threshold, the region is retained; if the probability is less than the global threshold, the region is discarded. This embodiment saves storage resources and computation time by discarding some object-free regions.

[0054] In a preferred embodiment of the present invention, based on the reserved region obtained in S200 and the high-resolution image obtained in S100, S300 is implemented to crop the high-resolution image using the reserved region as an index, generate sub-images of different regions, and pass them to the next stage.

[0055] In a preferred embodiment of the present invention, S400 is implemented based on the region sub-image obtained by cropping 300. Specifically, it includes:

[0056] S401, adjusts multiple regional submaps.

[0057] S402, perform inspection on each adjusted region sub-map.

[0058] S403, merges the detection results of each region sub-map.

[0059] In a preferred embodiment, S401 uses the smallest microscale as the standard scale, and the region sub-images at other scales are scaled to the same size as the microscale. After adjustment, all region sub-images have the same scale, and the objects to be detected in the image maintain the same scale.

[0060] In a preferred embodiment, the S402 object detection method is a traditional megapixel object detection method, which has good replaceability and can be adapted to current mainstream object detection methods. During execution, multiple images can be processed in parallel, greatly reducing the latency required for computation.

[0061] In a preferred embodiment, S403 maps the detection results of different region sub-images back to the original image to obtain the detection results of the original image. The detection results are represented in coordinate form, and the mapping method is to add the coordinate position of the region sub-image in the original image to the detection result of the region sub-image, and the result is the actual position of the detection result in the region sub-image in the original image.

[0062] This embodiment combines a low-resolution crowd density estimation method with a high-resolution object detection method. First, density maps of objects at different scales are estimated at low resolution. Then, the number of people in each region is calculated based on the density maps, and local images of the regions containing the objects are generated on the high-resolution image. These local images at different scales are then detected, typically through scale normalization. An object detection algorithm is then used to detect each image normalized to the standard scale. Finally, the detection results from all local images are merged. This method significantly improves inference and detection time. Because some object-free regions can be discarded during the density estimation stage, it saves computational storage resources and computation time.

[0063] Based on the same inventive concept, another embodiment of the present invention provides a real-time target detection system for ultra-high resolution image scenes, including a resolution processing module, a region determination module, a scale processing module, and a target detection module. The resolution processing module processes the original image to generate a high-resolution image and a low-resolution image; the region determination module processes the low-resolution image to obtain a retained region; the scale processing module crops the high-resolution image based on the retained region to generate multiple region sub-images requiring fine-grained detection; and the target detection module detects the region sub-images to obtain detection results.

[0064] Based on further optimizations of the above embodiments, the present invention also provides a preferred embodiment, see [link to preferred embodiment]. Figure 2 A real-time target detection system for ultra-high resolution image scenes includes: a resolution processing module 1, a multi-scale density regression module 2, a region determination module 3, a region generation module 4, a scale normalization module 5, a target detection module 6, and a result merging module 7. The system in this embodiment is implemented on a Linux platform, which supports at least one graphics processing unit (GPU) card.

[0065] The resolution processing module 1 receives the original-size image, processes it into images of different resolutions, and then transmits them to subsequent modules. The low-resolution images can be used by subsequent modules for rough estimation, while the high-resolution images can be used by subsequent modules for detailed detection.

[0066] The multi-scale density regression module 2 further processes the low-resolution image generated in the resolution processing module 1 to obtain density maps at different scales. These include four scales: micro-scale, small-scale, medium-scale, and large-scale. Each scale regresses targets of different sizes, and the range of the four scales is set according to the size distribution of the targets to be detected during execution.

[0067] The region determination module 3 processes the four density maps separately and networks them according to the corresponding scales. Then, it calculates for each grid to determine whether the grid should be retained.

[0068] The region generation module 4 receives the results of the region retention from the region determination module 3 and the high-resolution image from the resolution processing module 1, and uses the region retention index to crop the high-resolution image to generate different region sub-images.

[0069] The scale normalization module 5 receives the set of regional maps from the previous module and performs scale scaling processing, and then passes the normalized sub-maps to the next module.

[0070] The target detection module 6 will process different sub-image regions to detect all the targets to be detected, and pass the detection results to the next module.

[0071] The result merging module 7 takes the detection results from different regions of the previous module and summarizes the results to generate the final detection results of the high-resolution image.

[0072] It should be noted that the steps in the method provided by the present invention can be implemented using the corresponding modules, devices, units, etc. in the system. Those skilled in the art can implement the steps of the method by referring to the technical solution of the system. That is, the embodiments in the system can be understood as preferred examples of implementing the method, and will not be elaborated here.

[0073] Those skilled in the art will understand that, in addition to implementing the system and its various devices provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices of this invention function as logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices provided by this invention can be considered as a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0074] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention. The above preferred features can be used in any combination without conflict.

Claims

1. A real-time target detection method in an ultrahigh-resolution image scene, characterized in that, include: S1 processes the original image to generate high-resolution and low-resolution images; S2, processing the low-resolution image to obtain the preserved region includes: S21, perform density estimation on the low-resolution image to generate multiple density maps at different scales, specifically: S211 sets multiple scales according to the size of the target being detected; S212 generates multiple density maps from the low-resolution image according to the multiple scales; The different scales include: microscale, small scale, medium scale, and large scale; each scale regresses targets of different sizes; during regression execution, the ranges of the above four scales are set according to the size of the target to be detected; based on the above four scales, four density maps are generated. S22, process each density map to obtain the retained region for each density map, specifically: Each density map is divided into grids, and the probability of an object existing in each grid is calculated to determine whether to retain that region; S3, the high-resolution image is cropped according to the preserved region to generate multiple region sub-images that require fine-grained detection; S4, Detect the region sub-image to obtain detection results, including: S41, Adjust multiple regional submaps, specifically: Use the smallest scale as the standard scale, scale the regional submaps at other scales to the same size as the standard scale, and keep all regional submaps the same size; S42, perform inspection on each adjusted region sub-map. S43, merge the detection results of each region sub-image. 2.The real-time target detection method in an ultra-high resolution image scene according to claim 1, wherein, The probability of an object's existence within each grid is calculated by integration. A global threshold is used to determine whether to retain the region. If the probability is greater than the global threshold, the region will be retained; if the probability is less than the global threshold, the region will be discarded. 3.The real-time target detection method in an ultra-high resolution image scene according to claim 2, characterized in that, Each of the segmented grids is larger than the size of the target at that scale; The larger the density map, the sparser the grid and the larger the area of ​​each grid region.

4. The real-time target detection method in an ultra-high resolution image scene according to claim 3, characterized in that, The merged detection results of each region sub-map include: The detection results of different sub-image regions are mapped back to the original image to obtain the detection results of the original image.

5. A real-time target detection system for ultra-high resolution image scenes, used to implement the method according to any one of claims 1-4, characterized in that, include: A resolution processing module that processes the original image to generate high-resolution and low-resolution images; A region determination module processes the low-resolution image to obtain a preserved region; The scaling module crops the high-resolution image based on the preserved region, generating multiple sub-images of regions requiring fine-grained detection. The target detection module detects the region sub-image and obtains the detection results.

Images