Open set target detection method and device for ultrahigh-resolution image and electronic equipment

Abstract

The invention provides an open set target detection method and device for an ultrahigh-resolution image and electronic equipment, and relates to the technical field of computers. The open set target detection method for the ultrahigh-resolution image comprises the following steps: acquiring the ultrahigh-resolution image; determining density information and scale information of a target in the ultrahigh-resolution image; segmenting the ultrahigh-resolution image into a plurality of adaptive image areas according to density information and scale information of a target in the ultrahigh-resolution image; recognizing a target contained in each self-adaptive image region by using a target recognition network to obtain a target detection result of each self-adaptive image region; and generating a target detection result of the ultrahigh-resolution image according to the target detection result of each self-adaptive image area. According to the method, various targets can be accurately and efficiently detected from the ultrahigh-resolution image in various complex and changeable practical application scenes.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an open set target detection method, device and electronic equipment for ultra-high resolution images. Background Art

[0002] With the development of image sensing technology, the acquisition of high-resolution images has become increasingly common in the fields of remote sensing imaging, medical imaging, security monitoring, etc. In particular, ultra-high-resolution images at the billion-pixel level provide great potential for detail capture and precise analysis.

[0003] However, existing object detection technologies face many challenges when processing ultra-high-resolution images at the gigapixel level. First, high computing resource requirements and long processing time are significant issues; second, when the resolution is reduced to reduce the computational burden, key details may be lost. These issues have seriously affected the efficient and accurate object detection of ultra-high-resolution images in the above-mentioned fields. In addition, traditional object detection algorithms are based on the closed set assumption, that is, the training and test data come from the same distribution. But in practical applications, the ultra-large field of view of gigapixel images often contains different scenes, which may contain a large number of unknown categories of objects that do not appear in the limited training data, making it difficult for existing algorithms to effectively identify these unknown objects.

[0004] Therefore, how to accurately and efficiently detect various types of targets from ultra-high-resolution images in various complex and changeable practical application scenarios is a technical problem that needs to be solved urgently. Summary of the invention

[0005] In view of the above problems existing in the prior art, the present invention provides an open set target detection method, device and electronic equipment for ultra-high resolution images, which are used to accurately and efficiently detect various types of targets from ultra-high resolution images.

[0006] The present invention provides an open set target detection method for ultra-high resolution images, comprising the following steps.

[0007] Acquire an ultra-high resolution image; acquire density information and scale information of a target in the ultra-high resolution image; divide the ultra-high resolution image into a plurality of adaptive image regions according to the density information and scale information of the target in the ultra-high resolution image; wherein each of the adaptive image regions contains at least one target; utilize a target recognition network to identify the target contained in each of the adaptive image regions, and obtain a target detection result for each of the adaptive image regions; wherein the target detection result includes location information and category information of the target contained in the adaptive image region; and generate a target detection result for the ultra-high resolution image according to the target detection result of each of the adaptive image regions.

[0008] According to an open set target detection method for ultra-high resolution images provided by the present invention, the density information and scale information of the target in the ultra-high resolution image are obtained, including: uniformly dividing the ultra-high resolution image according to multiple preset sizes to obtain multiple image sets; wherein each of the image sets includes image blocks obtained by cutting the ultra-high resolution image according to the same preset size; inputting each image block of each of the image sets into an unclassified target density estimation network to obtain a first unclassified target density estimation map of each image block; wherein the first unclassified target density estimation map includes the density information and scale information of the target contained in the corresponding image block; and inputting the first unclassified target density estimation map corresponding to all the image blocks contained in each of the image sets into an unclassified target density estimation network to obtain a first unclassified target density estimation map of each image block; wherein the first unclassified target density estimation map includes the density information and scale information of the target contained in the corresponding image block; and The method comprises the following steps of: stitching the second unclassified target density estimation maps to obtain a second unclassified target density estimation map corresponding to each of the image sets; wherein the second unclassified target density estimation map includes density information and scale information of the targets contained in the corresponding image set; fusing all the second unclassified target density estimation maps to obtain a third unclassified target density estimation map of the ultra-high resolution image; wherein the third unclassified target density estimation map includes density information and scale information of the targets contained in the ultra-high resolution image; and dividing the ultra-high resolution image into a plurality of adaptive image regions according to the density information and scale information of the targets in the ultra-high resolution image, comprising: dividing the ultra-high resolution image into a plurality of adaptive image regions according to the third unclassified target density estimation map.

[0009] According to an open set target detection method for an ultra-high resolution image provided by the present invention, the ultra-high resolution image is divided into multiple adaptive image regions according to the third classless target density estimation map, including: determining scale information and position information of multiple connected regions in the third classless target density estimation map according to density information of the third classless target density estimation map; and dividing the ultra-high resolution image into multiple adaptive image regions according to the scale information and position information of each of the connected regions.

[0010] According to an open set target detection method for ultra-high resolution images provided by the present invention, the target recognition network includes a region proposal network, a ROI detection head and a pre-trained CLIP model; the target recognition network is used to identify the targets contained in each of the adaptive image regions, and the target detection result of each of the adaptive image regions is obtained, including: for each of the adaptive image regions, the following operations are performed: using the region proposal network and the ROI detection head to obtain a region proposal for all targets contained in the adaptive image region; based on an image corresponding to the region proposal of the target contained in the adaptive image region, using the CLIP model to obtain the target detection result of the adaptive image region.

[0011] According to an open set target detection method for ultra-high resolution images provided by the present invention, the image corresponding to the region proposal of the target contained in the adaptive image region is used to obtain the target detection result of the adaptive image region by using the CLIP model, including: for the image corresponding to the region proposal of each target contained in the adaptive image region, performing the following operations: according to the image and a text representation set containing multiple potential categories, using the CLIP model, obtaining the similarity between the image and each text representation in the text representation set; based on the similarity between the image and each text representation in the text representation set, determining the category of the target corresponding to the image; and obtaining the target detection result of the adaptive image region according to the categories of all targets in the adaptive image region.

[0012] According to an open set target detection method for ultra-high resolution images provided by the present invention, the category of the target corresponding to the image is determined based on the similarity between the image and each text representation in the text representation set, including: performing the following operations for the similarity between the image and each text representation in the text representation set: determining, based on the similarity, a first probability that the target contained in the image belongs to a category identified by the text representation; utilizing a division operation to obtain a second probability that the target belongs to the category based on the first probability and the prior probability of the category; and determining the category of the target corresponding to the image based on the second probability that the target contained in the image belongs to a category identified by each text representation in the text representation set.

[0013] The present invention also provides an open set target detection device for ultra-high resolution images, comprising the following modules: A first acquisition module is used to acquire an ultra-high resolution image; a second acquisition module is used to acquire density information and scale information of the target in the ultra-high resolution image; a segmentation module is used to segment the ultra-high resolution image into a plurality of adaptive image regions according to the density information and scale information of the target in the ultra-high resolution image; wherein each of the adaptive image regions contains at least one target; an identification module is used to identify the targets contained in each of the adaptive image regions by using a target recognition network, and obtain a target detection result for each of the adaptive image regions; wherein the target detection result includes the location information and category information of the target contained in the adaptive image region; a generation module is used to generate a target detection result for the ultra-high resolution image according to the target detection result of each of the adaptive image regions.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, an open set target detection method for ultra-high resolution images as described in any one of the above-described methods is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described open-set target detection methods for ultra-high resolution images.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the open set target detection method for ultra-high resolution images as described in any one of the above-mentioned methods.

[0017] The open set target detection method, device and electronic device of ultra-high resolution images provided by the present invention divides the ultra-high resolution image into multiple adaptive image regions according to the density information and scale information of the target in the ultra-high resolution image, effectively reducing the complexity of single processing, and being able to more accurately identify and locate the target in complex and changeable practical application scenarios; using the target recognition network independently for each adaptive image region, local refined target detection is achieved, and the accuracy and processing efficiency of target detection are effectively improved; each adaptive image region can be processed in parallel, significantly shortening the overall detection time and improving the detection speed. Therefore, various types of targets can be accurately and efficiently detected from ultra-high resolution images in various complex and changeable practical application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a flow chart of the open set target detection method for ultra-high resolution images provided by the present invention.

[0020] Figure 2 It is a flow chart of a method for dividing an ultra-high resolution image into multiple adaptive image regions provided by the present invention.

[0021] Figure 3 It is a flowchart of a method for obtaining target detection results for each adaptive image region provided by the present invention.

[0022] Figure 4 It is a schematic diagram of the structure of the target recognition network provided by the present invention.

[0023] Figure 5 It is a structural schematic diagram of the region proposal network and ROI detection head provided by the present invention.

[0024] Figure 6 It is a schematic diagram of the third process of generating a classless target density estimation map provided by the present invention.

[0025] Figure 7 It is a structural schematic diagram of an open set target detection device for ultra-high resolution images provided by the present invention.

[0026] Figure 8 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] Combine the following Figure 1-Figure 6 The open set object detection method for ultra-high resolution images of the present invention is described.

[0029] Figure 1 is a flow chart of the open set target detection method for ultra-high resolution images provided by the present invention, such as Figure 1As shown, the method includes the following: Step 101: Acquire an ultra-high resolution image.

[0030] Ultra-high-resolution images refer to high-definition images with image resolution far exceeding general standards, such as billion-pixel images. Such images have a significant increase in the number of pixels and can present more delicate and clear image details.

[0031] In the specific implementation process, high-resolution image sensors can be used to collect image data of large scenes in real time. During the image acquisition process, ensure that the camera array covers the entire scene field of view, and use the high-resolution cameras in the camera array to splice and collect in real time to obtain ultra-high-resolution image data at the billion-pixel level. Subsequently, these collected ultra-high-resolution image data can be transmitted to the data processing system through a high-speed data transmission interface for subsequent target detection and recognition.

[0032] Step 102: Obtain density information and scale information of the target in the ultra-high resolution image.

[0033] In ultra-high resolution images, the target is the object or entity of interest or need to be identified in the image. Depending on the application scenario and requirements, the target can be any object with specific characteristics or attributes, such as people, vehicles, buildings, etc.

[0034] The density information of the target is the density or distribution of the target in the image. The density information describes the number of targets in the image and their relative position relationship. For example, in a crowded scene, the target density information will be very high, indicating that there are many targets (people) gathered together.

[0035] The scale information of an object is the size or dimension of the object in the image. The scale information describes the number of pixels or actual physical size of the object in the image.

[0036] In the specific implementation process, the density information and scale information of the target in the ultra-high resolution image can be obtained in a variety of ways, which are not limited to the description of this specification.

[0037] For an example of obtaining density information and scale information of a target in an ultra-high resolution image, see Figure 2 The relevant content in will not be repeated here.

[0038] Step 103: divide the super-high resolution image into a plurality of adaptive image regions according to the density information and scale information of the target in the super-high resolution image.

[0039] In the specific implementation process, the ultra-high resolution image can be divided into multiple adaptive image regions in a variety of ways according to actual needs and based on the density information and scale information of the target in the ultra-high resolution image. Each adaptive image region contains at least one target to ensure the pertinence and effectiveness of subsequent processing. For example, for an ultra-high resolution image containing a dense crowd, since the scale and density of people in the image are different, the image can be divided into multiple adaptive image regions based on these density and scale information, each of which is surrounded by one or a group of dense people. In the subsequent target detection task, these adaptive image regions can be processed more specifically, thereby improving the processing efficiency and accuracy of target detection.

[0040] For an embodiment of dividing a super-high resolution image into a plurality of adaptive image regions according to density information and scale information of an object in the super-high resolution image, see Figure 2 The relevant content in will not be repeated here.

[0041] Step 104: using the target recognition network to identify the target contained in each adaptive image region, and obtaining the target detection result of each adaptive image region.

[0042] The object detection result includes the location information and category information of the object contained in the adaptive image region.

[0043] The target recognition network is used to learn and extract features from images, and then classify and locate targets in the images. The target recognition network can be a machine learning model or a combination of multiple machine learning models. For example, the target recognition network can be built based on a convolutional neural network (CNN), which is not limited to the description of this specification.

[0044] For an embodiment of a target recognition network and using the target recognition network to recognize targets contained in each adaptive image region and obtain target detection results for each adaptive image region, see Figure 3 The relevant content in will not be repeated here.

[0045] Step 105: Generate an object detection result of an ultra-high resolution image according to the object detection result of each adaptive image region.

[0046] In the specific implementation process, the target detection results (e.g., detection boxes) of all adaptive image regions can be collected into a set, and these detection results can be sorted from large to small according to the confidence level; the detection boxes with low confidence levels are filtered out by a preset IOU (Intersection over Union) threshold, and then non-maximum suppression is applied to the filtered detection results to eliminate duplicate detection targets and ensure the accuracy of the detection results. Finally, the detection results after non-maximum suppression are merged to generate the overall target detection results of the ultra-high resolution image.

[0047] During the specific implementation process, the merged detection results can be annotated on the ultra-high resolution image, including the location information and category information of the target; the detected targets can be displayed in an intuitive manner through the user interface to facilitate user viewing and analysis, and the detected targets can also be highlighted to enhance the visual effect.

[0048] In a specific implementation process, steps 101 to 105 may be repeatedly executed multiple times to cyclically perform target detection on the ultra-high resolution image collected in real time and display the detection results.

[0049] Figure 2 is a flow chart of a method for dividing an ultra-high resolution image into a plurality of adaptive image regions provided by the present invention, such as Figure 2 As shown, the method includes the following: Step 201 : Evenly segment the ultra-high resolution image according to a plurality of preset sizes to obtain a plurality of image sets; wherein each image set includes image blocks obtained by segmenting the ultra-high resolution image according to the same preset size.

[0050] In the specific implementation process, Figure 6 As shown, the ultra-high resolution image can be evenly divided according to a variety of preset sizes, such as 1×1, 2×2, 3×3, etc., to obtain 1×1 image blocks. , 2×2 image blocks , 3×3 image blocks Through such segmentation, multiple image sets can be obtained, each of which contains image blocks obtained by cutting the ultra-high resolution image according to the same preset size. Small-scale image blocks help to better perceive and recognize small-sized objects.

[0051] In order to facilitate subsequent processing, each image block obtained by segmentation can be scaled to a uniform default size.

[0052] Step 202: Input each image block of each image set into the classless target density estimation network to obtain a first classless target density estimation map for each image block; wherein the first classless target density estimation map includes density information and scale information of the target contained in the corresponding image block.

[0053] The unclassified target density estimation network is a deep learning network whose main task is to estimate the density and scale of targets in an image, but it does not need to distinguish the specific categories of targets. In the specific implementation process, the unclassified target density estimation network can be constructed based on deep learning networks such as convolutional neural networks (CNN) or Transformer. Through the learning of a large amount of training data, it can accurately predict the distribution density and size information of targets in the image, providing an important reference for subsequent target detection, tracking and other tasks.

[0054] In the specific implementation process, each image set can be Each image block in is input into the unclassified target density estimation network respectively; the unclassified target density estimation network outputs the first unclassified target density estimation map corresponding to each image block. The first unclassified target density estimation map contains the density information and scale information of the target contained in its corresponding image block.

[0055] Step 203: splice the first unclassified target density estimation maps corresponding to all image blocks contained in each image set to obtain a second unclassified target density estimation map corresponding to each image set; wherein the second unclassified target density estimation map includes density information and scale information of the targets contained in the corresponding image set.

[0056] In the specific implementation process, for each image set, the first unclassified target density estimation maps corresponding to all image blocks therein can be spliced ​​to integrate the density estimation maps of the scattered image blocks to form a second unclassified target density estimation map corresponding to each image set. The second unclassified target density estimation map contains the density information and scale information of the targets contained in the corresponding image set. For ease of processing, before splicing the first unclassified target density estimation maps, the first unclassified target density estimation maps of different size levels need to be scaled to ensure that they can be spliced ​​in the correct proportion and position.

[0057] Step 204: fuse all the second unclassified target density estimation maps to obtain a third unclassified target density estimation map of the ultra-high resolution image; wherein the third unclassified target density estimation map includes density information and scale information of the targets contained in the ultra-high resolution image.

[0058] As an example only, all the second unclassified target density estimation maps can be fused in the following way to obtain Figure 6 The third unclassified object density estimate map of the super-resolution image shown.

[0059] Since different second unclassified target density estimation maps may have different density ranges, each second unclassified target density estimation map needs to be normalized and scaled to the same numerical range to ensure consistent scale during fusion.

[0060] Each second unclassified target density estimation map is resampled so that they have the same resolution, and the consistency of the spatial positions of all second unclassified target density estimation maps is ensured through alignment processing.

[0061] Considering that the importance of different second unclassified target density estimation maps may be different, different weights can be assigned to each second unclassified target density estimation map, and these weights are used to perform weighted fusion on all second unclassified target density estimation maps.

[0062] Step 205: divide the ultra-high resolution image into a plurality of adaptive image regions according to the third unclassified object density estimation map.

[0063] In some embodiments, the scale information and position information of multiple connected regions in the third unclassified target density estimation map can be determined based on the density information of the third unclassified target density estimation map; and the ultra-high resolution image can be divided into multiple adaptive image regions based on the scale information and position information of each connected region.

[0064] As an example only, the super-high resolution image may be segmented into a plurality of adaptive image regions in the following manner.

[0065] For the third unclassified target density estimation map, a threshold value τ can be used for high-pass filtering to remove the influence of noise in the density estimation map; the third unclassified target density estimation map is connected by marking the connected domains, and the areas of all connected domains are calculated to obtain the scale information and position information of different targets. In a binary image, if two pixels are adjacent (that is, their positions are adjacent and the pixel values ​​are the same), then the two pixels are considered to be connected. The purpose of the connected domain labeling algorithm is to assign a unique identifier (i.e., label) to each connected domain in the image. Common algorithms include itinerary-based algorithms, depth-first search (DFS), breadth-first search (BFS), etc.

[0066] Using Thresholds and The connected domain areas are classified into large targets, medium targets and small targets.

[0067] In the target detection task, the size and shape of the target are often diverse. In order to achieve multi-scale target detection on the same image, detection windows can be set for targets of three different sizes: large, medium, and small. Each detection window can be set to detect a maximum of m objects. In order to quickly group the detected targets of different sizes, a kd tree structure can be used. Finally, for targets of different groups, an inscribed fixed shape rectangle is used on the third unclassified target density estimation map for adaptive segmentation, thereby obtaining multiple adaptive image regions. .

[0068] Figure 3 It is a flowchart of a method for obtaining target detection results for each adaptive image region provided by the present invention.

[0069] In this embodiment, if Figure 4 As shown in Figure 1, the target recognition network includes a region proposal network, a ROI detection head, and a pre-trained CLIP model. The region proposal network and the ROI detection head (ROI classification head) are shown in Figure 1. Figure 5 shown.

[0070] like Figure 3 As shown, for each adaptive image area, perform the following operations: Step 301: Use the region proposal network and the ROI detection head to obtain region proposals for all objects contained in the adaptive image region.

[0071] The Region Proposal Network (RPN) can be implemented based on a convolutional neural network to generate candidate regions that may contain objects in an image, namely, region proposals.

[0072] ROI detection head (Region of Interest Head) is a neural network module that further classifies and regresses the candidate regions generated by the region proposal network in the object detection framework. The ROI detection head usually consists of two parts: a classification subnetwork and a regression subnetwork.

[0073] In the embodiment provided by the present application, in order to detect a set of targets in an open world, a localization network consisting of a region proposal network and a ROI detection head that is independent of classification is used to generate a proposal region frame of a common object in an adaptive image region. In the specific implementation process, the multiple adaptive image regions obtained by segmentation in the previous step can be Input into the positioning network to obtain the location information and region proposals of all targets contained in the adaptive image region.

[0074] Step 302: Based on the image corresponding to the region proposal of the target contained in the adaptive image region, the CLIP model is used to obtain the target detection result of the adaptive image region.

[0075] In a specific implementation process, the following operations may be performed for the image corresponding to the region proposal of each target contained in the adaptive image region: According to an image and a set of text representations containing multiple potential categories, the CLIP model is used to obtain the similarity between the image and each text representation in the text representation set; based on the similarity between the image and each text representation in the text representation set, the category of the target corresponding to the image is determined.

[0076] The CLIP model is a multimodal vision and language model that performs tasks by learning the correspondence between images and the text associated with them. The CLIP model consists of two parts: an image encoder and a text encoder.

[0077] In a specific implementation process, an image can be input into the image encoder of the CLIP model to obtain a feature representation of the image. At the same time, each text representation in the text representation set is input into the text encoder of CLIP to obtain a feature representation of each text.

[0078] In a specific implementation process, cosine similarity or other similarity measurement methods can be used to calculate the similarity between the feature representation of the image and the feature representation of each text, thereby obtaining a similarity set, each similarity representing the degree of matching between the image and the corresponding text representation.

[0079] In some embodiments, the category represented by the text corresponding to the maximum similarity may be used as the category of the target corresponding to the image.

[0080] In some embodiments, considering a large number of new categories, in order to avoid the CLIP model from outputting old categories, the category of the object corresponding to the image can be determined based on the similarity between the image and each text representation in the text representation set in the following manner: For the similarity between the image and each text representation in the text representation set, the following operations are performed: based on the similarity, a first probability that the target contained in the image belongs to the category identified by the text representation is determined; using a division operation, a second probability that the target belongs to the category is obtained based on the first probability and the prior probability of the category; based on the second probability that the target contained in the image belongs to the category identified by each text representation in the text representation set, the category of the target corresponding to the image is determined. The calculation formula of the second probability is expressed as follows.

[0081] (1)

[0082] in, Representing images i Belong to text j a second probability of the identified target class; For text j The prior probability of the identified target class, The larger it is, the more the model prefers this category; are predefined hyperparameters; Representing images i Belong to text j The first probability of the identified target class; Represents text j The language embedding vector of is a hyperparameter factor used to balance the probability distribution of the output.

[0083] Step 303: Obtain the target detection result of the adaptive image region according to the categories of all targets in the adaptive image region.

[0084] In a specific implementation process, the categories of all targets in the adaptive image region may be summarized to obtain the target detection result of the adaptive image region.

[0085] In the embodiment provided by the present invention, a region proposal network and a ROI detection head are used to obtain region proposals of all objects contained in the adaptive image region; based on the image corresponding to the region proposal of the object contained in the adaptive image region, the CLIP model is used to obtain the object detection result of the adaptive image region. Thus, not only the predefined known object categories can be identified, but also the unknown object categories appearing in the image can be detected and identified.

[0086] The open set target detection device for ultra-high resolution images provided by the present invention is described below. The open set target detection device for ultra-high resolution images described below and the open set target detection method for ultra-high resolution images described above can be referenced to each other.

[0087] Figure 7 It is a structural schematic diagram of an open set target detection device for ultra-high resolution images provided by the present invention.

[0088] like Figure 7 As shown, the device 700 includes the following modules.

[0089] The first acquisition module 710 is used to acquire an ultra-high resolution image.

[0090] The second acquisition module 720 is used to acquire density information and scale information of the target in the ultra-high resolution image.

[0091] The segmentation module 730 is used to segment the super-high resolution image into a plurality of adaptive image regions according to density information and scale information of the target in the super-high resolution image; wherein each of the adaptive image regions contains at least one target.

[0092] The recognition module 740 is used to use the target recognition network to identify the targets contained in each of the adaptive image regions, and obtain the target detection results of each of the adaptive image regions; wherein the target detection results include the location information and category information of the targets contained in the adaptive image regions.

[0093] The generating module 750 is used to generate the target detection result of the ultra-high resolution image according to the target detection result of each of the adaptive image regions.

[0094] Figure 8 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 8 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the open set target detection method of the ultra-high resolution image, the method comprising: obtaining an ultra-high resolution image; obtaining the density information and scale information of the target in the ultra-high resolution image; dividing the ultra-high resolution image into a plurality of adaptive image regions according to the density information and scale information of the target in the ultra-high resolution image; wherein each of the adaptive image regions contains at least one target; using a target recognition network, identifying the target contained in each of the adaptive image regions, and obtaining the target detection result of each of the adaptive image regions; wherein the target detection result includes the location information and category information of the target contained in the adaptive image region; and generating the target detection result of the ultra-high resolution image according to the target detection result of each of the adaptive image regions.

[0095] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0096] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the open set target detection method for ultra-high resolution images provided by the above methods. The method includes: obtaining an ultra-high resolution image; obtaining density information and scale information of the target in the ultra-high resolution image; according to the density information and scale information of the target in the ultra-high resolution image, dividing the ultra-high resolution image into multiple adaptive image regions; wherein each of the adaptive image regions contains at least one target; using a target recognition network, identifying the target contained in each of the adaptive image regions, and obtaining a target detection result for each of the adaptive image regions; wherein the target detection result includes the location information and category information of the target contained in the adaptive image region; and generating a target detection result for the ultra-high resolution image according to the target detection result of each of the adaptive image regions.

[0097] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the open set target detection method for ultra-high resolution images provided by the above-mentioned methods, the method comprising: obtaining an ultra-high resolution image; obtaining density information and scale information of the target in the ultra-high resolution image; dividing the ultra-high resolution image into a plurality of adaptive image regions according to the density information and scale information of the target in the ultra-high resolution image; wherein each of the adaptive image regions contains at least one target; using a target recognition network, identifying the targets contained in each of the adaptive image regions, and obtaining target detection results for each of the adaptive image regions; wherein the target detection results include location information and category information of the targets contained in the adaptive image regions; and generating target detection results for the ultra-high resolution image according to the target detection results of each of the adaptive image regions.

[0098] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0099] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An open set object detection method for ultra-high resolution images, characterized in that: include: Acquire ultra-high resolution images; Obtaining density information and scale information of the target in the ultra-high resolution image; According to the density information and scale information of the target in the super-high resolution image, the super-high resolution image is divided into a plurality of adaptive image regions; wherein each of the adaptive image regions contains at least one target; Using a target recognition network, identifying targets contained in each of the adaptive image regions, and obtaining target detection results for each of the adaptive image regions; wherein the target detection results include location information and category information of the targets contained in the adaptive image regions; The object detection result of the ultra-high resolution image is generated according to the object detection result of each of the adaptive image regions.

2. The open set object detection method for ultra-high resolution images according to claim 1, characterized in that: The obtaining of density information and scale information of the target in the ultra-high resolution image includes: According to a plurality of preset sizes, the ultra-high resolution image is evenly divided respectively to obtain a plurality of image sets; wherein each of the image sets includes image blocks obtained by cutting the ultra-high resolution image according to the same preset size; Inputting each image block of each image set into the unclassified target density estimation network respectively to obtain a first unclassified target density estimation map of each image block; wherein the first unclassified target density estimation map includes density information and scale information of the target contained in the corresponding image block; splicing the first unclassified target density estimation maps corresponding to all the image blocks contained in each of the image sets to obtain a second unclassified target density estimation map corresponding to each of the image sets; wherein the second unclassified target density estimation map includes density information and scale information of the targets contained in the corresponding image set; Fusing all the second unclassified target density estimation maps to obtain a third unclassified target density estimation map of the ultra-high resolution image; wherein the third unclassified target density estimation map includes density information and scale information of the targets contained in the ultra-high resolution image; The step of dividing the super-high resolution image into a plurality of adaptive image regions according to the density information and scale information of the target in the super-high resolution image comprises: According to the third unclassified object density estimation map, the ultra-high resolution image is divided into a plurality of adaptive image regions.

3. The open set object detection method for ultra-high resolution images according to claim 2, characterized in that: The step of dividing the ultra-high resolution image into a plurality of adaptive image regions according to the third unclassified target density estimation map comprises: Determining scale information and position information of a plurality of connected regions in the third unclassified target density estimation map according to density information of the third unclassified target density estimation map; According to the scale information and position information of each of the connected regions, the super-high resolution image is divided into a plurality of adaptive image regions.

4. The open set object detection method for ultra-high resolution images according to claim 1, characterized in that: The target recognition network includes a region proposal network, a ROI detection head and a pre-trained CLIP model; The using of the target recognition network to identify the target contained in each of the adaptive image regions and obtaining the target detection result of each of the adaptive image regions includes: For each of the adaptive image regions, perform the following operations: Using the region proposal network and the ROI detection head, obtaining region proposals for all objects contained in the adaptive image region; Based on an image corresponding to a region proposal of the target contained in the adaptive image region, the CLIP model is used to obtain a target detection result of the adaptive image region.

5. The open set object detection method for ultra-high resolution images according to claim 4, characterized in that: The region proposal corresponding to the target contained in the adaptive image region, and obtaining the target detection result of the adaptive image region by using the CLIP model, comprises: For the image corresponding to the region proposal of each target contained in the adaptive image region, perform the following operations: According to the image and a set of text representations including a plurality of potential categories, using a CLIP model, obtaining a similarity between the image and each text representation in the set of text representations; Determining a category of an object corresponding to the image based on a similarity between the image and each text representation in the text representation set; According to the categories of all objects in the adaptive image area, an object detection result of the adaptive image area is obtained.

6. The open set object detection method for ultra-high resolution images according to claim 5, characterized in that: The determining, based on the similarity between the image and each text representation in the text representation set, a category of the target corresponding to the image, comprises: For the similarity between the image and each text representation in the text representation set, perform the following operations: Determining, based on the similarity, a first probability that an object contained in the image belongs to a category identified by the text representation; Obtaining, by a division operation, a second probability that the target belongs to the category according to the first probability and the prior probability of the category; The category of the object corresponding to the image is determined according to a second probability that the object contained in the image belongs to the category identified by each text representation in the text representation set.

7. An open set target detection device for ultra-high resolution images, characterized in that: include: A first acquisition module, used for acquiring ultra-high resolution images; A second acquisition module is used to acquire density information and scale information of the target in the ultra-high resolution image; A segmentation module, configured to segment the ultra-high resolution image into a plurality of adaptive image regions according to density information and scale information of the target in the ultra-high resolution image; wherein each of the adaptive image regions contains at least one target; A recognition module, used to use a target recognition network to recognize the target contained in each of the adaptive image regions, and obtain a target detection result for each of the adaptive image regions; wherein the target detection result includes location information and category information of the target contained in the adaptive image region; A generation module is used to generate the target detection result of the ultra-high resolution image according to the target detection result of each of the adaptive image regions.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the open set target detection method for ultra-high resolution images as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the open set target detection method for ultra-high resolution images as claimed in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the open set target detection method for ultra-high resolution images as claimed in any one of claims 1 to 6 is implemented.

Images