Target detection method, device, equipment and storage medium

Abstract

The present application discloses a target detection method, device, equipment and storage medium, wherein the method comprises: acquiring a target image, the target image comprises at least one target object; performing feature extraction processing according to the target image to obtain a first image feature; performing density estimation processing according to the first image feature to obtain a density map; determining the target confidence of at least two candidate frames according to the density map and the first image feature; determining at least one target candidate frame according to the target confidence of at least two candidate frames; performing target detection processing on at least one target object according to at least one target candidate frame to obtain at least one target detection result. In this way, the target confidence difference between at least two candidate frames can be increased, so that the ranking between at least two candidate frames is clearer, so that it is easier to select a better candidate frame from at least two candidate frames to detect the target image, thereby improving the accuracy of target detection.

Description

Technical Field

[0001] The present application relates to, but is not limited to, the field of computer vision technology, and in particular to a target detection method, device, equipment and storage medium. Background Art

[0002] Object detection is one of the basic tasks in computer vision and can be applied to scenarios such as smart retail. Usually, a better candidate box can be selected from at least two candidate boxes to detect the target image. However, since the confidence between at least two candidate boxes is relatively close, the ranking between at least two candidate boxes is relatively fuzzy, making it difficult to select a better candidate box, reducing the accuracy of object detection. Summary of the invention

[0003] In view of this, the present application provides at least one target detection method, apparatus, device and storage medium.

[0004] The technical solution of this application is implemented as follows:

[0005] On the one hand, the present application provides a target detection method, the method comprising:

[0006] Acquire a target image, where the target image includes at least one target object;

[0007] Performing feature extraction processing on the target image to obtain a first image feature;

[0008] Performing density estimation processing according to the first image feature to obtain a density map;

[0009] Determine, according to the density map and the first image feature, target confidences of at least two candidate boxes respectively; determine, according to the target confidences of at least two candidate boxes respectively, at least one target candidate box;

[0010] According to at least one target candidate frame, target detection processing is performed on at least one target object to obtain at least one target detection result.

[0011] In the above embodiment, based on the density map and the first image feature, the target confidence of each of at least two candidate boxes is determined, which can increase the target confidence difference between the at least two candidate boxes, making the sorting between the at least two candidate boxes clearer, thereby making it easier to select a better candidate box (i.e., at least one target candidate box) from the at least two candidate boxes to detect the target image, thereby improving the accuracy of target detection.

[0012] In some embodiments, determining the target confidence of each of at least two candidate boxes based on the density map and the first image feature includes: performing classification processing based on the first image feature to obtain the initial confidence of each of the at least two candidate boxes; performing accumulation processing based on the density map and the initial confidence of each of the at least two candidate boxes to obtain the target confidence of each of the at least two candidate boxes.

[0013] In the above embodiment, accumulation processing is performed based on the density map and the initial confidence of each of the at least two candidate boxes, so that the at least two candidate boxes can be reordered once, and based on the obtained target confidence of each of the at least two candidate boxes, a better candidate box (i.e., at least one target candidate box) can be easily selected from the at least two candidate boxes to detect the target image, thereby improving the accuracy of target detection.

[0014] In some embodiments, determining the target confidence of each of at least two candidate boxes based on the density map and the first image feature includes: performing a product processing based on the density map and the first image feature to obtain a second image feature; performing an accumulation processing based on the first image feature and the second image feature to obtain a third image feature; and determining the target confidence of each of the at least two candidate boxes based on the third image feature.

[0015] In the above embodiment, the target confidence of each of the at least two candidate boxes can be determined according to the third image feature, so that based on the target confidence of each of the at least two candidate boxes, it is easier to select a better candidate box (i.e., at least one target candidate box) from the at least two candidate boxes to detect the target image, thereby improving the accuracy of target detection.

[0016] In some embodiments, determining the target confidence of each of the at least two candidate boxes based on the third image feature includes: performing classification processing based on the third image feature to obtain the target confidence of each of the at least two candidate boxes.

[0017] In the above embodiment, since the extraction accuracy of the third image feature is high, the accuracy of the target confidence of each of the at least two candidate frames obtained by the classification processing of the third image feature is also high. In this way, based on the target confidence of each of the at least two candidate frames obtained, it is easier to select a better candidate frame (i.e., at least one target candidate frame) from the at least two candidate frames to detect the target image, thereby improving the accuracy of target detection.

[0018] In some embodiments, determining the target confidence of each of the at least two candidate boxes based on the third image feature includes: performing classification processing based on the third image feature to obtain the initial confidence of each of the at least two candidate boxes; performing accumulation processing based on the density map and the initial confidence of each of the at least two candidate boxes to obtain the target confidence of each of the at least two candidate boxes.

[0019] In the above embodiment, accumulation processing is performed based on the density map and the initial confidence of each of the at least two candidate boxes, so that the at least two candidate boxes can be reordered once, and based on the obtained target confidence of each of the at least two candidate boxes, a better candidate box (i.e., at least one target candidate box) can be easily selected from the at least two candidate boxes to detect the target image, thereby improving the accuracy of target detection.

[0020] In some embodiments, the method also includes: selecting the first N target confidences from the target confidences of the at least two candidate boxes, the candidate boxes corresponding to the first N target confidences are N candidate boxes among the at least two candidate boxes; determining the positions of the at least two candidate boxes in the density map according to the first image feature; determining the positions of the N candidate boxes in the density map according to the positions of the at least two candidate boxes in the density map; determining the updated target confidences of the N candidate boxes according to the target confidences of the N candidate boxes, the positions of the N candidate boxes in the density map and the density map; wherein N is an integer greater than 1.

[0021] In the above embodiment, the target confidence of each of the N candidate boxes can be updated based on the target confidence of each of the N candidate boxes, the position of each of the N candidate boxes in the density map and the density map, so that the N candidate boxes can be reordered once, and the difference between the target confidences of each of the N candidate boxes can be increased. In the subsequent post-processing process (such as NMS), the better candidate box can be retained, thereby improving the accuracy of target detection.

[0022] In some embodiments, determining the updated target confidence of each of the N candidate boxes includes: performing feature extraction processing on the density map according to the positions of each of the N candidate boxes in the density map to obtain a sub-density map corresponding to each of the N candidate boxes; performing similarity comparison on the sub-density map corresponding to each of the N candidate boxes with a preset sub-density map to obtain the similarity corresponding to each of the N candidate boxes; performing weighted processing on the similarity corresponding to each of the N candidate boxes and the target confidence of each of the N candidate boxes to determine the updated target confidence of each of the N candidate boxes.

[0023] In the above embodiment, the similarities corresponding to each of the N candidate boxes and the target confidences of each of the N candidate boxes are weighted, and the target confidences of each of the N candidate boxes can be updated, so that the N candidate boxes can be reordered, and the difference between the target confidences of each of the N candidate boxes can be increased. In the subsequent post-processing process (such as NMS), the better candidate boxes can be retained, thereby improving the accuracy of target detection.

[0024] In some embodiments, determining the positions of each of the at least two candidate boxes in the density map based on the first image feature includes: performing regression processing based on the first image feature to obtain the positions of each of the at least two candidate boxes in the density map.

[0025] In the above embodiment, directly performing regression processing based on the first image feature can improve the efficiency of obtaining the positions of at least two candidate boxes in the density map.

[0026] In some embodiments, determining the positions of each of the at least two candidate boxes in the density map based on the first image feature includes: performing a product processing based on the density map and the first image feature to obtain a second image feature; performing an accumulation processing based on the first image feature and the second image feature to obtain a third image feature; and performing a regression processing based on the third image feature to obtain the positions of each of the at least two candidate boxes in the density map.

[0027] In the above embodiment, since the extraction accuracy of the third image feature is relatively high, the positions of the at least two candidate boxes in the density map can be made more accurate by performing regression processing on the third image feature.

[0028] In some embodiments, performing density estimation processing according to the first image feature to obtain a density map includes: performing feature extraction processing according to the target image to obtain a fourth image feature, wherein the fourth image feature has a different scale from the first image feature; performing density estimation processing according to the first image feature to obtain a first density feature; performing density estimation processing according to the fourth image feature to obtain a second density feature; and performing fusion processing according to the first density feature and the second density feature to obtain the density map.

[0029] In the above embodiment, compared with obtaining the density map through the first image feature, when the density map is obtained through the first image feature and the fourth image feature, the accuracy of the density map can be higher, so that the accuracy of the target confidence of at least two candidate boxes obtained subsequently based on the density map is higher.

[0030] On the other hand, the present application provides a target detection device, the device comprising:

[0031] An acquisition module, used for acquiring a target image, wherein the target image includes at least one target object;

[0032] A processing module, configured to perform feature extraction processing according to the target image to obtain a first image feature; and perform density estimation processing according to the first image feature to obtain a density map;

[0033] A determination module, configured to determine target confidences of at least two candidate boxes according to the density map and the first image feature; and determine at least one target candidate box according to the target confidences of the at least two candidate boxes;

[0034] The processing module is further used to perform target detection processing on the at least one target object according to the at least one target candidate frame to obtain at least one target detection result.

[0035] In the above embodiment, based on the density map and the first image feature, the target confidence of each of at least two candidate boxes is determined, which can increase the target confidence difference between the at least two candidate boxes, making the sorting between the at least two candidate boxes clearer, thereby making it easier to select a better candidate box (i.e., at least one target candidate box) from the at least two candidate boxes to detect the target image, thereby improving the accuracy of target detection.

[0036] On the other hand, the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor implements part or all of the steps in the above method when executing the program.

[0037] On the other hand, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements part or all of the steps in the above method when executed by a processor.

[0038] On the other hand, the present application provides a computer program, comprising computer-readable codes. When the computer-readable codes are executed in a computer device, a processor in the computer device executes some or all of the steps for implementing the above method.

[0039] On the other hand, the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, it implements part or all of the steps in the above method.

[0040] In the present application, a target image can be obtained; feature extraction processing is performed on the target image to obtain a first image feature; density estimation processing is performed on the first image feature to obtain a density map; target confidences of at least two candidate frames are determined based on the density map and the first image feature; wherein at least two candidate frames are used to perform target detection on the target image. In this way, the target confidence difference between at least two candidate frames can be increased, so that the ranking between at least two candidate frames is clearer, thereby making it easier to select a better candidate frame (i.e., at least one target candidate frame) from at least two candidate frames to detect the target image, thereby improving the accuracy of target detection.

[0041] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0043] Figure 1 It is a flowchart of a target detection method provided in an embodiment of the present application;

[0044] Figure 2 It is a schematic diagram of a framework for determining a density map provided in an embodiment of the present application;

[0045] Figure 3 This is a detailed process diagram of a target detection method provided in an embodiment of the present application. Figure 1 ;

[0046] Figure 4 This is a detailed process diagram of a target detection method provided in an embodiment of the present application. Figure 2 ;

[0047] Figure 5 This is a detailed process diagram of a target detection method provided in an embodiment of the present application. Figure 3 ;

[0048] Figure 6 This is a detailed process diagram of a target detection method provided in an embodiment of the present application. Figure 4 ;

[0049] Figure 7 It is a schematic diagram of a framework for determining target confidence of a candidate box provided in an embodiment of the present application;

[0050] Figure 8 It is a schematic diagram of the composition structure of a target detection device provided in an embodiment of the present application;

[0051] Fig. 9It is a hardware entity schematic diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application are further elaborated in detail below in conjunction with the drawings and embodiments. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0053] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0054] In the embodiments of the present application, the terms "first / second / third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first / second / third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0055] It should be understood that the term "at least one" may refer to one or more (two or more), for example, at least one target object may refer to one or two or more targets; the term "at least two" may refer to two or more, for example, at least two candidate boxes may refer to two or more candidate boxes.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing this application and are not intended to limit this application.

[0057] Before further describing the embodiments of the present application in detail, the nouns involved in the embodiments of the present application are first described. The nouns involved in the embodiments of the present application are subject to the following explanations.

[0058] Candidate box: The candidate box can detect the target object (such as goods) in the target image. According to the positional relationship between the candidate box and the target object, the candidate box can include positive sample candidate boxes and negative sample candidate boxes, where:

[0059] If there is a target object at the position corresponding to the candidate frame and the candidate frame is relatively close to the target object, the candidate frame is a positive sample candidate frame;

[0060] When there is no target object at the position corresponding to the candidate box, the candidate box is a negative sample candidate box;

[0061] When there is a target object at the position corresponding to the candidate frame and the candidate frame is not closely aligned with the target object, the candidate frame is a negative sample candidate frame.

[0062] It should be noted that the closer the candidate box fits the target object, the higher the confidence of the candidate box, and the more accurate the candidate box is in detecting the target object; the less closely the candidate box fits the target object, the lower the confidence of the candidate box, and the less accurate the candidate box is in detecting the target object.

[0063] In order to better understand the embodiments of the present application, the target detection scheme in the related art is first described.

[0064] Object detection is one of the basic tasks in computer vision and can be applied to scenarios such as smart retail. For example, with the development of smart retail, object detection can be applied to shelf detection to develop unmanned vending scenarios and inventory management systems. For example, through object detection, the goods taken by users from unmanned vending cabinets and smart refrigerators can be detected.

[0065] In the related art, a better candidate frame can usually be selected from at least two candidate frames to detect the target image. However, since the confidence between at least two candidate frames is relatively close, the ranking between at least two candidate frames is relatively fuzzy, making it difficult to select a better candidate frame, reducing the accuracy of target detection. Taking the smart retail scenario as an example, since the shelves (such as shelves in supermarkets and shelves in warehouses) are densely packed and overlapped, most of the goods are small and have rich textures, resulting in many goods being missed.

[0066] For example, when the confidences of the positive sample candidate box and the negative sample candidate box are relatively close, the ordering between the positive sample candidate box and the negative sample candidate box is relatively vague. In some cases, the confidence of the negative sample candidate box may be higher than that of the positive sample candidate box, making the positive sample candidate box more difficult to select.

[0067] The present application embodiment provides a target detection method, which can be executed by a processor of a computer device. The computer device may refer to a server, a laptop, a tablet computer, a desktop computer, a smart TV, a set-top box, a mobile device (such as a mobile phone, a portable video player, a personal digital assistant, a dedicated messaging device, a portable gaming device), or other device with target detection capabilities. Figure 1 is a flow chart of a target detection method provided in an embodiment of the present application, such as Figure 1 As shown, the method may include the following steps S101 to S104:

[0068] S110: Acquire a target image, where the target image includes at least one target object.

[0069] It should be noted that the target image can be obtained by taking a photo, downloaded from the Internet, scanned by a scanner, or obtained by other means, which is not limited in the embodiments of the present application.

[0070] It should also be noted that the target image may contain one target object or at least two target objects, which is not limited in the embodiments of the present application.

[0071] For example, in the detection of "cats" and "dogs", if a target image includes both cats and dogs, and there are 3 cats and 2 dogs, then the target objects in the target image can be divided into two categories: cats and dogs, and there are 5 target objects in the target image.

[0072] For example, in shelf detection, if a target image includes 35 commodities, there are 35 target objects in the target image.

[0073] S120: Perform feature extraction processing on the target image to obtain a first image feature.

[0074] It should be noted that the target image can be passed through a convolutional layer in a neural network to extract features in the target image (such as the first image features).

[0075] Furthermore, when the neural network is a shallow neural network, the first image feature is a shallower feature, and the scale of the first image feature is coarser; when the neural network is a deep neural network, the first image feature is a deeper feature, and the deeper feature may be a more semantic and abstract feature, and the scale of the first image feature is finer.

[0076] S130 , performing density estimation processing according to the first image feature to obtain a density map.

[0077] Exemplarily, the density map may be a Gaussian density map.

[0078] It should be noted that each point in the density map is a number between 0 and 1. The closer to 1, the closer to the center of the target object, and the closer to 0, the farther from the center of the target object. Therefore, the location of the target object can be known through the density map. Furthermore, the density map can be used to strengthen the network's learning of the center of the target object.

[0079] It should also be noted that, when feature extraction processing is performed on the target image to obtain at least two image features (such as the first image feature), density estimation processing can be performed on the at least two image features respectively to obtain a density map.

[0080] The following takes the example that at least two image features include a first image feature and a fourth image feature to illustrate a technical solution for obtaining a density map through the first image feature and the fourth image feature.

[0081] In some embodiments, performing density estimation processing based on the first image feature to obtain a density map may include: performing feature extraction processing based on the target image to obtain a fourth image feature, the fourth image feature having a different scale from the first image feature; performing density estimation processing based on the first image feature to obtain a first density feature; performing density estimation processing based on the fourth image feature to obtain a second density feature; and performing fusion processing based on the first density feature and the second density feature to obtain a density map.

[0082] It should be noted that the target image can be passed through a convolutional layer in a neural network to extract features in the target image (such as the fourth image feature).

[0083] It should also be noted that the target image can be passed through the convolution layer in a shallow neural network to obtain a shallower first image feature; and the target image can be passed through the convolution layer in a deep neural network to obtain a deeper fourth image feature.

[0084] Furthermore, at this time, the scale of the first image feature is relatively coarse, and the scale of the fourth image feature is relatively fine. When both small and large targets exist in the target image, the first image feature has a better feature extraction effect for the smaller target, and the fourth image feature has a better feature extraction effect for the larger target. In this way, the first image feature and the fourth image feature can better extract large and small targets in the target image, thereby improving the accuracy of feature extraction.

[0085] In some embodiments, performing a fusion process on the first density feature and the second density feature to obtain a density map may include: performing a fusion process on the first density feature and the second density feature, and performing a convolution process on the fusion result to obtain a density map.

[0086] It should be understood that when obtaining the density map through the first image feature and the fourth image feature, a network structure of Feature Pyramid Network (FPN) can be introduced.

[0087] For example, Figure 2 1 is a schematic diagram of a framework for determining a density map provided by an embodiment of the present application. The framework may include a Gaussian Generating Module (GGM) 210, wherein the Gaussian Generating Module 210 may include a Confidence Block 220 and a Convolution (Conv) module 230.

[0088] like Figure 2 As shown, Indicates cumulative processing, Indicates fusion processing, assuming that the first image feature is recorded as f1, the fourth image feature is recorded as f2, the first density feature is recorded as d1, and the second density feature is recorded as d2. Obtaining a density map through f1 and f2 may include the following steps:

[0089] In the first step, the target image can be passed through the convolution layer in the shallow neural network to obtain the feature f 11 ; Pass the target image through the convolution layer in the deep neural network to obtain the feature f 12 .

[0090] The second step is to 12 By performing convolution processing, we can get the feature f 21 ; for f 21 By upsampling, we can get the feature f 22 ; Among them, f 11 and f 22 The scale of both is coarse; f 12 and f 21 The scales of the two are the same, and both are relatively fine.

[0091] The third step is to 11 and f 22 By performing accumulation processing (such as element-by-element addition), we can get f1. 12 and f 21 By performing accumulation processing (such as element-by-element addition), f2 can be obtained. The network structure based on the first to third steps to obtain f1 and f2 can be called a feature pyramid.

[0092] In the fourth step, the confidence module 220 performs density estimation processing on f1 to obtain d1; the confidence module 220 performs density estimation processing on f2 to obtain d2.

[0093] In the fifth step, d1 and d2 are fused, and the fused result is convolved by the convolution module 230 to obtain a density map.

[0094] Through this method, compared with obtaining a density map through the first image feature, when the density map is obtained through the first image feature and the fourth image feature, the accuracy of the density map can be made higher, so that the accuracy of the target confidence of at least two candidate boxes obtained subsequently based on the density map is higher.

[0095] Furthermore, the introduction of the feature pyramid network structure can improve the accuracy of feature extraction, making the first image feature and the fourth image feature more accurate; convolution processing of the fused first density feature and the second density feature can further improve the accuracy of the density map.

[0096] S140, determining target confidences of at least two candidate boxes according to the density map and the first image feature;

[0097] S150, determining at least one target candidate box according to the target confidences of the at least two candidate boxes respectively;

[0098] S160: Perform target detection processing on at least one target object according to at least one target candidate frame to obtain at least one target detection result.

[0099] It should be noted that, when there is only one target object, the target object may correspond to at least two candidate frames. According to the target confidences of the at least two candidate frames corresponding to the target object, the candidate frame with the highest target confidence may be selected as the target candidate frame, so that target detection can be performed on the target object according to the target candidate frame to obtain a target detection result.

[0100] It should also be noted that, when there are at least two targets, each of the at least two targets may correspond to at least two candidate frames. According to the target confidence of each of the at least two candidate frames corresponding to each target, the candidate frame with the highest target confidence may be selected as the target candidate frame, so that a target candidate frame corresponding to each of the at least two targets may be obtained, and then target detection may be performed on the at least two targets according to the target candidate frame corresponding to each of the at least two targets, so as to obtain at least two target detection results.

[0101] The embodiment of the present application provides a target detection method, which can obtain a target image; perform feature extraction processing based on the target image to obtain a first image feature; perform density estimation processing based on the first image feature to obtain a density map; determine the target confidence of each of at least two candidate frames based on the density map and the first image feature; wherein at least two candidate frames are used to perform target detection on the target image. In this way, the target confidence difference between at least two candidate frames can be increased, so that the ranking between at least two candidate frames is clearer, so that it is easier to select a better candidate frame (i.e., at least one target candidate frame) from at least two candidate frames to detect the target image, thereby improving the accuracy of target detection.

[0102] Exemplarily, when determining the target confidence of at least two candidate boxes based on the density map and the first image feature, the confidence difference between the positive sample candidate box and the negative sample candidate box can be increased, so that the ranking between the positive sample candidate box and the negative sample candidate box is clearer, thereby making it easier to select the positive sample candidate box.

[0103] In the embodiment of the present application, the target confidence of each of the at least two candidate boxes can be determined directly based on the density map and the first image feature; or the third image feature can be determined based on the density map and the first image feature, and then the target confidence of each of the at least two candidate boxes can be determined based on the third image feature. Figure 3 and Figure 4 , two implementation methods of determining the target confidence of at least two candidate boxes are described.

[0104] Figure 3 This is a detailed process diagram of a target detection method provided in an embodiment of the present application. Figure 1 .like Figure 3 As shown, the method may include the following steps S310 and S320:

[0105] S310, performing classification processing according to the first image feature to obtain initial confidence scores of at least two candidate boxes;

[0106] S320 , performing accumulation processing according to the density map and the initial confidences of the at least two candidate boxes to obtain the target confidences of the at least two candidate boxes.

[0107] It should be understood that the classification process according to the first image feature can be understood as passing the first image feature through a classification module (such as a classification subnet) to obtain the initial confidence of at least two candidate boxes. That is, the input of the classification module is the first image feature, and the output is the initial confidence of at least two candidate boxes.

[0108] It should also be understood that by performing classification processing based on the first image feature to obtain the initial confidence of at least two candidate frames, it can be understood that by performing classification processing on the first image feature, at least two candidate frames can be sorted according to the initial confidence, and the positive sample candidate frame and the negative sample candidate frame can be preliminarily distinguished.

[0109] It should be noted that the accuracy of the initial confidence of at least two candidate frames is low, the confidence difference between the positive sample candidate frame and the negative sample candidate frame is small, the order between the positive sample candidate frame and the negative sample candidate frame is relatively vague, and it is possible that the confidence of the negative sample candidate frame is greater than the confidence of the positive sample candidate frame. Based on this, the density map and the initial confidence of at least two candidate frames can be accumulated, so that at least two candidate frames are reordered once, the confidence difference between the positive sample candidate frame and the negative sample candidate frame is increased, so that the positive sample candidate frame can obtain a higher confidence and the confidence of the negative sample candidate frame can be suppressed.

[0110] That is to say, after obtaining the initial confidence of at least two candidate boxes, the density map can be used to reorder the initial confidence of the at least two candidate boxes, so that the candidate box closer to the center of the density map (i.e., the point close to 1) has a higher target confidence, and the candidate box farther away from the center of the density map (i.e., the point close to 0) has a lower target confidence, thereby making the confidence difference between the positive sample candidate box and the negative sample candidate box greater.

[0111] Furthermore, by correcting the initial confidence of each of the at least two candidate boxes again, the ranking between the at least two candidate boxes can be made better, and more negative sample candidate boxes can be deleted in the post-processing process (such as non-maximum suppression (NMS)), thereby improving the accuracy of the at least two candidate boxes.

[0112] Through this method, accumulation processing is performed based on the density map and the initial confidence of each of the at least two candidate boxes, so that the at least two candidate boxes can be reordered once, and based on the target confidence of each of the at least two candidate boxes obtained, a better candidate box (i.e., at least one target candidate box) can be easily selected from the at least two candidate boxes to detect the target image, thereby improving the accuracy of target detection.

[0113] Figure 4 This is a detailed process diagram of a target detection method provided in an embodiment of the present application. Figure 2 .like Figure 4 As shown, the method may include the following steps S410 to S430:

[0114] S410, performing product processing on the density map and the first image feature to obtain a second image feature;

[0115] S420, performing accumulation processing according to the first image feature and the second image feature to obtain a third image feature;

[0116] S430: Determine target confidences of at least two candidate boxes according to the third image feature.

[0117] It should be understood that the multiplication of the density map and the first image feature can be understood as a weight scaling of the first image feature through the density map, so that the features close to the center of the density map (i.e., points close to 1) can be retained and the features far from the center of the density map (i.e., points close to 0) can be deleted, thereby improving the extraction accuracy of the second image feature.

[0118] It should also be understood that the accumulation processing of the first image feature and the second image feature can be understood as that the missing part of the features in the second image feature can be increased by accumulating the first image feature and the second image feature, thereby further improving the extraction accuracy of the third image feature.

[0119] Through this method, the target confidence of each of at least two candidate boxes can be determined according to the third image feature, so that based on the target confidence of each of the at least two candidate boxes, it is easier to select a better candidate box (i.e., at least one target candidate box) from the at least two candidate boxes to detect the target image, thereby improving the accuracy of target detection.

[0120] In some embodiments, performing a product process on the density map and the first image feature to obtain the second image feature may include: performing a convolution process on the density map to obtain a convolved density map; performing a product process on the convolved density map and the first image feature to obtain the second image feature.

[0121] Through this method, based on the convolution processing of the density map, the accuracy of the density map can be further improved. When the second image feature is obtained based on the density map after the convolution, the extraction accuracy of the second image feature can also be improved.

[0122] Based on S410 and S420, the third image feature can be obtained. Since the extraction accuracy of the third image feature is relatively high, the target confidence of each of the at least two candidate boxes can be determined directly based on the third image feature; in addition, the target confidence of each of the at least two candidate boxes can also be determined based on the third image feature and the density map.

[0123] Based on this, according to the third image feature, the target confidence of each of at least two candidate boxes is determined, and there are two possible implementation methods.

[0124] In a possible implementation, classification processing is performed according to the third image feature to obtain target confidences of at least two candidate boxes.

[0125] It should be understood that the classification process according to the third image feature can be understood as passing the third image feature through a classification module (such as a classification subnetwork) to obtain the target confidence of at least two candidate boxes. That is, the input of the classification module is the third image feature, and the output is the target confidence of at least two candidate boxes.

[0126] It should also be understood that by performing classification processing based on the third image feature to obtain the target confidence of each of at least two candidate frames, it can be understood that by performing classification processing on the third image feature, at least two candidate frames can be sorted according to the target confidence, thereby distinguishing the positive sample candidate frame from the negative sample candidate frame.

[0127] Through this method, since the extraction accuracy of the third image feature is high, the accuracy of the target confidence of each of the at least two candidate frames obtained through the classification processing of the third image feature is also high. In this way, based on the target confidence of each of the at least two candidate frames obtained, it is easier to select a better candidate frame (that is, at least one target candidate frame) from the at least two candidate frames to detect the target image, thereby improving the accuracy of target detection.

[0128] Another possible implementation method is to perform classification processing based on the third image feature to obtain the initial confidence of each of at least two candidate boxes; and perform accumulation processing based on the density map and the initial confidence of each of the at least two candidate boxes to obtain the target confidence of each of the at least two candidate boxes.

[0129] It should be understood that the classification process according to the third image feature can be understood as passing the third image feature through a classification module (such as a classification subnetwork) to obtain the initial confidence of at least two candidate boxes. That is, the input of the classification module is the third image feature, and the output is the initial confidence of at least two candidate boxes.

[0130] It should also be understood that by performing classification processing based on the third image feature to obtain the initial confidence of at least two candidate frames, it can be understood that by performing classification processing on the third image feature, at least two candidate frames can be sorted according to the initial confidence, and the positive sample candidate frame and the negative sample candidate frame can be preliminarily distinguished.

[0131] It should be noted that in order to further improve the accuracy of the initial confidence of at least two candidate boxes, the density map and the initial confidence of at least two candidate boxes can be accumulated so that at least two candidate boxes are reordered once, further increasing the confidence difference between the positive sample candidate box and the negative sample candidate box, so that the positive sample candidate box can obtain a higher confidence and the confidence of the negative sample candidate box can be suppressed.

[0132] That is to say, after obtaining the initial confidence of at least two candidate boxes, the density map can be used to reorder the initial confidence of the at least two candidate boxes, so that the candidate box closer to the center of the density map (i.e., the point close to 1) has a higher target confidence, and the candidate box farther away from the center of the density map (i.e., the point close to 0) has a lower target confidence, thereby making the confidence difference between the positive sample candidate box and the negative sample candidate box greater.

[0133] Furthermore, by correcting the initial confidence of at least two candidate boxes again, the ranking between the at least two candidate boxes can be made better, and more negative sample candidate boxes can be deleted in the post-processing process (such as NMS), thereby further improving the accuracy of the positive sample candidate boxes.

[0134] Through this method, accumulation processing is performed based on the density map and the initial confidence of each of the at least two candidate boxes, so that the at least two candidate boxes can be reordered once, and based on the target confidence of each of the at least two candidate boxes obtained, a better candidate box (i.e., at least one target candidate box) can be easily selected from the at least two candidate boxes to detect the target image, thereby improving the accuracy of target detection.

[0135] In some embodiments, performing cumulative processing based on the density map and the initial confidences of at least two candidate boxes to obtain the target confidences of at least two candidate boxes can include: performing convolution processing based on the density map to obtain the convolved density map; performing cumulative processing based on the convolved density map and the initial confidences of at least two candidate boxes to obtain the target confidences of at least two candidate boxes.

[0136] Through this method, based on the convolution processing of the density map, the accuracy of the density map can be further improved. When the target confidence of at least two candidate boxes is obtained based on the density map after the convolution, the accuracy of the target confidence of at least two candidate boxes can also be improved.

[0137] In the embodiment of the present application, the target confidence of each of the at least two candidate boxes can be determined according to the density map and the first image feature; wherein the target confidence of each of the at least two candidate boxes can be determined by Figure 3 or Figure 4 However, the target confidence of some of the at least two candidate frames may be high and close, resulting in random suppression of these candidate frames in the post-processing process (such as NMS), and a large number of excellent candidate frames may be suppressed by suboptimal candidate frames, thereby reducing the accuracy of target detection.

[0138] For example, taking a smart retail scenario as an example, in a densely arranged shelf scene, the number of goods (i.e., target objects) in the target image may be large, and at least two goods are stacked together relatively densely, and the seriously overlapping goods may have the same appearance, resulting in more candidate boxes having higher and closer target confidences.

[0139] Based on this, the embodiment of the present application can update the target confidence of some candidate boxes after obtaining the target confidence of at least two candidate boxes, thereby reordering the part of candidate boxes before post-processing, increasing the difference between the target confidences of the part of candidate boxes, and thus improving the accuracy of target detection.

[0140] Figure 5 This is a detailed process diagram of a target detection method provided in an embodiment of the present application. Figure 3 .like Figure 5 As shown, the method may include the following steps S510 to S530:

[0141] S510, selecting the first N target confidences from the target confidences of the at least two candidate boxes, and the candidate boxes corresponding to the first N target confidences are the N candidate boxes in the at least two candidate boxes;

[0142] S520, determining positions of at least two candidate boxes in the density map according to the first image feature; determining positions of N candidate boxes in the density map according to the positions of at least two candidate boxes in the density map;

[0143] S530 , determining the updated target confidence of each of the N candidate boxes according to the target confidence of each of the N candidate boxes, the position of each of the N candidate boxes in the density map, and the density map.

[0144] Wherein, N is an integer greater than 1.

[0145] It should be understood that the candidate boxes corresponding to the first N target confidences are N candidate boxes among at least two candidate boxes. It can be understood that the first N target confidences correspond to N candidate boxes, and the target confidences of the N candidate boxes are among the first N target confidences.

[0146] Through this method, the target confidence of each of the N candidate boxes can be updated based on the target confidence of each of the N candidate boxes, the position of each of the N candidate boxes in the density map and the density map, so that the N candidate boxes can be reordered once, and the difference between the target confidences of each of the N candidate boxes can be increased. In the subsequent post-processing process (such as NMS), the better candidate boxes can be retained, thereby improving the accuracy of target detection.

[0147] In some embodiments, determining the target confidence of each of the N candidate boxes after update may include: performing feature extraction processing on the density map according to the positions of each of the N candidate boxes in the density map to obtain a sub-density map corresponding to each of the N candidate boxes; performing similarity comparison on the sub-density map corresponding to each of the N candidate boxes with a preset sub-density map to obtain the similarity corresponding to each of the N candidate boxes; performing weighted processing on the similarity corresponding to each of the N candidate boxes and the target confidence of each of the N candidate boxes to determine the target confidence of each of the N candidate boxes after update.

[0148] It should be noted that the preset sub-density map may be a standard normal distribution. For example, the preset sub-density map may be a standard two-dimensional (0,1) normal distribution.

[0149] Furthermore, the preset sub-density map may be set manually or in other ways, which is not limited in the embodiments of the present application.

[0150] In some embodiments, the sub-density map corresponding to each of the N candidate boxes is compared with a preset sub-density map for similarity to obtain the similarity corresponding to each of the N candidate boxes, which may include: vectorizing the sub-density map corresponding to each of the N candidate boxes to obtain the vector corresponding to each of the N candidate boxes; vectorizing the preset sub-density map to obtain the preset vector; comparing the vector corresponding to each of the N candidate boxes with the preset vector for similarity to obtain the similarity corresponding to each of the N candidate boxes.

[0151] Exemplarily, the similarity comparison may be L1 similarity comparison, L2 similarity comparison, cosine similarity comparison, etc., which is not limited in the embodiments of the present application.

[0152] Taking the smart retail scenario as an example, when N is equal to 4, the process of obtaining the similarities corresponding to the four candidate boxes is explained below.

[0153] Figure 6 This is a detailed process diagram of a target detection method provided in an embodiment of the present application. Figure 4 .like Figure 6 As shown, the method may include the following steps S610 and S620:

[0154] S610: Feature extraction processing.

[0155] By performing feature extraction processing on the density map according to the positions of the four candidate boxes in the density map, sub-density maps corresponding to the four candidate boxes can be obtained.

[0156] Taking the leftmost candidate box on the fourth row of the density map as an example, the density map is subjected to feature extraction processing based on the candidate box, and the bottommost sub-density map among the sub-density maps corresponding to each of the four candidate boxes can be obtained.

[0157] S620: Similarity comparison.

[0158] The sub-density maps corresponding to the four candidate boxes are compared with the preset sub-density maps for similarity, so as to obtain the similarities corresponding to the four candidate boxes.

[0159] in, Indicates that the sub-density map corresponding to the candidate box has a low similarity with the preset sub-density map; It indicates that the sub-density map corresponding to the candidate box has a high similarity with the preset sub-density map.

[0160] Through this method, the similarities corresponding to each of the N candidate boxes and the target confidences of each of the N candidate boxes are weighted, and the target confidences of each of the N candidate boxes can be updated, so that the N candidate boxes can be reordered once, and the difference between the target confidences of each of the N candidate boxes can be increased. In the subsequent post-processing process (such as NMS), the better candidate boxes can be retained, thereby improving the accuracy of target detection.

[0161] In the embodiment of the present application, when determining the positions of at least two candidate boxes in the density map, the positions of at least two candidate boxes in the density map can be obtained directly based on the first image feature; or the third image feature can be obtained based on the first image feature, and then the positions of at least two candidate boxes in the density map can be obtained based on the third image feature. Based on this, there are two possible implementation methods for determining the positions of at least two candidate boxes in the density map.

[0162] In a possible implementation, regression processing is performed according to the first image feature to obtain positions of at least two candidate boxes in the density map.

[0163] It should be understood that performing regression processing according to the first image feature can be understood as passing the first image feature through a regression module (such as a box subnet) to obtain the positions of at least two candidate boxes in the density map. That is, the input of the regression module is the first image feature, and the output is the positions of at least two candidate boxes in the density map.

[0164] By using this method, regression processing is performed directly based on the first image feature, which can improve the efficiency of obtaining the positions of at least two candidate boxes in the density map.

[0165] Another possible implementation method is to perform a multiplication process on the density map and the first image feature to obtain a second image feature; perform an accumulation process on the first image feature and the second image feature to obtain a third image feature; and perform a regression process on the third image feature to obtain the positions of at least two candidate boxes in the density map.

[0166] It should be understood that the process of obtaining the third image feature corresponds to the aforementioned step S410 and step S420, and the specific implementation of the aforementioned step S410 and step S420 may be referred to during implementation.

[0167] It should also be understood that performing regression processing according to the third image feature can be understood as passing the third image feature through a regression module (such as Box Subnet) to obtain the positions of at least two candidate boxes in the density map. That is, the input of the regression module is the third image feature, and the output is the positions of at least two candidate boxes in the density map.

[0168] Through this method, since the extraction accuracy of the third image feature is relatively high, the positions of at least two candidate boxes in the density map can be made more accurate by performing regression processing on the third image feature.

[0169] In an embodiment of the present application, after obtaining the target confidence of each of at least two candidate boxes, the target confidence of each of the at least two candidate boxes can be post-processed to further improve the accuracy of the at least two candidate boxes; wherein, the target confidence of each of the N candidate boxes can be the target confidence before the update or the target confidence after the update, and the embodiment of the present application is not limited to this.

[0170] In some embodiments, the method may further include: performing non-maximum suppression (NMS) processing according to the target confidence of each of the at least two candidate boxes to delete the candidate boxes whose target confidence is less than a first threshold among the at least two candidate boxes.

[0171] It should be noted that the first threshold may be set manually or in other ways, and the embodiments of the present application are not limited to this.

[0172] It should also be noted that the candidate frames whose target confidences among at least two candidate frames are less than the first threshold may be negative sample candidate frames or positive sample candidate frames, which is not limited in this embodiment of the present application.

[0173] It should also be noted that, in an embodiment of the present application, when the positions of some candidate boxes in at least two candidate boxes are relatively close in the density map, these some candidate boxes can also be deduplicated, thereby improving the detection efficiency of at least two candidate boxes.

[0174] Through this method, at least two candidate frames whose target confidence is less than the first threshold can be deleted through NMS processing, thereby further improving the accuracy of the at least two candidate frames.

[0175] The following takes a high-density shelf scene (such as a supermarket, warehouse, and other shelf scenes) as an example to illustrate the application of the target detection method provided in the embodiment of the present application in an actual scene.

[0176] Figure 7 is a schematic diagram of a framework for determining the target confidence of a candidate box provided in an embodiment of the present application. Figure 7 As shown, the framework may include a Gaussian generation module 210, a confidence ranking module (Confidence Ranking Module, CRM) 760, and a confidence re-ranking procedure (Confidence Re-Ranking Procedure, CR 2 P)770, represents the product processing, where:

[0177] The Gaussian generation module 210 may be used to determine a density map. For an explanation of determining a density map by using the Gaussian generation module 210, reference may be made to the aforementioned Figure 2 The description in will not be repeated here.

[0178] The confidence ranking module 760 can be used to determine the target confidence of each of the at least two candidate boxes, so that the at least two candidate boxes can be ranked based on the target confidence of each of the at least two candidate boxes. The confidence ranking module 760 may include a convolution module 230, a classification module 710, and a regression module 720. The confidence ranking module 760 determines the target confidence of each of the at least two candidate boxes and may include the following steps:

[0179] In the first step, the density map may be convolved by the convolution module 230 to obtain a convolved density map; the convolved density map may be multiplied with the first image feature f1 to obtain a second image feature;

[0180] In the second step, the second image feature can be convolved by the convolution module 230, and the convolved image feature and the first image feature f1 are accumulated to obtain a third image feature.

[0181] In the third step, the third image feature can be classified by the classification module 710 to obtain initial confidences of at least two candidate boxes.

[0182] In the fourth step, the density map can be convolved by the convolution module 230 to obtain a convolved density map; the convolved density map can be accumulated with the initial confidence of at least two candidate boxes to obtain the target confidence of at least two candidate boxes. In addition, the candidate box can be represented by "K*H*W", where K represents the type of target object that the candidate box can detect (such as in the detection of "cat" and "dog", K is equal to 2), H is the height of the candidate box, and W is the width of the candidate box.

[0183] In step 5, the third image feature can be regressed by the regression module 720 to obtain the positions of at least two candidate boxes in the density map. In addition, the candidate box can be represented by "4K*H*W", where "4" means that the candidate box has 4 borders.

[0184] It should be noted that the embodiment of the present application does not limit the execution order of the third step, the fourth step and the fifth step. That is, the third step and the fourth step can be executed first, and then the fifth step; or the fifth step can be executed first, and then the third step and the fourth step; or the third step and the fourth step can be executed at the same time as the fifth step.

[0185] It should also be noted that, in the dense target detection in the related art, after the image features are classified and regressed, the difference between the confidences of at least two candidate boxes is small. At this time, the at least two candidate boxes are directly put into the post-processing process, so that the accuracy of target detection is low. In the embodiment of the present application, the confidence of the candidate box whose center is closer to the center of the density map can be higher, so that the difference between the confidences of the positive sample candidate box and the negative sample candidate box is increased. In other words, the embodiment of the present application can add a cascade sorting structure on the basis of the classification and regression of the single-stage target detection network (RetinaNet), and correct the initial confidence of at least two candidate boxes again based on the density map, so that the accuracy of the target confidence of at least two candidate boxes is higher, and more negative sample candidate boxes can be deleted in the post-processing process, thereby improving the accuracy of target detection.

[0186] The confidence reordering module 770 can be used to determine the target confidence of each of the N candidate boxes in at least two candidate boxes, so that the N candidate boxes can be reordered based on the target confidence of each of the N candidate boxes. The confidence reordering module 770 may include a feature extraction module 730, a similarity comparison module 740, and a weighting module 750. The confidence reordering module 770 determines the target confidence of each of the N candidate boxes and may include the following steps:

[0187] In the first step, the first N target confidences are selected from the target confidences of at least two candidate boxes, and the candidate boxes corresponding to the first N target confidences are the N candidate boxes in the at least two candidate boxes.

[0188] In the second step, according to the positions of at least two candidate boxes in the density map, the positions of N candidate boxes in the density map are determined.

[0189] In the third step, according to the positions of the N candidate boxes in the density map, the density map is subjected to feature extraction processing by the feature extraction module 730 to obtain sub-density maps corresponding to the N candidate boxes.

[0190] Step 4: The sub-density maps corresponding to the N candidate boxes are compared with the preset sub-density maps through the similarity comparison module 740 to obtain the similarities corresponding to the N candidate boxes.

[0191] In the fifth step, the similarities corresponding to the N candidate boxes and the target confidences of the N candidate boxes are weighted by the weighting module 750 to determine the updated target confidences of the N candidate boxes.

[0192] It should be noted that, in the densely arranged shelf scene, there are a large number of commodities in the target image (such as more than 500 commodities), and at least two commodities are densely placed, and these severely overlapping commodities often have the same appearance, resulting in more positive sample candidate frames having higher confidence after passing through the confidence sorting module 760. In the embodiment of the present application, the region of interest alignment (RoI Align) technology can be used to perform feature extraction processing on the density map through N candidate frames to obtain the sub-density map corresponding to each of the N candidate frames, and the sub-density map corresponding to each of the N candidate frames is compared with the preset sub-density map for similarity (such as cosine similarity comparison), and the similarity corresponding to each of the obtained N candidate frames is weighted with the target confidence of each of the N candidate frames, so that the target confidence of each of the N candidate frames can be reordered. In this way, the difference between the confidences of each of the N candidate frames can be increased, which is more conducive to selecting a more accurate positive sample candidate frame.

[0193] The embodiment of the present application provides a dense target detection algorithm based on confidence re-ranking, and the Gaussian generation module 210 can be used to generate a density map, and the initial confidences of at least two candidate boxes can be re-ranked by the density map in the confidence ranking module 760, thereby increasing the difference between the target confidences of at least two candidate boxes. In the confidence re-ranking module 770, the density map can be used to re-rank the target confidences of the N candidate boxes again, thereby increasing the difference between the target confidences of the N candidate boxes and improving the accuracy of target detection.

[0194] The density map is not combined with the confidence of the candidate boxes in the related art; however, the confidence ranking module 760 proposed in the embodiment of the present application can re-rank the initial confidence of at least two candidate boxes based on the density map.

[0195] In the related art, the confidences of at least two positive sample candidate boxes may be mixed together and cannot be distinguished. In the post-processing process (such as NMS), the confidences of the at least two positive sample candidate boxes will be randomly suppressed. However, in the embodiment of the present application, the target confidences of at least two positive sample candidate boxes can be reordered again through the confidence reordering module 770, so that the positive sample candidate boxes with poor target confidences can be suppressed in the processing process (such as NMS), thereby improving the accuracy of target detection.

[0196] The embodiment of the present application proposes a confidence ranking module, which can re-rank the confidence of at least two candidate boxes, increase the difference in confidence between the positive sample candidate box and the negative sample candidate box, so that the positive sample candidate box can obtain a higher confidence, and the negative sample candidate box can be suppressed in the post-processing process; the embodiment of the present application also proposes a confidence re-ranking module, which can re-rank the confidence of each of the N candidate boxes, increase the difference in confidence between the better positive sample candidate box and other candidate boxes, and retain the better positive sample candidate box in the post-processing process, thereby improving the accuracy of target detection. Taking the smart retail scenario as an example, users can improve the accuracy of target detection in the shelf scenario by using the dense target detection solution proposed in the embodiment of the present application.

[0197] Based on the foregoing embodiments, an embodiment of the present application provides a target detection device, which includes the units included and the modules included in the units, which can be implemented by a processor in a computer device; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA), etc.

[0198] Figure 8 is a schematic diagram of the structure of a target detection device provided in an embodiment of the present application, such as Figure 8 As shown, the target detection device 800 may include: an acquisition module 810, a processing module 820 and a determination module 830, wherein:

[0199] An acquisition module 810 is used to acquire a target image, where the target image includes at least one target object;

[0200] The processing module 820 is used to perform feature extraction processing according to the target image to obtain a first image feature; perform density estimation processing according to the first image feature to obtain a density map;

[0201] A determination module 830 is used to determine target confidences of at least two candidate boxes according to the density map and the first image feature; and to determine at least one target candidate box according to the target confidences of the at least two candidate boxes.

[0202] The processing module 820 is further configured to perform target detection processing on at least one target object according to at least one target candidate frame to obtain at least one target detection result.

[0203] In some embodiments, the processing module 820 is further used to perform classification processing according to the first image feature to obtain the initial confidence of each of at least two candidate boxes; and perform accumulation processing according to the density map and the initial confidence of each of at least two candidate boxes to obtain the target confidence of each of at least two candidate boxes.

[0204] In some embodiments, the processing module 820 is further used to perform a product process based on the density map and the first image feature to obtain a second image feature; perform an accumulation process based on the first image feature and the second image feature to obtain a third image feature; and the determination module 830 is further used to determine the target confidence of each of at least two candidate boxes based on the third image feature.

[0205] In some embodiments, the processing module 820 is further configured to perform classification processing according to the third image feature to obtain target confidences of at least two candidate boxes.

[0206] In some embodiments, the processing module 820 is further used to perform classification processing according to the third image feature to obtain the initial confidence of at least two candidate boxes; and perform accumulation processing according to the density map and the initial confidence of at least two candidate boxes to obtain the target confidence of at least two candidate boxes.

[0207] In some embodiments, Figure 8 As shown, the target detection device 800 may further include: a selection module 840, wherein:

[0208] A selection module 840 is used to select the first N target confidences from the target confidences of the at least two candidate boxes, and the candidate boxes corresponding to the first N target confidences are the N candidate boxes in the at least two candidate boxes;

[0209] The determination module 830 is also used to determine the positions of at least two candidate boxes in the density map based on the first image feature; determine the positions of N candidate boxes in the density map based on the positions of at least two candidate boxes in the density map; determine the updated target confidence of each of the N candidate boxes based on the target confidence of each of the N candidate boxes, the positions of each of the N candidate boxes in the density map and the density map; wherein N is an integer greater than 1.

[0210] In some embodiments, Figure 8 As shown, the target detection device 800 may further include: a comparison module 850, wherein:

[0211] The processing module 820 is further used to perform feature extraction processing on the density map according to the positions of the N candidate boxes in the density map, so as to obtain sub-density maps corresponding to the N candidate boxes.

[0212] A comparison module 850 is used to compare the sub-density maps corresponding to the N candidate boxes with the preset sub-density maps to obtain the similarities corresponding to the N candidate boxes.

[0213] The processing module 820 is further used to perform weighted processing on the similarities corresponding to the N candidate boxes and the target confidences of the N candidate boxes to determine the updated target confidences of the N candidate boxes.

[0214] In some embodiments, the processing module 820 is further configured to perform regression processing according to the first image feature to obtain respective positions of at least two candidate boxes in the density map.

[0215] In some embodiments, the processing module 820 is further used to perform a multiplication process based on the density map and the first image feature to obtain a second image feature; perform an accumulation process based on the first image feature and the second image feature to obtain a third image feature; and perform a regression process based on the third image feature to obtain the positions of at least two candidate boxes in the density map.

[0216] In some embodiments, the processing module 820 is further used to perform non-maximum suppression processing according to the target confidence of each of the at least two candidate boxes, so as to delete the candidate boxes whose target confidence is less than the first threshold among the at least two candidate boxes.

[0217] In some embodiments, the processing module 820 is also used to perform feature extraction processing based on the target image to obtain a fourth image feature, and the scale of the fourth image feature is different from that of the first image feature; perform density estimation processing based on the first image feature to obtain a first density feature; perform density estimation processing based on the fourth image feature to obtain a second density feature; perform fusion processing based on the first density feature and the second density feature to obtain a density map.

[0218] The embodiment of the present application provides a target detection device, which can obtain a target image; perform feature extraction processing according to the target image to obtain a first image feature; perform density estimation processing according to the first image feature to obtain a density map; determine the target confidence of each of at least two candidate frames according to the density map and the first image feature; wherein the at least two candidate frames are used to perform target detection on the target image. In this way, the target confidence difference between the at least two candidate frames can be increased, so that the ranking between the at least two candidate frames is clearer, so that it is easier to select a better candidate frame (i.e., at least one target candidate frame) from the at least two candidate frames to detect the target image, thereby improving the accuracy of target detection.

[0219] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. In some embodiments, the functions or modules included in the device provided in the embodiment of the present application can be used to execute the method described in the above method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0220] It should be noted that in the embodiment of the present application, if the above-mentioned target detection method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific hardware, software or firmware, or any combination of hardware, software, and firmware.

[0221] An embodiment of the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program that can be executed on the processor, and when the processor executes the program, some or all of the steps in the above method are implemented.

[0222] The embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, some or all of the steps in the above method are implemented. The computer-readable storage medium can be transient or non-transient.

[0223] An embodiment of the present application provides a computer program, including a computer-readable code. When the computer-readable code is run in a computer device, a processor in the computer device executes some or all of the steps for implementing the above method.

[0224] The embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, some or all of the steps in the above method are implemented. The computer program product can be implemented specifically by hardware, software or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium, and in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK) and the like.

[0225] It should be noted here that the description of the various embodiments above tends to emphasize the differences between the various embodiments, and the same or similar aspects can be referenced to each other. The description of the above device, storage medium, computer program and computer program product embodiments is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the embodiments of the device, storage medium, computer program and computer program product of this application, please refer to the description of the method embodiment of this application for understanding.

[0226] It should be noted that Fig. 9 is a hardware entity diagram of a computer device provided in an embodiment of the present application, such as Fig. 9 As shown, the hardware entity of the computer device 900 includes: a processor 901, a communication interface 902 and a memory 903, wherein:

[0227] Processor 901 generally controls the overall operation of computer device 900 .

[0228] The communication interface 902 enables the computer device to communicate with other terminals or servers through a network.

[0229] The memory 903 is configured to store instructions and applications executable by the processor 901, and can also cache data to be processed or processed by the processor 901 and each module in the computer device 900 (for example, image data, audio data, voice communication data, and video communication data), which can be implemented by flash memory (FLASH) or random access memory (Random Access Memory, RAM). Data transmission between the processor 901, the communication interface 902 and the memory 903 can be carried out through the bus 904.

[0230] It should be understood that in various embodiments of the present application, the size of the serial number of each step / process does not mean the order of execution, and the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The serial numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0231] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0232] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0233] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0234] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be separately configured as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0235] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0236] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0237] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A target detection method, characterized in that: The method comprises: Acquire a target image, wherein the target image includes at least one target object; Performing feature extraction processing on the target image to obtain a first image feature; Performing density estimation processing according to the first image feature to obtain a density map; Determining target confidences of at least two candidate boxes, respectively, according to the density map and the first image feature; Selecting first N target confidences from the target confidences of the at least two candidate frames, the candidate frames corresponding to the first N target confidences are the N candidate frames in the at least two candidate frames; Determining, according to the first image feature, respective positions of the at least two candidate boxes in the density map; Determining the positions of the N candidate boxes in the density map according to the positions of the at least two candidate boxes in the density map; Determine the updated target confidence of each of the N candidate boxes according to the target confidence of each of the N candidate boxes, the position of each of the N candidate boxes in the density map, and the density map; wherein N is an integer greater than 1; Determine at least one target candidate frame according to the target confidences of the at least two candidate frames respectively; According to the at least one target candidate frame, target detection processing is performed on the at least one target object to obtain at least one target detection result.

2. The method according to claim 1, characterized in that The determining, according to the density map and the first image feature, respective target confidences of at least two candidate boxes includes: Performing classification processing according to the first image feature to obtain initial confidence scores of the at least two candidate frames; The density map and the initial confidences of the at least two candidate boxes are respectively accumulated to obtain the target confidences of the at least two candidate boxes.

3. The method according to claim 1, characterized in that The determining, according to the density map and the first image feature, respective target confidences of at least two candidate boxes includes: Performing a product process on the density map and the first image feature to obtain a second image feature; performing an accumulation process on the first image feature and the second image feature to obtain a third image feature; According to the third image feature, target confidences of the at least two candidate boxes are determined.

4. The method according to claim 3, characterized in that Determining the target confidence of each of the at least two candidate frames according to the third image feature includes: Classification processing is performed according to the third image feature to obtain target confidence levels of the at least two candidate boxes.

5. The method according to claim 3, characterized in that: Determining the target confidence of each of the at least two candidate frames according to the third image feature includes: Performing classification processing according to the third image feature to obtain initial confidence scores of the at least two candidate frames; The density map and the initial confidences of the at least two candidate boxes are respectively accumulated to obtain the target confidences of the at least two candidate boxes.

6. The method according to claim 1, characterized in that The determining of the updated target confidence of each of the N candidate frames includes: According to the positions of the N candidate boxes in the density map, feature extraction is performed on the density map to obtain sub-density maps corresponding to the N candidate boxes. Comparing the sub-density maps corresponding to the N candidate frames with the preset sub-density maps for similarity, to obtain the similarities corresponding to the N candidate frames; The similarities corresponding to the N candidate frames and the target confidences of the N candidate frames are weighted to determine the updated target confidences of the N candidate frames.

7. The method according to claim 1, characterized in that The determining, according to the first image feature, positions of the at least two candidate boxes in the density map respectively includes: Regression processing is performed according to the first image feature to obtain positions of the at least two candidate boxes in the density map.

8. The method according to claim 1, characterized in that The determining, according to the first image feature, positions of the at least two candidate boxes in the density map respectively includes: Performing a product process on the density map and the first image feature to obtain a second image feature; performing an accumulation process on the first image feature and the second image feature to obtain a third image feature; Regression processing is performed according to the third image feature to obtain the positions of the at least two candidate boxes in the density map.

9. The method according to any one of claims 1 to 8, characterized in that The performing density estimation processing according to the first image feature to obtain a density map includes: Performing feature extraction processing on the target image to obtain a fourth image feature, wherein the fourth image feature has a different scale from the first image feature; Density estimation is performed based on the first image feature to obtain a first density feature; density estimation is performed based on the fourth image feature to obtain a second density feature; and fusion processing is performed based on the first density feature and the second density feature to obtain the density map.

10. A target detection device, characterized in that: include: An acquisition module, used for acquiring a target image, wherein the target image includes at least one target object; A processing module, configured to perform feature extraction processing according to the target image to obtain a first image feature; and perform density estimation processing according to the first image feature to obtain a density map; A determination module, configured to determine target confidences of at least two candidate boxes according to the density map and the first image feature; and determine at least one target candidate box according to the target confidences of the at least two candidate boxes; The processing module is further configured to perform target detection processing on the at least one target object according to the at least one target candidate frame to obtain at least one target detection result; A selection module is used to select the first N target confidences from the target confidences of at least two candidate boxes, and the candidate boxes corresponding to the first N target confidences are the N candidate boxes in the at least two candidate boxes; The determination module is also used to determine the positions of at least two candidate boxes in the density map based on the first image feature; determine the positions of N candidate boxes in the density map based on the positions of at least two candidate boxes in the density map; determine the updated target confidence of each of the N candidate boxes based on the target confidence of each of the N candidate boxes, the positions of each of the N candidate boxes in the density map and the density map; wherein N is an integer greater than 1.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the program, the steps in the method according to any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 9 are implemented.

Images