Small scale target detection method, device and storage medium

Abstract

The present application provides a small-scale target detection method, which includes: inputting an image to be detected into a trained detection model to obtain a current-level detection result of a small-scale target; slicing the image to be detected to obtain a slice set and a covering small target set X. small The minimum convex set C of the image is obtained; if it is determined that all small-scale targets are contained in the slice set, the images in the minimum slice set are input into the trained detection model to obtain the secondary detection results of the small-scale targets. Otherwise, the minimum slice set is obtained by calculating the intersection and union ratio of the slices in the subset of the slice set and the remaining area of ​​the image to be detected, and the minimum slice set is input into the trained detection model to obtain the secondary detection results of the small-scale targets; the current level detection result of the small-scale target is superimposed with the secondary detection result of the small-scale target to obtain the ultimate detection result of the small-scale target. The technical solution of the present application can improve the detection effect of small-scale targets and ensure the efficiency of cascade detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of pattern recognition, and in particular to a small-scale target detection method, device and storage medium. BACKGROUND

[0002] Target detection refers to collecting a scene image through a camera, analyzing a real object in the image, and identifying what the target is. Target detection not only has a wide range of applications in daily life, but also has a demand for target detection in non-daily life fields such as satellite remote sensing and radar detection.

[0003] Due to the influence of factors such as shooting distance and target size, there are often small-scale targets in scene images. For the detection of small-scale targets, it is generally recognized in the industry that it is a key problem in the field of target detection. At present, the small-scale target detection method mainly enhances the representation of small-scale target features by designing a pyramid structure, so as to identify small-scale targets in scene images. However, when the scale of small-scale targets in the scene image is too small, the scale information loss caused by the down-sampling of the neural network still inevitably affects the detection of small-scale targets. SUMMARY

[0004] The present application provides a small-scale target detection method, device and storage medium, which can improve the detection effect of small-scale targets and ensure the efficiency of cascading detection.

[0005] In one aspect, the present application provides a small-scale target detection method, which comprises:

[0006] training a deep detection model to obtain a trained detection model;

[0007] inputting a to-be-detected image into the trained detection model to obtain a current stage detection result of a small-scale target, wherein the current stage detection result comprises a small target set X composed of a plurality of small-scale target bounding boxes small ;

[0008] obtaining a slice set and a minimum convex set C covering the small target set X small by slicing the to-be-detected image;

[0009] determining whether all the small-scale targets are contained in the slice set obtained by slicing the to-be-detected image;

[0010] if it is determined that all the small-scale targets are contained in the minimum slice set , the minimum slice set The image in is input into the trained detection model to obtain the secondary detection result of the small-scale target, otherwise, the slice set is calculated by The minimum slice set is obtained by the intersection and union ratio of the slices in the subset and the remaining area of ​​the image to be detected And input the trained detection model to obtain the secondary detection result of the small-scale target, the remaining area of ​​the image to be detected is the area after removing the selected slice from the image to be detected, and the selected slice is the slice set The slice in the subset of with the largest intersection ratio with the minimum convex set C;

[0011] The current level detection result of the small-scale object is superimposed with the secondary detection result of the small-scale object to obtain a final detection result of the small-scale object.

[0012] On the other hand, the present application provides a small-scale target detection device, the device comprising:

[0013] A training module is used to train the depth detection model to obtain a trained detection model;

[0014] The first detection module is used to input the image to be detected into the trained detection model to obtain the current level detection result of the small-scale target, and the current level detection result includes a small target set X consisting of multiple small-scale target bounding boxes. small ;

[0015] The acquisition module is used to obtain a slice set by slicing the image to be detected and covers the small target set X small The smallest convex set C of

[0016] A judgment module is used to judge whether all the small-scale targets are included in the slice set obtained when slicing the image to be detected Inside;

[0017] The second detection module is used to determine if all small-scale targets are included in the minimum slice set Then the minimum slice set The image in is input into the trained detection model to obtain the secondary detection result of the small-scale target, otherwise, the slice set is calculated by The minimum slice set is obtained by the intersection and union ratio of the slices in the subset and the remaining area of ​​the image to be detected and input into the trained detection model to obtain the secondary detection result of the small-scale target, and the remaining area of ​​the image to be detected is the area after removing the selected slice from the image to be detected, and the selected slice is the slice set The slice in the subset of with the largest intersection ratio with the minimum convex set C;

[0018] fusing the current stage detection result of the small scale target and the secondary detection result of the small scale target to obtain a final detection result of the small scale target.

[0019] In a third aspect, the present application provides an electronic device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the technical solution of the small scale target detection method as described above when executing the computer program.

[0020] In a fourth aspect, the present application provides a storage medium, which stores a computer program, and the computer program implements the steps of the technical solution of the small scale target detection method as described above when executed by a processor.

[0021] From the technical solution of the present application as described above, it can be known that the image to be detected is input into the trained detection model to obtain a current stage detection result of a small scale target, and if it is determined that all small scale targets are contained in a minimum slice set , then the pictures in the minimum slice set are input into the trained detection model to obtain a secondary detection result of the small scale target, and the current stage detection result of the small scale target and the secondary detection result of the small scale target are fused to obtain a final detection result of the small scale target. On the one hand, unlike the cascade of network structure in the prior art such as feature pyramid when detecting a small scale target, the technical solution of the present application is to slice the image to be detected with the small scale target as the center, so that the scale variance of the small scale target in the image set to be detected is reduced, that is, the region where the small scale target is located is cascaded to obtain a slice and is cascaded to detect, the scale of the small scale target in the slice is improved, and thus the detection effect of the trained detection model on the small scale target is improved. On the other hand, the image to be detected is input into the trained detection model to obtain a current stage detection result of a small scale target, which can be used as a first stage detection in cooperation with a fast large-scale detection method to ensure the efficiency of the cascade detection. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0023] Figure 1 is a flowchart of the small scale target detection method provided by the embodiments of the present application;

[0024] Figure 2 is a flowchart of judging whether all small-scale targets are contained in a slice set obtained by slicing a to-be-detected image, provided by an embodiment of the present application;

[0025] Figure 3 is a schematic diagram of superimposing cascade detection results of small-scale targets, provided by an embodiment of the present application;

[0026] Figure 4 is a structural schematic diagram of a small-scale target detection device, provided by an embodiment of the present application;

[0027] Figure 5 is a structural schematic of an electronic device, provided by an embodiment of the present application. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0029] In the present specification, adjectives such as first and second can only be used to distinguish one element or action from another element or action, without necessarily requiring or implying any actual such relationship or order. Where the context permits, reference to an element or component or step (etc.) by the indefinite article "a" or "an" does not exclude a plurality of such elements, components, or steps, and so forth.

[0030] In the present specification, for the convenience of description, the sizes of various parts shown in the drawings are not drawn in accordance with the actual proportional relationship.

[0031] Due to the influence of various factors such as shooting distance, target size, etc., there are often small-scale targets in scene images. For the detection of small-scale targets, it is generally recognized in the industry that it is a key problem in target detection field. At present, the small-scale target detection method mainly enhances the representation of small-scale target features by designing a pyramid structure, so as to identify small-scale targets in scene images. However, when the scale of small-scale targets in scene images is too small, the loss of scale information caused by the down-sampling of neural network still inevitably affects the detection of small-scale targets.

[0032] In view of the above problems of the prior art, the present application proposes a small-scale target detection method, the flowchart of which is shown in FIG. 1, mainly including steps S101 to S106, which are described in detail as follows: Figure 1

[0033] ​Step S101: training the depth detection model to obtain a trained detection model.

[0034] In the embodiments of the present application, training the depth detection model includes two stages of preparing a target detection image dataset and constructing and training the depth detection model, wherein the target detection image dataset includes to-be-detected images and image annotation information, and the annotation standard for the images is not fixed, for example, either a region coordinate position can be used for representation, or a typical detection bounding box can be used for representation (i.e., [x1, y1, x2, y2]), or a region pixel-level annotation, wherein (x1, y1) and (x2, y2) respectively represent the coordinates of the top left and bottom right vertices of the bounding box, and in the present application, the detection bounding box representation, i.e., [x1, y1, x2, y2], is used to represent the detection box where the target is located.

[0035] As for constructing and training the depth detection model, the present application uses the depth detection model to analyze the video stream or image received by the visual sensor to realize target detection, and the depth detection model can use a commonly used depth target detection network structure, for example, a yolov5 network, which includes a feature extraction backbone structure, a feature enhancement pyramid neck structure, and a multi-scale prediction head, etc. The training method uses a commonly used detection network model training method, and the structure of the depth detection model and the training method thereof are not limited in the present application.

[0036] Step S102: inputting the to-be-detected image into the trained detection model to obtain a current level detection result of the small scale target, wherein the current level detection result includes a small target set X composed of a plurality of small scale target bounding boxes small .

[0037] For the convenience of subsequent description, in the embodiments of the present application, the size of the input image of the trained detection model, i.e., the to-be-detected image, is denoted as HxW (the size of the original image is H I xW I ), and the output of any ith target bounding box is denoted as b (i) = [x (i) , y (i) , h (i) , w (i) ], wherein x (i) , y (i) represent the coordinates of the center of the target bounding box, h (i) and w (i)Respectively represent the height and width of the target frame. The corresponding targets can be divided according to the size of the target frame. For example, if the frame area is less than 32×32, the corresponding target is a small-scale target. If 32×32<frame area<96×96, the corresponding target is a medium-scale target. If the frame area is greater than 96×96, the corresponding target is a large-scale target. The above definition of the size of the target frame and the classification of the target are merely examples and should not be regarded as limitations of this application.

[0038] If the detection result output by the trained detection model is recorded as the bounding box set Among them, b (i) is the i-th target bounding box, i.e. b (i) =[x (i) ,y (i) , h (i) , w (i) ], c i Represents the confidence of the target category to which the i-th target bounding box corresponds, and N is the number of target bounding boxes in the bounding box set. By traversing all target bounding boxes in the bounding box set and following the above-mentioned criteria for dividing target bounding boxes and their corresponding targets, a small target set X consisting of multiple small-scale target bounding boxes can be screened out from the bounding box set output by the trained detection model. small As the current level detection result of small-scale targets, that is, in, is the i-th small-scale target bounding box, i.e. c i Indicates the confidence of the category to which the i-th small-scale target bounding box corresponds to the small-scale target. and Respectively represent the center coordinates of the i-th small-scale target frame, i.e., the horizontal axis coordinate and the vertical axis coordinate, and Represents the height and width of the i-th small-scale target border, N s is the small target set X small The number of small and medium-scale object bounding boxes.

[0039] Step S10 3: Slice the image to be detected to obtain a slice set and covers the small target set X small The smallest convex set C of .

[0040] As an embodiment of the present application, a slice set is obtained by slicing the image to be detected. and covers the small target set X small The minimum convex set C can be achieved through steps S1031 to S1033, as detailed below:

[0041] Step S1031: traverse the small target set Xsmall All small-scale target bounding boxes in the image.

[0042] Small target set X small That is, the current level detection result obtained in the foregoing step S102 contains Traverse the small target set X small All small-scale target bounding boxes in the image, that is, traverse N s Small-scale target bounding boxes.

[0043] Step S1032: For any one small-scale target bounding box traversed, slice the detection image centered on the center coordinates of the small-scale target bounding box and according to a preset size to obtain a slice set Wherein, N slice is the number of slices in the slice set .

[0044] In the field of pattern recognition, the input size of the detection image is usually set for the target detection network, and the original image larger than the size will be transformed in size through the Resize operation, which makes the size of the small-scale target smaller. However, when the target is too small, the loss of scale information caused by the down-sampling of the neural network inevitably affects the detection of the small-scale target by the target detection network. In view of the above problem, the idea is changed, that is, the size of the detection image is not changed, but for any one small-scale target bounding box traversed, the detection image is sliced centered on the center coordinates of the small-scale target bounding box and according to a preset size. For example, for the i-th small-scale target bounding box , the center is , the coordinates of the top left corner of the j-th slice is , and the coordinates of the top right corner of the j-th slice is , the i-th slice slice ( i ) is obtained by slicing the detection image, that is, Wherein, and are the horizontal axis coordinates and the vertical axis coordinates of the center of the j-th slice, and are the height and width of the j-th slice. When slicing, if the horizontal axis coordinates and / or the vertical axis coordinates of the center of the slice exceed the boundary constituted by the size H I ×W I of the original image, the exceeding part is padded with 0. Through the above slicing method, the detection image can be sliced into N slice slices, and the set constituted by the N slice slices is the slice set Wherein, N slice is the number of slices in the slice set .

[0045] It should be noted that although the image to be detected is sliced ​​with the center coordinates of any small-scale target bounding box as the center, since one slice may cover several small-scale targets, the slice set The number of slices N slice Not necessarily with the small target set X small The number of all small-scale object bounding boxes N s Equal, this is also the case for the i-th small-scale target bounding box by Slice the image to be detected as the center, and the resulting slice (j) The reason why the superscript is not marked with i but with .

[0046] Step S1033: According to the small target set X small The center coordinates and size of each small-scale target frame in the calculation of the coverage small target set X small The smallest convex set C of .

[0047] Specifically, according to the small target set X small The center coordinates and size of each small-scale target frame in the calculation of the coverage small target set X small The minimum convex set C can be: calculating the difference between the horizontal axis coordinate of the center of each small-scale target frame in multiple small-scale target frames and its half width, that is, the first difference, and the difference between the vertical axis coordinate and its half height, that is, the second difference, and taking the minimum of the first differences and the minimum of the second differences as the first horizontal axis coordinate and the first vertical axis coordinate of the minimum convex set C; calculating the sum of the horizontal axis coordinate of the center of each small-scale target frame in multiple small-scale target frames and its half width, that is, the first sum, and the sum of the vertical axis coordinate and its half height, that is, the second sum, and taking the minimum of the first sum and the minimum of the second sum as the second horizontal axis coordinate and the second vertical axis coordinate of the minimum convex set C.

[0048] If cx1 represents the first horizontal coordinate of the minimum convex set C, cy1 represents the first vertical coordinate of the minimum convex set C, cx2 represents the second horizontal coordinate of the minimum convex set C, and cy2 represents the second vertical coordinate of the minimum convex set C, that is, C = (cx1, cy1, cx2, cy2), then according to the small target set X small The center coordinates and size of each small-scale target frame in the calculation of the coverage small target set X small The minimum convex set C can be realized by the following formula:

[0049]

[0050] Among them, min() is the minimum function, which means taking the minimum value of the object in ().

[0051] Step S104: judging whether all small-scale targets are contained in the slice set obtained by slicing the image to be detected .

[0052] By judging whether all small-scale targets are contained in the minimum slice set obtained by slicing the image to be detected , the purpose is to obtain a minimum slice set, i.e., a set of the least slices covering all small-scale targets. In theory, the number of slices in the minimum slice set is less than the number of slices in the slice set , which is beneficial to the requirement of computing resources of the subsequent stage processing or can save the computing resources of the subsequent stage processing. As an embodiment of the present application, judging whether all small-scale targets are contained in the minimum slice set obtained by slicing the image to be detected may be implemented by steps S1 to S4, which are described in detail as follows:

[0053] Step S1: calculating the slice set . The subset of the slice set is the slice set obtained after the slice set is updated.

[0054] As the calculation process proceeds, the number of slices in the slice set changes, and therefore, the subset of the slice set may be the initial slice set or the slice set obtained after the slice set is updated.

[0055] Step S2: selecting the slice with the maximum intersection union ratio of the subset of the slice set to the minimum convex set C as the new element of the selected slice set S select . select The selected slice set S select is updated by adding the new element of the selected slice set S select , and the new element of the selected slice set S is removed from the slice set .

[0056] It should be noted that, in the embodiment of the present application, the element of the selected slice set S select comes from the slice with the maximum intersection union ratio of the subset of the slice set to the minimum convex set C, in other words, whenever a slice with the maximum intersection union ratio of the slice set to the minimum convex set C is calculated by step S1, the slice will be selected and added to the selected slice set Sselect , at the same time, remove the slice from the slice collection Culling to update the slice collection Get a slice collection For example, suppose that before the current calculation process starts, the slice set Select slice set S select ={slice (sel-1) , slice (sel-2) ,slice (sel-3)}, after calculation in step S1, slice (1) The intersection and union ratio with the minimum convex set C is the largest, then slice (1) Add to selected slice set S select To update the selected slice set S select (Remember that the selected slice set after updating is S′ elect ), that is, S′ elect ={slice (sel-1) , slice (sel-2) , slice (sel-3) ,slice (1)}; At the same time, slice (1) From a slice collection Eliminate and slice the collection Update to {slice (2) , slice (3) , slice (4) , slice (5)}. Obviously, {slice (2) , slice (3) , slice (4) , slice (5)}for A subset of .

[0057] Step S3: Calculate the coverage of the selected slice set S select The minimum convex set C* of the selected slice set after updating.

[0058] As an embodiment of the present application, the calculation covers the selected slice set S selectThe minimum convex set C* of the updated selected slice set can be: calculating the difference between the horizontal axis coordinate of the center of each slice in the updated selected slice set and its half width, that is, the first difference, and the difference between the vertical axis coordinate and its half height, that is, the second difference, and taking the minimum of the first differences and the minimum of the second differences as the first horizontal axis coordinate and the first vertical axis coordinate of the minimum convex set C*; calculating the sum of the horizontal axis coordinate of the center of each slice in the updated selected slice set and its half width, that is, the first sum, and the sum of the vertical axis coordinate and its half height, that is, the second sum, and taking the minimum of the first sum and the minimum of the second sum as the second horizontal axis coordinate and the second vertical axis coordinate of the minimum convex set G*.

[0059] If Represents the first horizontal axis coordinate of the minimum convex set C, with Represents the first vertical axis coordinate of the minimum convex set C, Represents the second horizontal axis coordinate of the minimum convex set C, Represents the second vertical axis coordinate of the minimum convex set C, that is, C = cx1, cy1, cx2, cy2), then calculate the coverage of the selected slice set S select The minimum convex set C* of the selected slice set after the update can be achieved by the following formula:

[0060]

[0061]

[0062] in, and They represent the horizontal and vertical coordinates of the center of the jth selected slice in the updated selected slice set, respectively. and Respectively represent the width and height of the th selected slice in the updated selected slice set. The slice with the largest intersection ratio with the minimum convex set in the subset of the selected slice set is added to the selected slice set to update the selected slice set, and the newly added elements of the dynamic selected slice set are removed from the slice set to update the slice set.

[0063] Step S4: If Then make sure that all small-scale targets are included in the minimum slice set and the selected slice set S′ elect Determine the minimum slice set, otherwise, the process turns to calculating the slice set The minimum slice set is obtained by the intersection and union ratio of the slices in the subset and the remaining area of ​​the image to be detected.

[0064] like Then make sure that all small-scale targets are included in the minimum slice set and the selected slice set S′ elect Determine the minimum slice set Once the minimum slice set is determined, the images in the minimum slice set are sequentially input into the trained detection model to obtain the secondary detection results of small-scale targets. elect That is, the slice set selected in step s1 The slice with the largest intersection ratio between the slice and the minimum convex set C in the subset of is selected as the slice set S select Add the new elements to the selected slice set S select Then, select the slice set S select The updated selected slice set, that is, according to the example of the above embodiment, S′ elect ={slice (sel -1) , slice (sel-2) , slice (sel-3) , slice (1)}.like Not established, that is Then the process turns to step S105 "by calculating the slice set The minimum slice set is obtained by the intersection and union ratio of the slices in the subset and the remaining area of ​​the image to be detected ”.

[0065] Step S105: If it is determined that all small-scale objects are included in the minimum slice set If the image in the minimum slice set is within the training detection model, the image in the minimum slice set is input into the trained detection model to obtain the secondary detection result of the small-scale target. Otherwise, the image in the minimum slice set is input into the trained detection model to obtain the secondary detection result of the small-scale target. The minimum slice set is obtained by the intersection and union ratio of the slices in the subset and the remaining area of ​​the image to be detected and input into the trained detection model to obtain the secondary detection result of the small-scale target, where the remaining area of ​​the image to be detected is the area after removing the selected slice from the image to be detected, and the selected slice is the slice set The intersection of the slice in the subset of and the smallest convex set C is greater than the largest slice.

[0066] From the above description of step S105, it can be seen that there are two ways to determine or obtain the "minimum slice set". One way is to determine that all small-scale objects are included in the minimum slice set. Internal judgment When the selected slice set S' elect Determine the minimum slice set Another approach is to calculate the slice set The minimum slice set is obtained by the intersection and union ratio of the slices in the subset and the remaining area of ​​the image to be detected. As mentioned above, as the calculation process proceeds, the slice set the number of slices in the slice set S select is changed, i.e., the slice set S is updated. Figure 2 The process from step S1 to step S105 in the above embodiment is a loop as shown in the flowchart.

[0067] As an embodiment of the present application, the minimum slice set S may be obtained by calculating the intersection-over-union of the slices in the subset of the slice set S

[0068] Step S’1: traverse the subset of the slice set S .

[0069] Step S’2: calculate the intersection-over-union IoU of the slices in the subset of the slice set S and the remaining region of the image to be detected according to the following formula: (i)

[0070]

[0071] wherein C (i) and c (j) represent the convex set of the small scale target in the i-th and j-th loop (i.e., the i-th and j-th judgment of whether all small scale targets are contained in the slice set S obtained when slicing the image to be detected), and slice (j) represents any one slice before the i-th slice slice (i) .

[0072] Step S’3: select the slice with the maximum intersection-over-union of the slices in the subset of the slice set S and the remaining region of the image to be detected as the new element of the selected slice set S select and add it to the selected slice set S select to update the selected slice set S selecc .

[0073] ​The above steps S'1 to S'3 are repeated until the complete slice set is traversed After the end of the loop, the selected slice set S at the end of the loop is outputted select The minimum slice set is determined.

[0074] Step S106: superimpose the current stage detection result of the small-scale target and the secondary detection result of the small-scale target to obtain the final detection result of the small-scale target.

[0075] Specifically, superimposing the current stage detection result of the small-scale target and the secondary detection result of the small-scale target to obtain the final detection result of the small-scale target can be: projecting the small-scale target bounding box corresponding to the secondary detection result of the small-scale target to the small-scale target bounding box corresponding to the current stage detection result of the small-scale target through translation and size adjustment operation; superimposing the current stage detection result of the small-scale target and the secondary detection result of the small-scale target to obtain a bounding box sequence list containing a series of fusion target bounding boxes, as shown in Figure 3 The following steps S'1 to S'4 are performed on the bounding box sequence list:

[0076] Step S'1: sort the confidence of the fusion target bounding box in the bounding box sequence list according to the high and low of the confidence;

[0077] Step S'2: select the fusion target bounding box with the highest confidence and add it to the final output list, and delete the fusion target bounding box with the highest confidence from the bounding box sequence list;

[0078] Step S'3: calculate the intersection over union of the fusion target bounding box with the highest confidence and other fusion target bounding boxes in the bounding box sequence list;

[0079] Step S'4: delete the fusion target bounding box with the intersection over union greater than the threshold in the bounding box sequence list;

[0080] Repeat the process of steps S'1 to S'4 until the bounding box sequence list is empty, and the fusion target bounding box in the final output list is taken as the final detection result of the small-scale target.

[0081] It should be noted that in the above steps S'1 to S'4, the confidence of the fusion target bounding box is the confidence of the bounding box set recorded target bounding box b (i) confidence c i .

[0082] From the above Figure 1 The small-scale target detection method can be known that the image to be detected is input into the trained detection model to obtain the current stage detection result of the small-scale target, and if it is determined that all small-scale targets are contained in the minimum slice set Within, the image in the minimum slice set is input into the trained detection model to obtain the secondary detection result of the small-scale target, and the current-level detection result of the small-scale target is superimposed with the secondary detection result of the small-scale target to obtain the ultimate detection result of the small-scale target. On the one hand, unlike the existing technologies such as feature pyramid, which only cascade the network structure when detecting small-scale targets, the technical solution of the present application is to slice the image to be detected with the small-scale target as the center, so that the scale variance of the small-scale target in the image set to be detected becomes smaller, that is, by cascading the slices of the area where the small-scale target is located and performing cascade detection, the scale of the small-scale target in the slice is improved, thereby improving the detection effect of the trained detection model on the small-scale target; on the other hand, the image to be detected is input into the trained detection model to obtain the current-level detection result of the small-scale target, which can be used in conjunction with a fast large-scale detection method as the first-level detection to ensure the efficiency of the cascade detection.

[0083] Please see the attached Figure 4 , is a small-scale target detection device provided in an embodiment of the present application. The device may include a training module 401, a first detection module 402, an acquisition module 403, a judgment module 404, a second detection module 405, and a fusion module 406, as detailed below:

[0084] A training module 401 is used to train the depth detection model to obtain a trained detection model;

[0085] The first detection module 402 is used to input the image to be detected into the trained detection model to obtain the current level detection result of the small-scale target, wherein the current level detection result includes a small target set X consisting of multiple small-scale target bounding boxes. small ;

[0086] The acquisition module 403 is used to obtain a slice set by slicing the image to be detected. and covers the small target set X small The smallest convex set C of

[0087] The judgment module 404 is used to judge whether all small-scale targets are included in the minimum slice set obtained when slicing the image to be detected. Inside;

[0088] The second detection module 405 is used to determine that all small-scale objects are included in the minimum slice. Then the minimum slice set The image in the input has been trained detection model to obtain the secondary detection results of small-scale targets, otherwise, by calculating the slice set the intersection-over-union ratio of the slice in the subset of the slice set and the remaining region of the image to be detected is the minimum slice set, and the minimum slice set is input into the trained detection model to obtain a secondary detection result of the small-scale target the slice with the maximum intersection-over-union ratio of the slice in the subset of the slice set and the minimum convex set C;

[0089] The fusion module 406 is configured to superimpose the current-level detection result of the small-scale target and the secondary detection result of the small-scale target to obtain a final detection result of the small-scale target.

[0090] From the above description of the accompanying drawings Figure 4 The small-scale target detection device can be known that the image to be detected is input into the trained detection model to obtain a current-level detection result of the small-scale target, and if it is determined that all the small-scale targets are included in the minimum slice set The small-scale target detection device can be known that the image to be detected is input into the trained detection model to obtain a current-level detection result of the small-scale target, and if it is determined that all the small-scale targets are included in the minimum slice set

[0091] Figure 5 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. As shown in the figure, the electronic device 5 of the embodiment mainly comprises a processor 50, a memory 51, and a computer program 52 stored in the memory 51 and executable on the processor 50, such as a program of a small-scale target detection method. The processor 50 implements the steps in the above-described small-scale target detection method embodiments when executing the computer program 52, such as steps S101-S106 shown in the figure. Figure 5 Alternatively, the processor 50 implements the functions of the modules / units in the above-described device embodiments when executing the computer program 52, such as the functions of the training module 401, the first detection module 402, the obtaining module 403, the judging module 404, the second detection module 405, and the fusion module 406 shown in the figure. Figure 1 Figure 4

[0092] ​​Exemplarily, the computer program 52 of the small-scale object detection method mainly includes: training a depth detection model to obtain a trained detection model; inputting an image to be detected into the trained detection model to obtain a current-level detection result of the small-scale object, wherein the current-level detection result includes a small target set X consisting of multiple small-scale object bounding boxes. small ; Obtain a slice set by slicing the image to be detected and covers the small target set X small The smallest convex set C of the image to be detected; determine whether all small-scale targets are included in the minimum slice set obtained when slicing the image to be detected If it is determined that all small-scale targets are included in the minimum slice set Then the minimum slice set The image in the input has been trained detection model to obtain the secondary detection results of small-scale targets, otherwise, by calculating the slice set The minimum slice set is obtained by the intersection and union ratio of the slices in the subset and the remaining area of ​​the image to be detected and input into the trained detection model to obtain the secondary detection result of the small-scale target, where the remaining area of ​​the image to be detected is the area after removing the selected slice from the image to be detected, and the selected slice is the slice set The slice with the largest intersection-and-union ratio of the slice in the subset of the smallest convex set C and the slice with the largest intersection-and-union ratio of the slice in the subset of the smallest convex set C; the current level detection result of the small-scale target is superimposed with the secondary detection result of the small-scale target to obtain the ultimate detection result of the small-scale target. The computer program 52 can be divided into one or more modules / units, one or more modules / units are stored in the memory 51 and executed by the processor 50 to complete the present application. One or more modules / units can be a series of computer program instruction segments that can perform specific functions, and the instruction segments are used to describe the execution process of the computer program 52 in the electronic device 5. For example, the computer program 52 can be divided into the functions of the training module 401, the first detection module 402, the acquisition module 403, the judgment module 404, the second detection module 405 and the fusion module 406 (modules in the virtual device), and the specific functions of each module are as follows: the training module 401 is used to train the depth detection model to obtain a trained detection model; the first detection module 402 is used to input the image to be detected into the trained detection model to obtain the current level detection result of the small-scale target, wherein the current level detection result includes a small target set X consisting of multiple small-scale target bounding boxes. small ; Acquisition module 403, for obtaining a slice set by slicing the image to be detected and covers the small target set X small The minimum convex set C of the image to be detected; the judgment module 404 is used to judge whether all small-scale targets are included in the minimum slice set obtained when slicing the image to be detected the second detection module 405 is configured to input the pictures in the minimum slice set into the trained detection model to obtain the secondary detection result of the small-scale target if it is determined that all the small-scale targets are contained in the minimum slice set the second detection module 405 is configured to input the pictures in the minimum slice set into the trained detection model to obtain the secondary detection result of the small-scale target if it is determined that all the small-scale targets are contained in the minimum slice set the second detection module 405 is configured to input the pictures in the minimum slice set into the trained detection model to obtain the secondary detection result of the small-scale target if it is determined that all the small-scale targets are contained in the minimum slice set the second detection module 405 is configured to input the pictures in the minimum slice set into the trained detection model to obtain the secondary detection result of the small-scale target if it is determined that all the small-scale targets are contained in the minimum slice set

[0093] The electronic device 5 can include but is not limited to a processor 50 and a memory 51. Those skilled in the art can understand that Figure 5 The electronic device 5 is only an example and does not constitute a limitation on the electronic device 5, and can include more or fewer components than the illustration, or combine certain components, or different components, for example, the electronic device can also include an input / output device, a network access device, a bus, etc.

[0094] The processor 50 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0095] The memory 51 can be an internal storage unit of the electronic device 5, for example, a hard disk or a memory of the electronic device 5. The memory 51 can also be an external storage device of the electronic device 5, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 5. Further, the memory 51 can also include both the internal storage unit and the external storage device of the electronic device 5. The memory 51 is used to store computer programs and other programs and data required by the electronic device. The memory 51 can also be used to temporarily store data that has been output or will be output.

[0096] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit. In addition, the specific name of each functional unit and module is only for easy distinction, and does not limit the protection scope of the present application. The specific working process of the unit and module in the above device can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0097] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0098] Those of ordinary skill in the art can appreciate that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0099] In the embodiments provided by the present application, it should be understood that the disclosed apparatuses / devices and methods can be implemented in other manners. For example, the embodiments of the apparatus / device described above are merely schematic, and the division of the modules or units can be different from the above. For example, one or more units can be combined, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0100] The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, that is, can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0101] In addition, each functional unit in the various embodiments of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0102] If the integrated module / unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can be instructed by a computer program to relevant hardware to complete, and the computer program of the small-scale target detection method can be stored in a storage medium. When the processor executes the computer program, the steps of each method embodiment can be implemented, that is, the trained detection model is obtained by training the depth detection model; the current level detection result of the small-scale target is obtained by inputting the to-be-detected image into the trained detection model, wherein the current level detection result contains a small target set X small composed of a plurality of small-scale target bounding boxes; the slice set and the smallest convex set C covering the small target set X small are obtained by slicing the to-be-detected image; it is determined whether all small-scale targets are contained in the smallest slice set obtained by slicing the to-be-detected image; if it is determined that all small-scale targets are contained in the smallest slice set , the picture in the smallest slice set is input into the trained detection model to obtain the secondary detection result of the small-scale target, otherwise, the secondary detection result of the small-scale target is obtained by calculating the slice set the intersection-over-union ratio of the slice in the subset of the slice set and the remaining region of the image to be detected is the minimum slice set, and the trained detection model is input to obtain a secondary detection result of the small-scale target the slice in the subset of the slice set and the minimum convex set C is the maximum intersection-over-union ratio; and the current-level detection result of the small-scale target and the secondary detection result of the small-scale target are superimposed to obtain a final detection result of the small-scale target. The computer program includes computer program code, which can be in the form of source code, object code, an executable file, or some intermediate form, etc. The storage medium can include any entity or device capable of carrying the computer program code, a recording medium, a U disk, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier wave signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the storage medium does not include electrical carrier wave signals and telecommunication signals.

[0103] The above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. The above specific embodiments further illustrate the purpose, technical solutions and beneficial effects of the present application, and it should be understood that the above description is only a specific embodiment of the present application, and is not used to limit the protection scope of the present application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A small scale target detection method, characterized in that, The method comprises: training a deep detection model to obtain a trained detection model; inputting the image to be detected into the trained detection model to obtain a current stage detection result of the small scale target, the current stage detection result comprising a small target set X composed of a plurality of small scale target bounding boxes small ; obtaining a slice set by slicing the image to be detected and a minimum convex set C covering the small target set X small , specifically comprising: traversing all small-scale target bounding boxes in the small target set X small ; for any one small-scale target bounding box traversed, slicing the image to be detected with the center coordinates as the center and according to a preset size to obtain a slice set where N slice is the number of slices in the slice set ; according to the center coordinates and size of each small-scale target bounding box in the small target set X small , calculating a minimum convex set C covering the small target set X small ; determining whether all of the small-scale targets are contained in a minimum slice set obtained when slicing the image to be detected inwardly If it is determined that all small-scale targets are contained in the minimum slice set , input the pictures in the minimum slice set into the trained detection model to obtain secondary detection results of the small-scale targets, otherwise, obtain the minimum slice set by calculating the intersection-over-union of the slices in the subset of the slice set and the remaining region of the image to be detected, and input the trained detection model to obtain secondary detection results of the small-scale targets, wherein the remaining region of the image to be detected is the region of the image to be detected after removing the selected slices, and the selected slice is the slice with the maximum intersection-over-union of the subset of the slice set and the minimum convex set C. superimposing the current-stage detection result of the small-scale target and the sub-stage detection result of the small-scale target to obtain a final detection result of the small-scale target.

2. The method of claim 1, wherein the small scale target detection method is characterized by, The minimum convex set C covering the small target set X small is calculated according to the center coordinates and sizes of each small-scale target bounding box small . calculating a difference between a horizontal axis coordinate of a center of each small-scale target bounding box in the plurality of small-scale target bounding boxes and a half width thereof, i.e., a first difference, and a difference between a vertical axis coordinate of the center and a half height thereof, i.e., a second difference, and taking a minimum value in the first difference and a minimum value in the second difference as a first horizontal axis coordinate and a first vertical axis coordinate of the minimum convex set C respectively; calculating a sum of the horizontal axis coordinate of the center of each small-scale target bounding box in the plurality of small-scale target bounding boxes and the half width thereof, i.e., a first sum, and a sum of the vertical axis coordinate of the center and the half height thereof, i.e., a second sum, and taking a minimum value in the first sum and a minimum value in the second sum as a second horizontal axis coordinate and a second vertical axis coordinate of the minimum convex set C respectively.

3. The method of claim 1, wherein the small scale target detection method is characterized by, the judging whether all the small-scale targets are contained in the minimum slice set obtained when slicing the image to be detected within the steps S1 to S4: Step S1: calculating the intersection ratio of the slices in a subset of the slice set with the smallest convex set C, the subset of the slice set being the slice set after updating the slice set ​ Step S2: select the slices in the slice set as the selected slice set S with the maximum intersection ratio with the minimum convex set C select Add the new elements of the selected slice set S select to update the selected slice set S select , and remove the new elements of the selected slice set S select from the slice set to update the slice set Step S3: Compute the convex hull C of the updated selected slice set S select the minimum convex set C of the updated selected slice set S * ; Step S4: If Then determine that all small-scale targets are included in the minimum slice set And the selected slice set S' elect Determine the minimum slice set Otherwise, the process turns to computing the slice set The minimum slice set is obtained by the intersection and union ratio of the slices in the subset and the remaining area of ​​the image to be detected 4. The small-scale target detection method according to claim 3, wherein: The computing covers the selected slice set S select The minimum convex set C of the updated selected slice set * , comprising: calculating a first difference value of a horizontal axis coordinate of a center of each slice in the updated selected slice set and a half width thereof and a second difference value of a vertical axis coordinate of the center and a half height thereof, and taking a minimum value of the first difference values and a minimum value of the second difference values as the minimum convex set C * a first horizontal axis coordinate and a first vertical axis coordinate of the center of the selected slice calculating a first sum value of a sum of a horizontal axis coordinate of a center of each slice in the updated selected slice set and a half width thereof and a second sum value of a sum of a vertical axis coordinate of the center of each slice in the updated selected slice set and a half height thereof, and taking a minimum value of the first sum value and a minimum value of the second sum value as the minimum convex set C * a second horizontal axis coordinate and a second vertical axis coordinate.

5. The method of claim 1, wherein the small scale target detection method is characterized by, The slice set is calculated by The minimum slice set is obtained by the intersection and union ratio of the slices in the subset and the remaining area of ​​the image to be detected, including steps S'1 to S'3: Step S'1 : traversing the set of slices subsets of the slices; Step S'2: Calculate the intersection over union IoU of the subset of slices of the set of slices with the remaining region of the image to be detected according to the following formula (i) : The C (i) and C (j) respectively denote the convex set in which the small-scale target is located in the i-th and j-th loop. Step S'3: selecting the slices in the subset of the slice set as the selected slice set S with the maximum intersection-over-union of the selected slice set S select with the remaining region of the image to be detected as the selected slice set S select to update the selected slice set S select ; The above steps S'1 to S'3 are cycled until the entire slice set is traversed After the loop ends, the selected slice set S at the end of the loop is determined select The minimum slice set is determined 6. The method of claim 1, wherein the small scale target detection method is characterized by, The superimposing the current-stage detection result of the small-scale target and the sub-stage detection result of the small-scale target to obtain a final detection result of the small-scale target comprises: projecting, through a translation and size adjustment operation, a small-scale target bounding box corresponding to the sub-stage detection result of the small-scale target to a small-scale target bounding box corresponding to the current-stage detection result of the small-scale target; superimposing the current-stage detection result of the small-scale target and the sub-stage detection result of the small-scale target to obtain a bounding box sequence list containing a series of fusion target bounding boxes, and performing the following steps S”1 to S”4 on the bounding box sequence list: Step S”1: sorting, according to the confidence levels, the confidence levels of the fusion target bounding boxes in the bounding box sequence list; Step S”2: selecting a fusion target bounding box with the highest confidence level to add to a final output list, and deleting the fusion target bounding box with the highest confidence level from the bounding box sequence list; Step S”3: calculating an intersection-over-union of the fusion target bounding box with the highest confidence level and other fusion target bounding boxes in the bounding box sequence list; Step S”4: deleting a fusion target bounding box with an intersection-over-union greater than a threshold value in the bounding box sequence list; repeating the steps S”1 to S”4 until the bounding box sequence list is empty, and taking the fusion target bounding boxes in the final output list as the final detection result of the small-scale target.

7. A small scale target detection apparatus, characterized by, The device comprises: a training module configured to train a deep detection model to obtain a trained detection model; The first detection module is configured to input a to-be-detected image into the trained detection model, and obtain a current stage detection result of small-scale targets, wherein the current stage detection result comprises a small target set X composed of a plurality of small-scale target bounding boxes. small ; The acquisition module is configured to acquire a slice set by slicing the image to be detected and a minimum convex set C covering the small target set X small , specifically comprising: traversing all small-scale target bounding boxes in the small target set X small ; for any one small-scale target bounding box traversed, slicing the image to be detected with the center coordinates of the small-scale target bounding box as the center and according to a preset size to obtain a slice set where N slice is the number of slices in the slice set ; calculating a minimum convex set C covering the small target set X small according to the center coordinates and size of each small-scale target bounding box in the small target set X small ; A judging module is configured to judge whether all the small-scale targets are contained in the minimum slice set obtained when the image to be detected is sliced In; a second detection module, configured to input pictures in the minimum slice set into the trained detection model to obtain secondary detection results of the small-scale target if it is determined that all the small-scale targets are contained in the minimum slice set , otherwise, obtain the minimum slice set by calculating the intersection-over-union of the slices in the subset of the slice set and the remaining region of the image to be detected, input the minimum slice set into the trained detection model to obtain the secondary detection results of the small-scale target, wherein the remaining region of the image to be detected is the region of the image to be detected after removing the selected slices, and the selected slices are the slices with the maximum intersection-over-union of the subset of the slice set and the minimum convex set C; a fusion module configured to superimpose the current-stage detection result of the small-scale target and the sub-stage detection result of the small-scale target to obtain a final detection result of the small-scale target.

8. An electronic device, the device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor implements the steps of the method of any one of claims 1 to 6 when executing the computer program.

9. A storage medium storing a computer program, characterized by The computer program, when executed by the processor, implements the steps of the method of any one of claims 1 to 6.

Images