Small-scale target detection method and device and storage medium

Abstract

The invention provides a small-scale target detection method, and the method comprises the steps: inputting a to-be-detected image into a trained detection model, and obtaining a current-level detection result of a small-scale target; the method comprises the following steps: slicing an image to be detected to obtain a slice set # imgabs0 # and a minimum convex set C covering a small target set Xsmall; if it is determined that all the small-scale targets are included in the slice set # imgabs1 #, pictures in the minimum slice set are input into the trained detection model to obtain secondary detection results of the small-scale targets, and otherwise, the pictures in the minimum slice set are input into the trained detection model to obtain the secondary detection results of the small-scale targets. Obtaining a minimum slice set by calculating the intersection-union ratio of slices in a subset of the slice set # imgabs2 # and the remaining area of the to-be-detected image, and inputting the minimum slice set into the trained detection model to obtain a secondary detection result of the small-scale target; and superposing the current-level detection result of the small-scale target and the secondary-level detection result of the small-scale target to obtain an ultimate detection result of the small-scale target. According to the technical scheme of the invention, the detection effect of a small-scale target can be improved, and the cascade detection efficiency can be ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of pattern recognition, and in particular to a small-scale target detection method, device and storage medium. Background Art

[0002] Object detection refers to the process of capturing scene images through a camera and analyzing the objects in the images to identify the objects in them. Object detection is not only widely used in daily life, but also in non-daily life fields such as satellite remote sensing and radar detection.

[0003] Due to the influence of various factors such as shooting distance and target size, small-scale targets often exist in scene images. The detection of small-scale targets is generally recognized by the industry as a key problem in the field of target detection. At present, small-scale target detection methods mainly enhance the representation of small-scale target features by designing a pyramid structure, thereby identifying small-scale targets in scene images. However, when the scale of small-scale targets in scene images is too small, the scale information loss caused by neural network downsampling will inevitably affect the detection of small-scale targets. Summary of the invention

[0004] The present application provides a small-scale target detection method, device and storage medium, which can improve the detection effect of small-scale targets and ensure the efficiency of cascade detection.

[0005] On the one hand, the present application provides a small-scale target detection method, the method comprising:

[0006] Train the depth detection model to obtain a trained detection model;

[0007] The image to be detected is input into the trained detection model to obtain the current level detection result of the small-scale target, wherein the current level detection result includes a small target set X composed of multiple small-scale target frames. small ;

[0008] By slicing the image to be detected, a slice set is obtained and covers the small target set X small The smallest convex set C of

[0009] Determine whether all the small-scale targets are included in the slice set obtained when slicing the image to be detected Inside;

[0010] If it is determined that all small-scale targets are included in the minimum slice set Within, the minimum slice set The image in is input into the trained detection model to obtain the secondary detection result of the small-scale target, otherwise, by calculating the slice set The minimum slice set is obtained by the intersection and union ratio of the slices in the subset and the remaining area of ​​the image to be detected. The trained detection model is input to obtain the secondary detection result of the small-scale target, the remaining area of ​​the image to be detected is the area after the selected slice is removed from the image to be detected, and the selected slice is the slice set The slice in the subset of with the largest intersection ratio with the minimum convex set C;

[0011] The current level detection result of the small-scale target is superimposed with the secondary level detection result of the small-scale target to obtain the ultimate detection result of the small-scale target.

[0012] On the other hand, the present application provides a small-scale target detection device, the device comprising:

[0013] A training module, used to train the deep detection model to obtain a trained detection model;

[0014] The first detection module is used to input the image to be detected into the trained detection model to obtain the current level detection result of the small-scale target, and the current level detection result includes a small target set X composed of multiple small-scale target frames. small ;

[0015] The acquisition module is used to obtain a slice set by slicing the image to be detected. and covers the small target set X small The smallest convex set C of

[0016] A judgment module is used to judge whether all the small-scale targets are included in the slice set obtained when slicing the image to be detected. Inside;

[0017] The second detection module is used to determine that all small-scale targets are included in the minimum slice set. If The image in is input into the trained detection model to obtain the secondary detection result of the small-scale target, otherwise, by calculating the slice set The minimum slice set is obtained by the intersection and union ratio of the slices in the subset and the remaining area of ​​the image to be detected and input into the trained detection model to obtain the secondary detection result of the small-scale target, the remaining area of ​​the image to be detected is the area after removing the selected slice from the image to be detected, and the selected slice is the slice set The slice in the subset of with the largest intersection ratio with the minimum convex set C;

[0018] A fusion module is used to superimpose the current level detection result of the small-scale target with the secondary level detection result of the small-scale target to obtain the ultimate detection result of the small-scale target.

[0019] In a third aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the technical solution of the small-scale target detection method as described above when executing the computer program.

[0020] In a fourth aspect, the present application provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the technical solution of the above-mentioned small-scale target detection method.

[0021] From the technical solution provided by the present application, it can be seen that the image to be detected is input into the trained detection model to obtain the current level detection result of the small-scale target. If it is determined that all small-scale targets are included in the minimum slice set, If The image in is input into the trained detection model to obtain the secondary detection result of the small-scale target, and the current level detection result of the small-scale target is superimposed with the secondary detection result of the small-scale target to obtain the ultimate detection result of the small-scale target. On the one hand, unlike the existing technologies such as feature pyramid, which are only cascaded in network structure when detecting small-scale targets, the technical solution of the present application is centered on the small-scale target, and the image to be detected is sliced ​​to reduce the scale variance of the small-scale target in the image set to be detected, that is, by cascading the slices of the area where the small-scale target is located and performing cascade detection, the scale of the small-scale target in the slice is improved, thereby improving the detection effect of the trained detection model on the small-scale target; on the other hand, the image to be detected is input into the trained detection model to obtain the current level detection result of the small-scale target, which can be used in conjunction with a fast large-scale detection method as the first level detection to ensure the efficiency of the cascade detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 is a flow chart of a small-scale target detection method provided in an embodiment of the present application;

[0024] Figure 2 is a flowchart for determining whether all small-scale targets are included in a slice set obtained when slicing an image to be detected, provided by an embodiment of the present application;

[0025] Figure 3 is a schematic diagram of superimposed cascade detection results of small-scale targets provided in an embodiment of the present application;

[0026] Figure 4 is a schematic diagram of the structure of a small-scale target detection device provided in an embodiment of the present application;

[0027] Figure 5 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] In this specification, adjectives such as first and second may be used only to distinguish one element or action from another element or action, without necessarily requiring or implying any actual such relationship or order. Where circumstances permit, reference to an element or component or step (etc.) should not be interpreted as being limited to only one of the elements, components, or steps, but may be one or more of the elements, components, or steps, etc.

[0030] In this specification, for the convenience of description, the sizes of various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0031] Due to the influence of various factors such as shooting distance and target size, small-scale targets often exist in scene images. The detection of small-scale targets is generally recognized by the industry as a key problem in the field of target detection. At present, small-scale target detection methods mainly enhance the representation of small-scale target features by designing a pyramid structure, thereby identifying small-scale targets in scene images. However, when the scale of small-scale targets in scene images is too small, the scale information loss caused by neural network downsampling will inevitably affect the detection of small-scale targets.

[0032] In view of the above problems in the prior art, this application proposes a small-scale target detection method, the flow chart of which is shown in the attached figure. Figure 1 As shown, it mainly includes steps S101 to S106, which are described in detail as follows:

[0033] Step S101: Train the depth detection model to obtain a trained detection model.

[0034] In an embodiment of the present application, training the depth detection model includes two stages: preparing a target detection image dataset and constructing a depth detection model and training it. The target detection image dataset includes an image to be detected and image annotation information, and the annotation standard for the image is not fixed. For example, it can be represented by regional coordinate position, or by a typical detection border representation (i.e., [x1, y1, x2, y2]) or regional pixel-level annotation, where (x1, y1) and (x2, y2) represent the coordinates of the upper left and lower right vertices of the border, respectively. In the present application, the detection border representation, i.e., [x1, y1, x2, y2", is used to represent the detection box where the target is located.

[0035] As for building a deep detection model and training it, this application uses a deep detection model to analyze the video stream or image received by the visual sensor to achieve target detection. The deep detection model can use a commonly used deep target detection network structure, such as the yolov5 network, which includes a feature extraction backbone structure, a feature enhancement pyramid neck structure, and a multi-scale prediction head, etc. The training method uses a commonly used detection network model training method, and this application does not limit the structure of the deep detection model and its training method.

[0036] Step S102: Input the image to be detected into the trained detection model to obtain the current level detection result of the small-scale target, wherein the current level detection result includes a small target set X consisting of multiple small-scale target bounding boxes. small .

[0037] For the convenience of subsequent description, in the embodiment of the present application, the size of the input image of the trained detection model, i.e., the image to be detected, is H×W (the original image size is H I ×W I ), output any i-th target bounding box as b (i) =[x (i) ,y (i) ,h (i) , w (i) ], where x (i) ,y (i) Indicates the coordinates of the center of the target bounding box, h (i) and w (i)Respectively represent the height and width of the target border. The corresponding targets can be divided according to the size of the target border. For example, if the border area is less than 32×32, the corresponding target is a small-scale target. If 32×32<border area<96×96, the corresponding target is a medium-scale target. If the border area is greater than 96×96, the corresponding target is a large-scale target. The above definition of the size of the target border and the classification of the target are merely examples and should not be regarded as limitations on this application.

[0038] If the detection result output by the trained detection model, i.e., the bounding box set, is recorded as Among them, b (i) is the i-th target bounding box, i.e. b (i) =[x (i) ,y (i) ,h (i) , w (i) ], c i represents the confidence of the category to which the i-th target bounding box corresponds, and N is the number of target bounding boxes in the bounding box set. By traversing all target bounding boxes in the bounding box set, according to the above-mentioned classification criteria for target bounding boxes and their corresponding targets, a small target set X consisting of multiple small-scale target bounding boxes can be screened out from the bounding box set output by the trained detection model. small As the current level detection result of small-scale targets, that is, in, is the i-th small-scale target bounding box, i.e. c i It represents the confidence of the category to which the i-th small-scale target bounding box corresponds to the small-scale target. and They represent the central coordinates of the i-th small-scale target frame, i.e., the horizontal axis coordinate and the vertical axis coordinate, respectively. and Respectively represent the height and width of the i-th small-scale target border, N s is the small target set X small The number of small and medium-scale object bounding boxes.

[0039] Step S10 3: Slice the image to be detected to obtain a slice set and covers the small target set X small The smallest convex set C of .

[0040] As an embodiment of the present application, a slice set is obtained by slicing the image to be detected. and covers the small target set X small The minimum convex set C of can be realized through steps S1031 to S1033, which are described in detail as follows:

[0041] Step S1031: Traverse the small target set Xsmall All small-scale object bounding boxes in .

[0042] Small target set X small That is, the current level detection result obtained in the aforementioned step S102 includes Traverse the small target set X small All small-scale target borders in N are traversed s A small-scale object bounding box.

[0043] Step S1032: For any small-scale target frame traversed, slice the image to be detected according to the preset size with its central coordinate as the center to obtain a slice set Among them, N slice For a slice collection The number of slices in .

[0044] In the field of pattern recognition, the target detection network usually sets the input size of the image to be detected. The original image larger than this size will be resized through the Resize operation. This operation makes the size of small-scale targets smaller. However, when the target is too small, the loss of scale information caused by the downsampling of the neural network inevitably affects the detection of small-scale targets by the target detection network. In response to the above problems, the present application has changed the idea, that is, it does not change the size of the image to be detected, but for any small-scale target bounding box traversed, it takes its center coordinates as the center and slices the image to be detected according to the preset size. For example, for the i-th small-scale target bounding box by As the center, As the coordinates of the upper left vertex of the jth slice, Slice the image to be detected using the coordinates of the upper right vertex of the jth slice to obtain the slice ( i ) ,remember in, and are the horizontal and vertical coordinates of the center of the j-th slice, respectively. and are the height and width of the jth slice, respectively. When slicing, if the horizontal and / or vertical coordinates of the center of the slice exceed the original image size H I ×W I The boundary formed by the image to be detected is padded with 0. slice slices, this N slice The collection of slices is called a slice collection. Among them, N slice For a slice collection The number of slices in .

[0045] It should be noted that although the image to be detected is sliced ​​with the center coordinates of any small-scale target bounding box as the center, since one slice may cover several small-scale targets, the slice set The number of slices N slice Not necessarily with the small target set X small The number of all small-scale object bounding boxes in N s is equal, which is also the case for the i-th small-scale target bounding box by Slice the image to be detected as the center, and the resulting slice (j) The superscript of is not marked with i but with .

[0046] Step S1033: According to the small target set X small The center coordinates and size of each small-scale target frame in the calculation of the coverage small target set X small The smallest convex set C of .

[0047] Specifically, according to the small target set X small The center coordinates and size of each small-scale target frame in the calculation of the coverage small target set X small The minimum convex set C can be: calculating the difference between the horizontal axis coordinate and the half width of the center of each small-scale target frame in multiple small-scale target frames, that is, the first difference, and the difference between the vertical axis coordinate and the half height, that is, the second difference, and taking the minimum value of the first difference and the minimum value of the second difference as the first horizontal axis coordinate and the first vertical axis coordinate of the minimum convex set C; calculating the sum of the horizontal axis coordinate and the half width of the center of each small-scale target frame in multiple small-scale target frames, that is, the first sum, and the sum of the vertical axis coordinate and the half height, that is, the second sum, and taking the minimum value of the first sum and the minimum value of the second sum as the second horizontal axis coordinate and the second vertical axis coordinate of the minimum convex set C.

[0048] If cx1 represents the first horizontal axis coordinate of the minimum convex set C, cy1 represents the first vertical axis coordinate of the minimum convex set C, cx2 represents the second horizontal axis coordinate of the minimum convex set C, and cy2 represents the second vertical axis coordinate of the minimum convex set C, that is, C = (cx1, cy1, cx2, cy2), then according to the small target set X small The center coordinates and size of each small-scale target frame in the calculation of the coverage small target set X small The minimum convex set C can be realized by the following formula:

[0049]

[0050] Among them, min() is the minimum function, which means taking the minimum value of the object in ().

[0051] Step S104: Determine whether all small-scale targets are included in the slice set obtained when slicing the image to be detected Inside.

[0052] By judging whether all small-scale targets are included in the minimum slice set obtained when slicing the image to be detected The goal is to obtain a minimum slice set that can cover all small-scale targets. Theoretically, the number of slices in the minimum slice set should be less than the number of slices in the slice set. This is beneficial to the requirement of computing resources for the subsequent processing during cascade detection or can save computing resources for the subsequent processing. As an embodiment of the present application, it is determined whether all small-scale targets are included in the minimum set of slices obtained when slicing the image to be detected. The process can be implemented through steps S1 to S4, which are described in detail as follows:

[0053] Step S1: Calculate the slice set The intersection and union ratio of the slices in the subset of and the minimum convex set C, where the slice set A subset of which is updated by the slice collection The resulting slice collection.

[0054] As the computation progresses, the slice collection The number of slices in is changing, so the slice set In addition to the initial slice set, the subset of In addition, it is also possible to update the slice collection as the calculation process progresses The resulting slice collection.

[0055] Step S2: Select a slice set The slice with the largest intersection ratio with the minimum convex set C in the subset of is selected as the slice set S select Add the new elements to the selected slice set S select To update the selected slice set S select , and select the slice set S select Add elements from the slice collection Culling to update the slice collection

[0056] It should be noted that, in the embodiment of the present application, the slice set S is selected select The elements come from the slice collection In other words, whenever the dynamic slice set is calculated by step S1, The slice with the largest intersection ratio between the slice and the minimum convex set C will be selected and added to the selected slice set S.select , at the same time, remove the slice from the slice collection Culling to update the slice collection Get a slice collection For example, suppose that before the current computation process begins, the slice collection Select the slice set S select ={slice (sel-1) , slice (sel-2) ,slice (sel-3)}, after calculation in step S1, slice (1) The intersection ratio with the minimum convex set C is the largest, then slice (1) Add to selected slice set S select To update the selected slice set S select (Remember that the selected slice set after updating is S′ elect ), namely S′ elect ={slice (sel-1) , slice (sel-2) , slice (sel-3) ,slice (1)}; At the same time, slice slice (1) From a slice collection Eliminate from the slice collection Updated to {slice (2) , slice (3) , slice (4) , slice (5)}. Obviously, {slice (2) , slice (3) , slice (4) , slice (5)}for A subset of .

[0057] Step S3: Calculate the set S of slices that cover the selected slices select The minimum convex set C* of the selected slice set after updating.

[0058] As an embodiment of the present application, the calculation covers the selected slice set S selectThe minimum convex set C* of the selected slice set after update can be: calculate the difference between the horizontal axis coordinate and the half width of the center of each slice in the selected slice set after update, that is, the first difference, and the difference between the vertical axis coordinate and the half height, that is, the second difference, and take the minimum value of the first difference and the minimum value of the second difference as the first horizontal axis coordinate and the first vertical axis coordinate of the minimum convex set C*; calculate the sum of the horizontal axis coordinate and the half width of the center of each slice in the selected slice set after update, that is, the first sum, and the sum of the vertical axis coordinate and the half height, that is, the second sum, and take the minimum value of the first sum and the minimum value of the second sum as the second horizontal axis coordinate and the second vertical axis coordinate of the minimum convex set G*.

[0059] If represents the first horizontal axis coordinate of the minimum convex set C, represents the first vertical axis coordinate of the minimum convex set C, represents the second horizontal axis coordinate of the minimum convex set C, Represents the second vertical axis coordinate of the minimum convex set C, that is, C = cx1, cy1, cx2, cy2), then calculate the coverage of the selected slice set S select The minimum convex set C* of the selected slice set after update can be realized by the following formula:

[0060]

[0061]

[0062] in, and They represent the horizontal and vertical coordinates of the center of the jth selected slice in the updated selected slice set, respectively. and Respectively represent the width and height of the th selected slice in the updated selected slice set. The slice with the largest intersection-and-joint ratio between the slice in the subset of the selected slice set and the minimum convex set is added to the selected slice set as the new element of the selected slice set to update the selected slice set, and the new elements of the dynamic selected slice set are removed from the slice set to update the slice set.

[0063] Step S4: If Then make sure that all small-scale targets are included in the minimum slice set and the selected slice set S′ elect Determine the minimum slice set, otherwise, the process turns to calculating the slice set The minimum slice set is obtained by the intersection and union ratio of the slices in the subset and the remaining area of ​​the image to be detected.

[0064] like Then make sure that all small-scale targets are included in the minimum slice set and the selected slice set S′ elect Determine the minimum slice set Once the minimum slice set is determined, the images in the minimum slice set are sequentially input into the trained detection model to obtain the secondary detection results of small-scale objects. elect That is, the slice set selected in step s1 The slice with the largest intersection ratio with the minimum convex set C in the subset of is selected as the slice set S select Add the new elements to the selected slice set S select Then select the slice set S select The updated selected slice set, that is, according to the example of the above embodiment, S′ elect ={slice (sel -1) , slice (sel-2) , slice (sel-3) , slice (1)}.like Not true, that is The process then turns to step S105 of "calculating the slice set The minimum slice set is obtained by the intersection and union ratio of the slices in the subset and the remaining area of ​​the image to be detected. ”.

[0065] Step S105: If it is determined that all small-scale objects are included in the minimum slice set If the minimum slice set is within the range, the image in the minimum slice set is input into the trained detection model to obtain the secondary detection result of the small-scale target. Otherwise, the slice set is calculated. The minimum slice set is obtained by the intersection and union ratio of the slices in the subset and the remaining area of ​​the image to be detected and input into the trained detection model to obtain the secondary detection result of the small-scale target, where the remaining area of ​​the image to be detected is the area after removing the selected slice from the image to be detected, and the selected slice is the slice set The slice in the subset of whose intersection with the smallest convex set C is greater than the slice with the largest intersection.

[0066] From the above description of step S105, it can be seen that there are two ways to determine or obtain the "minimum slice set". One way is to determine that all small-scale objects are included in the minimum slice set. Internal judgment When , the selected slice set S′ elect Determine the minimum slice set Another approach is to calculate the slice set The minimum slice set is obtained by the intersection and union ratio of the slices in the subset and the remaining area of ​​the image to be detected. As mentioned above, as the calculation process proceeds, the slice set The number of slices in is changing, that is, the selected slice set S select The new element is the slice collection Therefore, the process from step S1 to step S105 in the above embodiment is a cyclic process. Figure 2 The process shown. Regardless of which of the two approaches mentioned above is used to determine or obtain the minimum slice set, it will eventually be input into the trained detection model to obtain the secondary detection results of small-scale targets. It should be noted that the "secondary detection results of small-scale targets" here should be understood in a broad sense, that is, unless the small-scale target has not been cascaded through a first-level detection before the current-level detection result, the result obtained from any cascade detection after the current-level detection result can be regarded as a secondary detection result. This means that the secondary detection result of a small-scale target may be the result obtained after several cascade detections, rather than just the result obtained from a single cascade detection.

[0067] As an embodiment of the present application, by calculating the slice set Obtaining the minimum slice set by the intersection and union ratio of the slices in the subset and the remaining area of ​​the image to be detected can be achieved by steps S'1 to S'3, as detailed below:

[0068] Step S'1: Traverse the slice collection A subset of .

[0069] Step S'2: Calculate the slice set according to the following formula The intersection of the slices in the subset and the remaining area of ​​the image to be detected is the IoU (i) :

[0070]

[0071] Among them, C (i) and c (j) Respectively represent the i-th and j-th cycles (i.e., the i-th and j-th determinations of whether all small-scale targets are included in the slice set obtained when slicing the image to be detected). The convex set where the small-scale target is located, slice (j) Represents the i-th slice slice (i) Any previous slice.

[0072] Step S'3: Select a slice set The slice with the largest intersection-over-combination ratio with the remaining area of ​​the image to be detected in the subset is taken as the selected slice set S select Add the new elements to the selected slice set S select To update the selected slice set S selecc ;

[0073] Repeat the above steps S'1 to S'3 until the complete slice set is traversed After the loop ends, the selected slice set S at the end of the loop select Determine the minimum set of slices.

[0074] Step S106: superimpose the current level detection result of the small-scale target with the secondary level detection result of the small-scale target to obtain the ultimate detection result of the small-scale target.

[0075] Specifically, the current level detection result of the small-scale target is superimposed with the secondary detection result of the small-scale target to obtain the ultimate detection result of the small-scale target. The method can be as follows: through translation and resizing operations, the small-scale target frame corresponding to the secondary detection result of the small-scale target is projected to the small-scale target frame corresponding to the current level detection result of the small-scale target; the current level detection result of the small-scale target is superimposed with the secondary detection result of the small-scale target to obtain a frame sequence list containing a series of fused target frames, such as Figure 3 As shown, the following steps S"1 to S"4 are performed on the frame sequence table:

[0076] Step S"1: sorting the confidence of the fused target borders in the border sequence list according to the confidence level;

[0077] Step S"2: Select the fusion target border with the highest confidence and add it to the final output list, and delete the fusion target border with the highest confidence from the border sequence list;

[0078] Step S"3: Calculate the intersection and union ratio of the fusion target border with the highest confidence and other fusion target borders in the border sequence list;

[0079] Step S"4: Delete the fusion target borders whose intersection and union ratio is greater than the threshold in the border sequence table;

[0080] Repeat the process of step S"1 to step S"4 until the frame sequence list is empty, and use the fused target frame in the final output list as the ultimate detection result of the small-scale target.

[0081] It should be noted that in the above steps S1 to S4, the confidence of the fused target frame is the frame set mentioned in the above embodiment. Recorded target border b (i) The confidence level c i .

[0082] From the above attached Figure 1 The small-scale target detection method of the example shows that the image to be detected is input into the trained detection model to obtain the current level detection result of the small-scale target. If it is determined that all small-scale targets are included in the minimum slice set Within, the image in the minimum slice set is input into the trained detection model to obtain the secondary detection result of the small-scale target, and the current level detection result of the small-scale target is superimposed with the secondary detection result of the small-scale target to obtain the ultimate detection result of the small-scale target. On the one hand, unlike the existing technologies such as feature pyramid, which are only cascaded on the network structure when detecting small-scale targets, the technical solution of the present application is centered on the small-scale target, and the image to be detected is sliced ​​to reduce the scale variance of the small-scale target in the image set to be detected, that is, by cascading the slices of the area where the small-scale target is located and performing cascade detection, the scale of the small-scale target in the slice is improved, thereby improving the detection effect of the trained detection model on the small-scale target; on the other hand, the image to be detected is input into the trained detection model to obtain the current level detection result of the small-scale target, which can be used in conjunction with a fast large-scale detection method as the first level detection to ensure the efficiency of the cascade detection.

[0083] Please see attached Figure 4 , is a small-scale target detection device provided in an embodiment of the present application, which may include a training module 401, a first detection module 402, an acquisition module 403, a judgment module 404, a second detection module 405 and a fusion module 406, which are described in detail as follows:

[0084] A training module 401 is used to train the depth detection model to obtain a trained detection model;

[0085] The first detection module 402 is used to input the image to be detected into the trained detection model to obtain the current level detection result of the small-scale target, wherein the current level detection result includes a small target set X composed of multiple small-scale target frames. small ;

[0086] The acquisition module 403 is used to obtain a slice set by slicing the image to be detected. and covers the small target set X small The smallest convex set C of

[0087] The judgment module 404 is used to judge whether all small-scale targets are included in the minimum slice set obtained when slicing the image to be detected. Inside;

[0088] The second detection module 405 is used to determine that all small-scale objects are included in the minimum slice. If The image in is input into the trained detection model to obtain the secondary detection results of small-scale targets. Otherwise, the slice set is calculated The minimum slice set is obtained by the intersection and union ratio of the slices in the subset and the remaining area of ​​the image to be detected and input into the trained detection model to obtain the secondary detection result of the small-scale target, where the remaining area of ​​the image to be detected is the area after removing the selected slice from the image to be detected, and the selected slice is the slice set The slice in the subset of whose intersection with the minimum convex set C is greater than the slice with the largest intersection;

[0089] The fusion module 406 is used to superimpose the current level detection result of the small-scale target with the secondary level detection result of the small-scale target to obtain the final detection result of the small-scale target.

[0090] From the above attached Figure 4 It can be seen from the example of the small-scale target detection device that the image to be detected is input into the trained detection model to obtain the current level detection result of the small-scale target. If it is determined that all small-scale targets are included in the minimum slice set Within, the image in the minimum slice set is input into the trained detection model to obtain the secondary detection result of the small-scale target, and the current level detection result of the small-scale target is superimposed with the secondary detection result of the small-scale target to obtain the ultimate detection result of the small-scale target. On the one hand, unlike the existing technologies such as feature pyramid, which are only cascaded on the network structure when detecting small-scale targets, the technical solution of the present application is centered on the small-scale target, and the image to be detected is sliced ​​to reduce the scale variance of the small-scale target in the image set to be detected, that is, by cascading the slices of the area where the small-scale target is located and performing cascade detection, the scale of the small-scale target in the slice is improved, thereby improving the detection effect of the trained detection model on the small-scale target; on the other hand, the image to be detected is input into the trained detection model to obtain the current level detection result of the small-scale target, which can be used in conjunction with a fast large-scale detection method as the first level detection to ensure the efficiency of the cascade detection.

[0091] Figure 5 Schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 5 As shown, the electronic device 5 of this embodiment mainly includes: a processor 50, a memory 51, and a computer program 52 stored in the memory 51 and executable on the processor 50, such as a program of a small-scale target detection method. When the processor 50 executes the computer program 52, the steps in the above-mentioned small-scale target detection method embodiment are implemented, such as Figure 1 Alternatively, when the processor 50 executes the computer program 52, the functions of each module / unit in the above-mentioned device embodiments are implemented, for example Figure 4 The functions of the training module 401, the first detection module 402, the acquisition module 403, the judgment module 404, the second detection module 405 and the fusion module 406 are shown.

[0092] Exemplarily, the computer program 52 of the small-scale target detection method mainly includes: training a depth detection model to obtain a trained detection model; inputting an image to be detected into the trained detection model to obtain a current-level detection result of the small-scale target, wherein the current-level detection result includes a small target set X composed of multiple small-scale target bounding boxes. small ; Obtain a slice set by slicing the image to be detected and covers the small target set X small The smallest convex set C of the image to be detected; determine whether all small-scale targets are included in the minimum slice set obtained when slicing the image to be detected If it is determined that all small-scale targets are included in the minimum slice set If The image in is input into the trained detection model to obtain the secondary detection results of small-scale targets. Otherwise, the slice set is calculated The minimum slice set is obtained by the intersection and union ratio of the slices in the subset and the remaining area of ​​the image to be detected and input into the trained detection model to obtain the secondary detection result of the small-scale target, where the remaining area of ​​the image to be detected is the area after removing the selected slice from the image to be detected, and the selected slice is the slice set The slice with the largest intersection-and-union ratio of the slice in the subset of the minimum convex set C; the current level detection result of the small-scale target is superimposed with the secondary detection result of the small-scale target to obtain the ultimate detection result of the small-scale target. The computer program 52 can be divided into one or more modules / units, one or more modules / units are stored in the memory 51, and are executed by the processor 50 to complete the present application. One or more modules / units can be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the computer program 52 in the electronic device 5. For example, the computer program 52 can be divided into the functions of the training module 401, the first detection module 402, the acquisition module 403, the judgment module 404, the second detection module 405 and the fusion module 406 (modules in the virtual device), and the specific functions of each module are as follows: the training module 401 is used to train the depth detection model to obtain the trained detection model; the first detection module 402 is used to input the image to be detected into the trained detection model to obtain the current level detection result of the small-scale target, wherein the current level detection result includes a small target set X composed of multiple small-scale target borders. small ; Acquisition module 403, used to obtain a slice set by slicing the image to be detected and covers the small target set X small The minimum convex set C of the image to be detected; the judgment module 404 is used to judge whether all small-scale targets are included in the minimum slice set obtained when slicing the image to be detected. The second detection module 405 is used to determine if all small-scale targets are included in the minimum slice set If the minimum slice set is within the range, the image in the minimum slice set is input into the trained detection model to obtain the secondary detection result of the small-scale target. Otherwise, the slice set is calculated. The minimum slice set is obtained by the intersection and union ratio of the slices in the subset and the remaining area of ​​the image to be detected and input into the trained detection model to obtain the secondary detection result of the small-scale target, where the remaining area of ​​the image to be detected is the area after removing the selected slice from the image to be detected, and the selected slice is the slice set The slice in the subset of which the intersection-and-union ratio with the minimum convex set C is the largest; a fusion module 406 is used to superimpose the current level detection result of the small-scale target with the secondary detection result of the small-scale target to obtain the ultimate detection result of the small-scale target.

[0093] The electronic device 5 may include but is not limited to a processor 50 and a memory 51. Those skilled in the art will appreciate that Figure 5 It is only an example of the electronic device 5 and does not constitute a limitation of the electronic device 5. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0094] The processor 50 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0095] The memory 51 may be an internal storage unit of the electronic device 5, such as a hard disk or memory of the electronic device 5. The memory 51 may also be an external storage device of the electronic device 5, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 5. Further, the memory 51 may also include both an internal storage unit of the electronic device 5 and an external storage device. The memory 51 is used to store computer programs and other programs and data required by the electronic device. The memory 51 may also be used to temporarily store data that has been output or is to be output.

[0096] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0097] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0098] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0099] In the embodiments provided in the present application, it should be understood that the disclosed devices / equipment and methods can be implemented in other ways. For example, the device / equipment embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0100] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0101] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0102] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program of the small-scale target detection method can be stored in a storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments, that is, training the depth detection model to obtain a trained detection model; inputting the image to be detected into the trained detection model to obtain the current level detection result of the small-scale target, wherein the current level detection result includes a small target set X composed of multiple small-scale target borders. small ; Obtain a slice set by slicing the image to be detected and covers the small target set X small The smallest convex set C of the image to be detected; determine whether all small-scale targets are included in the minimum slice set obtained when slicing the image to be detected If it is determined that all small-scale targets are included in the minimum slice set If The image in is input into the trained detection model to obtain the secondary detection results of small-scale targets. Otherwise, the slice set is calculated The minimum slice set is obtained by the intersection and union ratio of the slices in the subset and the remaining area of ​​the image to be detected and input into the trained detection model to obtain the secondary detection result of the small-scale target, where the remaining area of ​​the image to be detected is the area after removing the selected slice from the image to be detected, and the selected slice is the slice set The slice in the subset of the maximum intersection-and-union ratio with the minimum convex set C is selected; the current level detection result of the small-scale target is superimposed with the secondary detection result of the small-scale target to obtain the ultimate detection result of the small-scale target. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The storage medium may include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the storage medium does not include electric carrier signal and telecommunication signal.

[0103] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application is described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or replace some of the technical features therein by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. The specific implementation methods described above further describe the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only the specific implementation method of the present application, and is not used to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present invention.

Claims

1. A small-scale target detection method, characterized in that: The method comprises: Train the depth detection model to obtain a trained detection model; The image to be detected is input into the trained detection model to obtain the current level detection result of the small-scale target, wherein the current level detection result includes a small target set X composed of multiple small-scale target frames. small ; By slicing the image to be detected, a slice set is obtained and covers the small target set X small The smallest convex set C of; Determine whether all the small-scale targets are included in the minimum slice set obtained when slicing the image to be detected Inside; If it is determined that all small-scale targets are included in the minimum slice set Within, the minimum slice set The image in is input into the trained detection model to obtain the secondary detection result of the small-scale target, otherwise, by calculating the slice set The minimum slice set is obtained by the intersection and union ratio of the slices in the subset and the remaining area of ​​the image to be detected. The trained detection model is input to obtain the secondary detection result of the small-scale target, the remaining area of ​​the image to be detected is the area after the selected slice is removed from the image to be detected, and the selected slice is the slice set The slice in the subset of with the largest intersection ratio with the minimum convex set C; The current level detection result of the small-scale target is superimposed with the secondary level detection result of the small-scale target to obtain the ultimate detection result of the small-scale target.

2. The small-scale target detection method according to claim 1, characterized in that: By slicing the image to be detected, the small target set X covering the image is obtained. small The smallest convex set C of includes: Traverse the small target set X small All small-scale object bounding boxes in; For any small-scale target frame traversed, slice the image to be detected according to the preset size with its central coordinate as the center to obtain a slice set The N slice For the slice set The number of slices in the According to the small target set X small The center coordinates and size of each small-scale target frame in the calculation covers the small target set X small The smallest convex set C of .

3. The small-scale target detection method according to claim 2, characterized in that: According to the small target set X small The center coordinates and size of each small-scale target frame in the calculation covers the small target set X small The smallest convex set C of includes: Calculate the difference between the horizontal axis coordinate and the half width of the center of each small-scale object frame in the multiple small-scale object frames, that is, the first difference, and the difference between the vertical axis coordinate and the half height, that is, the second difference, and take the minimum value of the first difference and the minimum value of the second difference as the first horizontal axis coordinate and the first vertical axis coordinate of the minimum convex set C respectively; Calculate the sum of the horizontal axis coordinate and its half-width, i.e., the first sum, and the sum of the vertical axis coordinate and its half-height, i.e., the second sum, of the center of each small-scale object bounding box in the multiple small-scale object bounding boxes, and take the minimum value of the first sum and the minimum value of the second sum as the second horizontal axis coordinate and the second vertical axis coordinate of the minimum convex set C, respectively.

4. The small-scale target detection method according to claim 1, characterized in that: Determining whether all the small-scale targets are included in the minimum slice set obtained when slicing the image to be detected The method comprises steps S1 to S4: Step S1: Calculate the slice set The intersection and union ratio of the slice in the subset of and the minimum convex set C, the slice set The subset of the slice collection is updated by The resulting slice set; Step S2: Select the slice set The slice with the largest intersection ratio with the minimum convex set C in the subset is taken as the selected slice set S select Add the new elements to the selected slice set S select To update the selected slice set S select , and the selected slice set S select Add elements from the slice collection culled to update the slice set Step S3: Calculate the coverage of the selected slice set S select The minimum convex set C* of the selected slice set after updating; Step S4: If Then determine that all small-scale targets are included in the minimum slice set and the selected slice set S′ elect Determine the minimum slice set Otherwise, the process turns to computing the slice set The minimum slice set is obtained by the intersection and union ratio of the slices in the subset and the remaining area of ​​the image to be detected.

5. The small-scale target detection method according to claim 4, characterized in that: The calculation covers the selected slice set S select The minimum convex set C of the selected slice set after update * ,include: Calculate the difference between the horizontal axis coordinate and the half width of the center of each slice in the updated selected slice set, i.e., the first difference, and the difference between the vertical axis coordinate and the half height, i.e., the second difference, and take the minimum value of the first difference and the minimum value of the second difference as the minimum convex set G respectively. * The first horizontal axis coordinate and the first vertical axis coordinate; Calculate the sum of the horizontal axis coordinate and its half width, i.e., the first sum, and the sum of the vertical axis coordinate and its half height, i.e., the second sum, of the center of each slice in the updated selected slice set, and take the minimum of the first sum and the minimum of the second sum as the minimum convex set G. * The second horizontal axis coordinate and the second vertical axis coordinate.

6. The small-scale target detection method according to claim 1, characterized in that: The slice set is calculated by The minimum slice set is obtained by the intersection and union ratio of the slices in the subset and the remaining area of ​​the image to be detected, including steps S'1 to S'3: Step S'1: Traverse the slice set A subset of Step S'2: Calculate the slice set according to the following formula The intersection of the slices in the subset and the remaining area of ​​the image to be detected is IoU (i) : The C (i) and C (j) Respectively represent the convex sets of the small-scale targets in the i-th and j-th cycles; Step S'3: Select the slice set The slice with the largest intersection-and-union ratio with the remaining area of ​​the image to be detected in the subset is taken as the selected slice set S select Add the new elements to the selected slice set S select To update the selected slice set S select ; Repeat the above steps S'1 to S'3 until the entire slice set is traversed. After the loop ends, the selected slice set S at the end of the loop select Determine the minimum slice set 7. The small-scale target detection method according to claim 1, characterized in that: The superimposing the current level detection result of the small-scale target with the secondary level detection result of the small-scale target to obtain the ultimate detection result of the small-scale target includes: Projecting the small-scale object frame corresponding to the secondary detection result of the small-scale object to the small-scale object frame corresponding to the current-level detection result of the small-scale object through translation and resizing operations; The current level detection result of the small-scale target is superimposed with the secondary detection result of the small-scale target to obtain a frame sequence list containing a series of fused target frames, and the following steps S"1 to S"4 are performed on the frame sequence list: Step S"1: sorting the confidences of the fused target borders in the border sequence table according to the confidences; Step S"2: Select the fusion target border with the highest confidence and add it to the final output list, and delete the fusion target border with the highest confidence from the border sequence list; Step S"3: Calculate the intersection and union ratio of the fusion target border with the highest confidence and other fusion target borders in the border sequence list; Step S"4: deleting the fusion target borders whose intersection and union ratio is greater than the threshold in the border sequence table; Repeat the process of step S"1 to step S"4 until the frame sequence list is empty, and use the fused target frame in the final output list as the ultimate detection result of the small-scale target.

8. A small-scale target detection device, characterized in that: The device comprises: A training module, used to train the deep detection model to obtain a trained detection model; The first detection module is used to input the image to be detected into the trained detection model to obtain the current level detection result of the small-scale target, and the current level detection result includes a small target set X composed of multiple small-scale target frames. small ; The acquisition module is used to obtain a slice set by slicing the image to be detected. and covers the small target set X small The smallest convex set C of; A judgment module is used to judge whether all the small-scale targets are included in the minimum slice set obtained when slicing the image to be detected. Inside; The second detection module is used to determine that all small-scale targets are included in the minimum slice set. If the image in the minimum slice set is within , the image in the minimum slice set is input into the trained detection model to obtain the secondary detection result of the small-scale target. Otherwise, the image in the minimum slice set is input into the trained detection model to obtain the secondary detection result of the small-scale target. The minimum slice set is obtained by the intersection and union ratio of the slices in the subset and the remaining area of ​​the image to be detected and input into the trained detection model to obtain the secondary detection result of the small-scale target, the remaining area of ​​the image to be detected is the area after removing the selected slice from the image to be detected, and the selected slice is the slice set The slice in the subset of with the largest intersection ratio with the minimum convex set C; A fusion module is used to superimpose the current level detection result of the small-scale target with the secondary level detection result of the small-scale target to obtain the ultimate detection result of the small-scale target.

9. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images