Target detection method and device based on stepwise regression
By applying scaling transformation and cascaded regression to the first target bounding box, the problem of low detection accuracy for small targets is solved, and efficient recognition of small targets is achieved.
Patent Information
- Application Number
- CN202111276366.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-29
- Publication Date
- 2025-12-12
- Estimated Expiration
- 2041-10-29
AI Technical Summary
Existing small target detection methods suffer from poor detection accuracy in deep learning-based target detection tasks, especially in lightweight networks where small target features are easily lost, leading to poor detection results.
By employing a regression approach that scales the first target bounding box down to smaller bounding boxes, a second target bounding box is generated through a scaling transformation. A specific target detection model is then used for feature extraction and offset prediction to achieve accurate detection of small targets.
It greatly improves the accuracy of small target detection and enhances the model's ability to recognize small targets by amplifying the feature representation capability.
Smart Images
Figure CN114119958B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of image processing, and in particular to a target detection method and device based on hierarchical regression. BACKGROUND
[0002] In a target detection task based on deep learning, especially in a face detection task widely used in actual scenes, it is difficult to detect small targets and small faces, and many technical challenges are faced, because the neural network has insufficient learning ability for small target features, so that the small target detection and recognition task has not been greatly improved in recent years.
[0003] At present, small target detection methods mostly use the following ways to improve the precision of small target detection: increasing input, fusing features of deep and shallow layers, attention mechanism, and oversampling small target data. However, because the sample hit is low in the detection task, and the pixel points are few, the features are not obvious, so the small target features are easily lost in the downsampling process, especially in lightweight networks, the existing methods are almost ineffective or make the calculation amount unusable. SUMMARY
[0004] The present application provides a target detection method and device based on hierarchical regression, which solves the defect that the small target features are not obvious in the prior art, resulting in poor detection precision, and adopts a regression method from large to small target frame, greatly improving the precision of small target detection.
[0005] The present application provides a target detection method based on hierarchical regression, comprising: obtaining a to-be-detected picture, the to-be-detected picture comprising a target picture and a target frame corresponding to the target picture, the target frame comprising a first target frame and a second target frame with a pixel point number greater than the first target frame; performing scale transformation on the first target frame according to a preset ratio to obtain a third target frame with a scale size of the second target frame; obtaining a candidate target frame according to the third target frame and the second target frame, and inputting the to-be-detected picture and the candidate target frame into a specific target detection model to obtain a target prediction result output by the specific target detection model; wherein the specific target detection model is trained based on a training picture, a candidate training target frame, and a target detection result corresponding to the training picture; the specific target detection model is used to obtain the target prediction result based on a candidate target feature extracted from the to-be-detected picture based on the candidate target frame, and an offset of the third target frame relative to the first target frame.
[0006] According to the target detection method based on hierarchical regression provided by the application, the specific target detection model comprises: a feature extraction layer, which performs feature extraction on the to-be-detected picture according to the candidate target frame, to obtain candidate target features; a scale transformation prediction layer, which obtains a scale transformation prediction result according to the candidate target frame and the candidate target features, wherein the scale transformation prediction result comprises candidate target features corresponding to the third target frame after the scale transformation and candidate target features corresponding to the second target frame without the scale transformation; a hierarchical regression layer, which obtains a prediction offset of the candidate target features corresponding to the third target frame relative to the to-be-detected target features corresponding to the first target frame according to the scale transformation prediction result and the to-be-detected picture; and a target prediction layer, which obtains a target prediction result corresponding to the first target frame according to the scale transformation prediction result, the candidate target features and the prediction offset.
[0007] According to the target detection method based on hierarchical regression provided by the application, the target prediction result corresponding to the first target frame is obtained according to the scale transformation prediction result, the candidate target features and the prediction offset, and the target prediction result is obtained by performing prediction frame coordinate regression and prediction frame category classification based on the to-be-predicted target features.
[0008] According to the target detection method based on hierarchical regression provided by the application, the initial target features corresponding to the third target frame are obtained according to the scale transformation prediction result and the candidate target features, and the initial target features corresponding to the third target frame are obtained by selecting, from the candidate target features, candidate target features corresponding to the third target frame after the scale transformation according to the scale transformation prediction result.
[0009] According to the target detection method based on stepwise regression provided by the application, the specific target detection model is trained, and the training comprises: acquiring a training picture, the training picture comprising a target training picture and a training target frame corresponding to the target training picture, the training target frame comprising a first training target frame and a second training target frame with a pixel point number greater than the first training target frame; performing scale transformation on the first training target frame to obtain a third training target frame with a scale size of the second training target frame; obtaining a candidate training target frame according to the third training target frame and the second training target frame, and adding a corresponding label to the candidate training target frame according to whether the scale transformation is performed; obtaining an actual offset according to the third training frame and the first training frame; taking the training picture and the candidate training target frame as input data for training, taking a target detection result corresponding to the training picture as a label, and training in a deep learning mode to obtain the specific target detection model for generating a target prediction result corresponding to a first target frame of a to-be-detected picture.
[0010] According to the target detection method based on stepwise regression provided by the application, the training of the specific target detection model further comprises: constructing a loss function according to the actual offset, the training prediction offset, the label added to the candidate training target frame and the scale transformation training result, and determining that the training is completed according to convergence of the loss function.
[0011] The application further provides a target detection device based on stepwise regression, comprising: a data acquisition module configured to acquire a to-be-detected picture, the to-be-detected picture comprising a target picture and a target frame corresponding to the target picture, the target frame comprising a first target frame and a second target frame with a pixel point number greater than the first target frame; a scale transformation module configured to perform scale transformation on the first target frame according to a preset scale to obtain a third target frame with a scale size of the second target frame; and a target detection module configured to input the to-be-detected picture and a candidate target frame into a specific target detection model to obtain a target prediction result output by the specific target detection model, wherein the specific target detection model is trained based on a training picture, a candidate training target frame and a target detection result corresponding to the training picture; and the specific target detection model is configured to obtain the target prediction result based on a candidate target feature corresponding to the third target frame in a candidate target feature extracted from the to-be-detected picture based on the candidate target frame and an offset of the third target frame relative to the first target frame.
[0012] The application further provides an electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the target detection method based on stepwise regression according to any one of the above embodiments when executing the program.
[0013] The application further provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of any of the above target detection methods based on stepwise regression.
[0014] The application further provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of any of the above target detection methods based on stepwise regression.
[0015] The application provides a target detection method and device based on stepwise regression, which performs scale transformation on a first target frame according to the scale size of a second target frame to enlarge the feature expression capability of the first target, and then detects the enlarged first target by using a specific target detection model, and adopts a regression mode from large to small to greatly improve the precision of small target detection. BRIEF DESCRIPTION OF DRAWINGS
[0016] In order to more clearly illustrate the technical solutions of the present application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings described below are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.
[0017] Figure 1 is a flowchart of the target detection method based on stepwise regression provided by the application;
[0018] Figure 2 is a training flowchart of the specific target detection model provided by the application;
[0019] Figure 3 is a structural diagram of the target detection device based on stepwise regression provided by the application;
[0020] Figure 4 is a structural diagram of the training module provided by the application;
[0021] Figure 5 is a structural diagram of the electronic device provided by the application. DETAILED DESCRIPTION
[0022] In order to make the purpose, technical solutions and advantages of the present application more clear, the technical solutions of the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.
[0023] Figure 1 A flowchart of a target detection method based on hierarchical regression is shown, which comprises:
[0024] S11, obtaining a to-be-detected picture, the to-be-detected picture comprising a target picture and a target box corresponding to the target picture, the target box comprising a first target box and a second target box with a pixel point number greater than the first target box;
[0025] S12, performing scale transformation on the first target box according to a preset ratio to obtain a third target box with a scale size of the second target box;
[0026] S13, obtaining a candidate target box according to the third target box and the second target box, and inputting the to-be-detected picture and the candidate target box into a specific target detection model to obtain a target prediction result output by the specific target detection model; wherein the specific target detection model is trained based on a training picture, a candidate training target box and a target detection result corresponding to the training picture; the specific target detection model is used to obtain the target prediction result based on a candidate target feature extracted from the to-be-detected picture by the candidate target box and an offset of the third target box relative to the first target box.
[0027] It should be noted that S1N in the present specification does not represent the order of the target detection method based on hierarchical regression, and the target detection method based on hierarchical regression of the present application will be described in detail below.
[0028] Step S11, obtaining a to-be-detected picture, the to-be-detected picture comprising a target picture and a target box corresponding to the target picture, the target box comprising a first target box and a second target box with a pixel point number greater than the first target box.
[0029] In the present embodiment, the to-be-detected picture is obtained, comprising: obtaining the target picture and the target box obtained in advance according to the target picture; or, obtaining the target picture; and obtaining the target box according to the target picture. It should be noted that the scale of the first target box is smaller than the scale of the second target box. Specifically, the first target box corresponds to a specific target that is difficult to identify, which can be understood as a small target box, and the corresponding pixel points are few and the target is not obvious; the second target box corresponds to other targets that are easy to identify, i.e. a large target box, and the corresponding pixel points are more, so that the corresponding target can be easily identified.
[0030] It should be noted that the target box can be obtained in advance according to the target picture, so that the corresponding target box is obtained when the target picture is obtained, or the target box can be obtained after the target picture is obtained, which is not limited further herein; the method of obtaining the target box according to the target picture can refer to the existing target detection method, or artificial recognition and labeling, which is not limited further herein.
[0031] In addition, the obtained target picture can be a picture required for behavior recognition, scene recognition, identity recognition, or other target recognition. For example, when automatic driving abnormal behavior detection is required, the obtained target picture is derived from a video stream or at least one picture sequence that is captured in real time by a vehicle for a driving position; for another example, when scene recognition is required for an automatic driving vehicle, the obtained target picture is derived from a picture sequence that is obtained in real time by the vehicle for the environment around the vehicle body, and the target picture can be obtained by a radar, a sensor, or a camera of the vehicle body, and the source of the target picture is not limited further herein.
[0032] In step S12, the first target frame is scaled according to a preset scale to obtain a third target frame with a second target frame size.
[0033] In this embodiment, before the first target frame is scaled, the preset scale is determined according to the sizes of the first target frame and the second target frame, so that the first target frame is scaled by using the preset scale subsequently, thereby expanding the size of the first target frame, and the subsequent model can recognize features according to the expanded first target frame, so as to improve the detection accuracy of the model.
[0034] In step S13, a candidate target frame is obtained according to the third target frame and the second target frame, and the target prediction result output by the specific target detection model is obtained by inputting the to-be-detected picture and the candidate target frame into the specific target detection model; wherein the specific target detection model is trained based on the training picture, the candidate training target frame, and the target detection result corresponding to the training picture; the specific target detection model is used to obtain the target prediction result based on the candidate target feature extracted from the to-be-detected picture by the candidate target frame and the offset of the third target frame relative to the first target frame.
[0035] In this embodiment, the specific target detection model includes: a feature extraction layer, which extracts features from the to-be-detected picture according to the candidate target frame to obtain a candidate target feature; a scale transformation prediction layer, which obtains a scale transformation prediction result according to the candidate target frame and the candidate target feature, the scale transformation prediction result including the candidate target feature corresponding to the third target frame after scale transformation and the candidate target feature corresponding to the second target frame without scale transformation; a level regression layer, which obtains a predicted offset of the candidate target feature corresponding to the third target frame relative to the to-be-detected target feature corresponding to the first target frame according to the scale transformation prediction result and the to-be-detected picture; and a target prediction layer, which obtains the target prediction result corresponding to the first target frame according to the scale transformation prediction result, the candidate target feature, and the predicted offset. Specifically:
[0036] Firstly, the feature extraction layer extracts features of the candidate target from the picture to be detected according to the candidate target frame. It should be noted that the features of the candidate target corresponding to the third target frame and the second target frame are extracted from the picture region corresponding to the candidate target frame in the picture to be detected.
[0037] Secondly, since the candidate target features include the candidate target features corresponding to the third target frame and the candidate target features corresponding to the second target frame, in order to extract the candidate target features corresponding to the third target frame from the candidate target features, the scale transformation prediction layer obtains the scale transformation prediction result from the candidate target features corresponding to the third target frame, in other words, the candidate target features are predicted according to the third target frame and the second target frame to obtain the candidate target features corresponding to the third target frame and the candidate target features corresponding to the second target frame.
[0038] Secondly, since the picture to be detected includes the first target frame and the candidate target frame includes the third target frame, the level regression layer can obtain the prediction offset of the candidate target features corresponding to the third target frame relative to the detected target features corresponding to the first target frame according to the scale transformation prediction result and the first target frame. It should be noted that the prediction offset is represented as (Δx, Δy), wherein, The prediction offset represents the offset of the center point coordinates of the third target frame after the scale transformation relative to the center point coordinates of the first target frame.
[0039] Finally, the target prediction layer obtains the target prediction result corresponding to the first target frame according to the scale transformation prediction result, the candidate target features and the prediction offset.
[0040] Further, the target prediction result corresponding to the first target frame is obtained according to the scale transformation prediction result, the candidate target features and the prediction offset, including: obtaining the initial target features corresponding to the third target frame according to the scale transformation prediction result and the candidate target features; obtaining the detected target features corresponding to the first target frame according to the predicted target features and the prediction offset; and obtaining the target prediction result by performing bounding box coordinate regression and bounding box class classification based on the predicted target features. It should be noted that the initial target features corresponding to the third target frame are obtained according to the scale transformation prediction result and the candidate target features, including: selecting the candidate target features corresponding to the third target frame after the scale transformation from the candidate target features according to the scale transformation prediction result to obtain the predicted target features corresponding to the third target frame.
[0041] In an optional embodiment, the to-be-detected picture and the candidate target frame are input into the specific target detection model to obtain a target prediction result output by the specific target detection model, including: inputting the to-be-detected picture and the candidate target frame into a feature extraction layer to obtain candidate target features output by the feature extraction layer; inputting the candidate target frame and the candidate target features into a scale transformation prediction layer to obtain a scale transformation prediction result output by the scale transformation prediction layer; inputting the scale transformation prediction result and the to-be-detected picture into a level regression layer to obtain a predicted offset output by the level regression layer; and inputting the scale transformation prediction result, the candidate target features and the predicted offset into a target prediction layer to obtain a target prediction result corresponding to the first target frame output by the target prediction layer.
[0042] When the feature extraction layer outputs the candidate target features, the candidate target features can be respectively input into the scale transformation prediction layer, the level regression layer and the target prediction layer, and the order of inputting into the target prediction layer and inputting into the scale transformation prediction layer is not limited. It should be noted that, since the input of the level regression layer also depends on the output of the scale transformation prediction layer, the feature extraction layer can input the candidate target features into the level regression layer after the feature extraction layer inputs the candidate target features into the scale transformation prediction layer, that is, before, after or at the same time as the scale transformation prediction layer inputs the scale transformation prediction result into the level regression layer. Similarly, since the input of the target prediction layer depends on the output of the level regression layer, the feature extraction layer can input the candidate target features into the target prediction layer after the feature extraction layer inputs the candidate target features into the level regression layer, that is, before, after or at the same time as the level regression layer inputs the predicted offset into the target prediction layer.
[0043] In an optional embodiment, with reference to Figure 2 The method further includes training the specific target detection model, including:
[0044] S21, acquiring a training picture, the training picture including a target training picture and a training target frame corresponding to the target training picture, the training target frame including a first training target frame and a second training target frame with a pixel point number greater than the first training target frame;
[0045] S22, performing scale transformation on the first training target frame to obtain a third training target frame with a scale size of the second training target frame;
[0046] S23, obtaining a candidate training target frame according to the third training target frame and the second training target frame, and adding a corresponding label to the candidate training target frame according to whether the candidate training target frame is subjected to scale transformation;
[0047] S24, obtaining an actual offset according to the third training frame and the first training frame;
[0048] S25, taking the training picture and the candidate training target frame as input data used for training, taking the target detection result corresponding to the training picture as a label, and training in a deep learning manner to obtain a specific target detection model used for generating a target prediction result corresponding to a first target frame of a to-be-detected picture.
[0049] It should be noted that S2N in the present specification does not represent the order of the target detection method based on hierarchical regression, and the target detection method based on hierarchical regression of the present application will be described in detail below.
[0050] S21, obtaining a training picture, the training picture including a target training picture and a training target frame corresponding to the target training picture, the training target frame including a first training target frame and a second training target frame having a larger number of pixels than the first training target frame.
[0051] In the present embodiment, the training picture is obtained by: obtaining a target training picture and a training target frame obtained in advance according to the target training picture; or obtaining a target training picture; and identifying the target training picture to obtain a training target frame. It should be noted that the scale of the first training target frame is smaller than the scale of the second training target frame. Specifically, the first training target frame corresponds to a specific target that is difficult to identify, which can be understood as a small target frame, and the corresponding pixels are few and the target is not obvious; the second training target frame corresponds to other targets that are easy to identify, i.e., a large target frame, and the corresponding pixels are more, so that the corresponding target can be easily identified.
[0052] It should be noted that the target frame can be obtained in advance according to the target picture, so that the corresponding target frame of the target picture can be obtained when the target picture is obtained, or the target frame can be obtained according to the target picture after the target picture is obtained, which is not limited further herein; the method for obtaining the target frame according to the target picture can refer to the existing target detection method, or artificial recognition and labeling, which is not limited further herein.
[0053] S22, performing scale transformation on the first training target frame to obtain a third training target frame having the scale size of the second training target frame.
[0054] It should be noted that before performing scale transformation on the first training target frame, a preset ratio is determined according to the scales of the first training target frame and the second training target frame, so that the preset ratio is used to perform scale transformation on the first training target frame subsequently, so as to expand the size of the first training target frame, and the characteristics are identified according to the expanded first training target frame by the subsequent model, so as to improve the detection accuracy of the model.
[0055] S23, obtaining a candidate training target frame according to the third training target frame and the second training target frame, and adding a corresponding label to the candidate training target frame according to whether the candidate training target frame is subjected to scale transformation.
[0056] In the embodiment, the third training target frame and the second training target frame have the same scale, and constitute a candidate training target frame; and then, according to whether the scale transformation is performed, a corresponding label is added to the candidate training target frame, that is, a first label is added to the third training target frame, and a second label is added to the second target frame. It should be noted that the first label represents that the scale transformation is performed, and the second label represents that the scale transformation is not performed. The specific representation form of the first label and the second label can be set according to actual use requirements, and is not limited further herein.
[0057] S24, obtaining an actual offset according to the third training frame and the first training frame.
[0058] In the embodiment, the first training target frame is represented as (x1, y1, w1, h1), the second training target frame is represented as (x2, y2, w2, h2), and the corresponding third training target frame is represented as (x1, h1, w2, h2, Δx, Δy), wherein (x i , y i ) represents the center point coordinates of the corresponding i target frame; w i represents the width of the i target frame, and h i represents the height of the i target frame. (Δx, Δy) represents the offset of the center point coordinates of the third training target frame after the scale transformation relative to the center point coordinates of the first target frame.
[0059] It should be noted that, before step S25, the actual offset and the candidate training target frame are added with a corresponding label, so as to facilitate subsequent training process to construct a loss function according to the deviation between the actual offset and a predicted offset, and the actual label and a scale transformation prediction result, and then to facilitate judging whether the model training is ended. As can be seen, the order of step S24 and step S23 is not limited herein, that is, step S24 can be executed before step S23, or step S24 can be executed after step S23, or step S24 can be executed at the same time as step S23.
[0060] In step S25, the training picture and the candidate training target frame are taken as input data for training, the target detection result corresponding to the training picture is taken as a label, a deep learning method is used for training, and a specific target detection model for generating a target prediction result corresponding to the first target frame of the to-be-detected picture is obtained.
[0061] In the embodiment, first, the training picture and the candidate training target frame are input into the feature extraction layer, and the candidate training target feature output by the feature extraction layer is obtained.
[0062] Secondly, the candidate training target frame and the candidate training target feature are input into the scale transformation prediction layer to obtain a scale transformation training result output by the scale transformation prediction layer.
[0063] Secondly, the scale transformation training result and the training picture are input into the hierarchical regression layer to obtain a training predicted offset output by the hierarchical regression layer.
[0064] Finally, the scale transformation training result, the candidate training target feature and the training predicted offset are input into the target prediction layer to obtain a target prediction result corresponding to the first training target frame output by the target prediction layer.
[0065] It should be noted that, in order to facilitate the judgment of whether the training is ended, the training method of the specific target detection model further comprises: constructing a loss function according to the actual offset, the training predicted offset, the adding of the corresponding label to the candidate training target frame and the scale transformation training result, and judging the end of the training according to the convergence of the loss function.
[0066] The loss function is represented as:
[0067] L = L1 + 0.2L2 + L3
[0068] L1 is a category loss, L2 is a detection frame length-width loss, and L3 is a bias loss of a target frame center point.
[0069] The smaller the loss function value is, the higher the model accuracy is. When the loss function calculated converges, the training is ended, and the trained specific target detection model is obtained.
[0070] In summary, the embodiment of the present application enlarges the feature expression ability of the first target by performing scale transformation on the first target frame according to the scale size of the second target frame, and then detects the enlarged first target by using the specific target detection model, and adopts a regression mode from large to small target frame, so as to greatly improve the precision of small target detection.
[0071] The target detection device based on hierarchical regression provided by the present application is described below, and the target detection device based on hierarchical regression described below can be correspondingly referred to the target detection method based on hierarchical regression described above.
[0072] Figure 3 A structural schematic diagram of a target detection device based on hierarchical regression is shown, and the device comprises:
[0073] A data acquisition module 31 acquires a picture to be detected, and the picture to be detected comprises a target picture and a target frame corresponding to the target picture, and the target frame comprises a first target frame and a second target frame with a pixel point number greater than the first target frame.
[0074] The scale transformation module 32 performs scale transformation on the first target frame according to a preset scale to obtain a third target frame with a scale size of the second target frame.
[0075] The target detection module 33 obtains a candidate target frame according to the third target frame and the second target frame, and inputs the to-be-detected picture and the candidate target frame into the specific target detection model to obtain a target prediction result output by the specific target detection model.
[0076] The specific target detection model is trained based on a training picture, a candidate training target frame, and a target detection result corresponding to the training picture.
[0077] The specific target detection model is used to obtain the target prediction result based on a candidate target feature extracted from the to-be-detected picture according to the candidate target frame and an offset of the third target frame relative to the first target frame.
[0078] In the embodiment, the data acquisition module 31 includes a picture acquisition unit configured to acquire a target picture and a target frame obtained in advance according to the target picture, or the data acquisition module 31 includes a picture acquisition unit configured to acquire a target picture and an identification unit configured to identify the target picture to obtain a target frame. It should be noted that the scale of the first target frame is smaller than the scale of the second target frame. Specifically, the first target frame corresponds to a specific target that is difficult to identify, which can be understood as a small target frame, and the corresponding pixel points are few and the target is not obvious; the second target frame corresponds to other targets that are easy to identify, that is, a large target frame, and the corresponding pixel points are more, so that the corresponding target can be easily identified.
[0079] It should be noted that the target frame can be obtained in advance according to the target picture to obtain the corresponding target frame when the target picture is acquired, or the target frame can be obtained according to the target picture after the target picture is acquired, which is not limited further herein. The method of obtaining the target frame according to the target picture can refer to the existing target detection method or use artificial identification labeling, which is not limited further herein.
[0080] The scale transformation module 32 includes a scale adjustment unit configured to perform scale transformation on the first target frame according to a preset scale to obtain a third target frame with a scale size of the second target frame.
[0081] In an optional embodiment, the scale transformation module 32 further includes a scale preset unit configured to determine a preset scale according to the scales of the first target frame and the second target frame, so as to subsequently perform scale transformation on the first target frame by using the preset scale, thereby expanding the size of the first target frame, facilitating subsequent identification of features by the model according to the expanded first target frame, and improving the detection accuracy of the model.
[0082] The target detection module 33 comprises a data input sub-module, a specific target detection model sub-module and a data output sub-module, wherein: the data input sub-module inputs the picture to be detected and the candidate target frame into the specific target detection model sub-module; the specific target model detection sub-module detects according to the input picture to be detected and the candidate target frame to obtain a target prediction result; and the data output sub-module outputs the target prediction result.
[0083] Specifically, the specific target model detection sub-module comprises: a feature extraction unit, which extracts features of the picture to be detected according to the candidate target frame to obtain candidate target features; a scale transformation prediction unit, which obtains a scale transformation prediction result according to the candidate target frame and the candidate target features, the scale transformation prediction result comprising candidate target features corresponding to the third target frame after scale transformation and candidate target features corresponding to the second target frame without scale transformation; a level regression unit, which obtains a prediction offset of the candidate target features corresponding to the third target frame relative to the target features to be detected corresponding to the first target frame according to the scale transformation prediction result and the picture to be detected; and a target prediction unit, which obtains the target prediction result corresponding to the first target frame according to the scale transformation prediction result, the candidate target features and the prediction offset.
[0084] In an optional embodiment, the target prediction unit comprises: a first feature acquisition sub-unit, which obtains initial target features corresponding to the third target frame according to the scale transformation prediction result and the candidate target features; a second feature acquisition sub-unit, which obtains the target features to be detected corresponding to the first target frame according to the target features to be predicted and the prediction offset; and a detection sub-unit, which performs prediction frame coordinate regression and prediction frame category classification based on the target features to be predicted to obtain the target prediction result. It should be noted that the first feature acquisition sub-unit comprises: a feature acquisition son unit, which selects the candidate target features corresponding to the third target frame after scale transformation from the candidate target features according to the scale transformation prediction result to obtain the initial target features corresponding to the third target frame.
[0085] In an optional embodiment, the device further comprises a training module 34 for training the specific target detection model unit.
[0086] Reference Figure 4 The training module comprises:
[0087] The data acquisition unit 41 acquires training pictures, the training pictures comprising target training pictures and training target frames corresponding to the target training pictures, the training target frames comprising first training target frames and second training target frames with a pixel point number greater than the first training target frames;
[0088] The scale transformation unit 42 performs scale transformation on the first training target frame to obtain a third training target frame with a scale size of the second training target frame;
[0089] The label adding unit 43 obtains a candidate training target frame according to the third training target frame and the second training target frame, and adds a corresponding label to the candidate training target frame according to whether the scale transformation is performed;
[0090] The offset amount obtaining unit 44 obtains an actual offset amount according to the third training frame and the first training frame;
[0091] The training unit 45 uses the training picture and the candidate training target frame as input data for training, uses the target detection result corresponding to the training picture as a label, and uses a deep learning manner to train to obtain a specific target detection model for generating a target prediction result corresponding to the first target frame of the to-be-detected picture.
[0092] In the embodiment, the data obtaining unit 41 comprises: a picture obtaining subunit that obtains a target training picture and a training target frame obtained in advance according to the target training picture; or the data obtaining unit 41 comprises: a picture obtaining subunit that obtains a target training picture; and a target recognizing subunit that recognizes the target training picture to obtain a training target frame. It should be noted that the scale of the first training target frame is smaller than the scale of the second training target frame. Specifically, the first training target frame corresponds to a specific target that is difficult to recognize, which can be understood as a small target frame, and the corresponding pixel points are few and the target is not obvious; the second training target frame corresponds to other targets that are easy to recognize, that is, a large target frame, and the corresponding pixel points are more, so that the corresponding target can be easily recognized.
[0093] It should be noted that the target frame can be obtained in advance according to the target picture to obtain the target frame corresponding to the target picture when the target picture is obtained, or the target frame can be obtained according to the target picture after the target picture is obtained, which is not limited further herein. The method of obtaining the target frame according to the target picture can refer to an existing target detection method, or artificial recognition and labeling, which is not limited further herein.
[0094] The scale replacing unit 42 comprises: a scale replacing subunit that performs scale transformation on the first training target frame to obtain a third training target frame having the scale size of the second training target frame.
[0095] In an optional embodiment, the scale replacing unit 42 further comprises: a proportion presetting subunit that determines a preset proportion according to the scales of the first training target frame and the second training target frame, so as to subsequently perform scale transformation on the first training target frame by using the preset proportion, thereby expanding the size of the first training target frame, and facilitating the subsequent model to recognize the features according to the expanded first training target frame, so as to improve the detection accuracy of the model.
[0096] The label adding unit 43 comprises: a target frame obtaining subunit, which obtains a candidate training target frame according to the third training target frame and the second training target frame; and a label adding subunit, which adds a corresponding label to the candidate training target frame according to whether the candidate training target frame has undergone a scale transformation. Specifically, the label adding subunit comprises: a judgment grandson unit, which judges whether the candidate training target frame has undergone a scale transformation; and a label adding grandson unit, which adds a corresponding label to the candidate training target frame according to a judgment result of the judgment grandson unit. For example, if the judgment grandson unit judges that the candidate target frame is the third target frame, the label adding grandson unit adds a first label to the corresponding candidate training target frame; and for another example, if the judgment grandson unit judges that the candidate target frame is the second target frame, the label adding grandson unit adds a second label to the corresponding candidate training target frame. It should be noted that the first label represents that a scale transformation has been undergone, and the second label represents that a scale transformation has not been undergone. The specific representation forms of the first label and the second label can be set according to actual use requirements, and are not further limited here.
[0097] The offset amount obtaining unit 44 comprises a calculation subunit, which calculates an actual offset amount according to the third training frame and the first training frame. In this embodiment, the first training target frame is represented as (x1, y1, w1, h1), the second training target frame is represented as (x2, y2, w2, h2), and the corresponding third training target frame is represented as (x1, h1, w2, h2, Δx, Δy), wherein (x i , y i ) represents the center point coordinates of the corresponding ith target frame; w i represents the width of the ith target frame, and h i represents the height of the ith target frame. (Δx, Δy) represents the offset amount of the center point coordinates of the third training target frame after the scale transformation relative to the center point coordinates of the first target frame.
[0098] The training unit 45 comprises a feature extraction subunit, a scale transformation prediction subunit, a hierarchical regression subunit, and a target prediction subunit. Specifically, the training picture and the candidate training target frame are input into the feature extraction subunit to obtain candidate training target features output by the feature extraction subunit; the candidate training target frame and the candidate training target features are input into the scale transformation prediction subunit to obtain a scale transformation training result output by the scale transformation prediction subunit; the scale transformation training result and the training picture are input into the hierarchical regression subunit to obtain a training prediction offset amount output by the hierarchical regression subunit; and the scale transformation training result, the candidate training target features, and the training prediction offset amount are input into the target prediction subunit to obtain a target prediction result of the corresponding first training target frame output by the target prediction subunit.
[0099] It should be noted that in order to facilitate the judgment of whether to end the training, the training unit 45 also includes a loss function calculation subunit, which constructs a loss function according to the actual offset and the training predicted offset, and the label added to the candidate training target frame and the scale transformation training result, and judges the end of training according to the convergence of the loss function.
[0100] Figure 5 An example of a schematic diagram of the physical structure of an electronic device is shown in Figure 5 As shown, the electronic device can include a processor 51, a communications interface 52, a memory 53, and a communications bus 54, wherein the processor 51, the communications interface 52, and the memory 53 complete mutual communication through the communications bus 54. The processor 51 can call the logical instructions in the memory 53 to execute the target detection method based on the hierarchical regression, which includes: obtaining a to-be-detected picture, the to-be-detected picture including a target picture and a target frame corresponding to the target picture, the target frame including a first target frame and a second target frame with a pixel point number greater than the first target frame; performing scale transformation on the first target frame according to a preset ratio to obtain a third target frame with a scale size of the second target frame; obtaining a candidate target frame according to the third target frame and the second target frame, and inputting the to-be-detected picture and the candidate target frame into a specific target detection model to obtain a target prediction result output by the specific target detection model; wherein the specific target detection model is trained based on a training picture, a candidate training target frame, and a target detection result corresponding to the training picture; and the specific target detection model is used to obtain the target prediction result based on the candidate target feature corresponding to the third target in the candidate target feature extracted from the to-be-detected picture by the candidate target frame, and the offset of the third target frame relative to the first target frame.
[0101] In addition, the logical instructions in the memory 53 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.
[0102] In another aspect, the present application also provides a computer program product, which comprises a computer program, the computer program being stored in a non-transitory computer readable storage medium, and the computer program being executable by a processor to enable the computer to perform the target detection method based on stepwise regression provided by the above method, the method comprising: obtaining a to-be-detected picture, the to-be-detected picture comprising a target picture and a target box corresponding to the target picture, the target box comprising a first target box and a second target box with a number of pixels greater than the first target box; performing scale transformation on the first target box according to a preset scale to obtain a third target box with a scale size of the second target box; obtaining a candidate target box according to the third target box and the second target box, and inputting the to-be-detected picture and the candidate target box into a specific target detection model to obtain a target prediction result output by the specific target detection model; wherein the specific target detection model is trained based on a training picture, a candidate training target box and a target detection result corresponding to the training picture; and the specific target detection model is used to obtain the target prediction result based on a candidate target feature corresponding to the third target in candidate target features extracted from the to-be-detected picture based on the candidate target box, and an offset of the third target box relative to the first target box.
[0103] In another aspect, the present application also provides a non-transitory computer readable storage medium, which stores a computer program, the computer program being executable by a processor to implement the target detection method based on stepwise regression provided by the above method, the method comprising: obtaining a to-be-detected picture, the to-be-detected picture comprising a target picture and a target box corresponding to the target picture, the target box comprising a first target box and a second target box with a number of pixels greater than the first target box; performing scale transformation on the first target box according to a preset scale to obtain a third target box with a scale size of the second target box; obtaining a candidate target box according to the third target box and the second target box, and inputting the to-be-detected picture and the candidate target box into a specific target detection model to obtain a target prediction result output by the specific target detection model; wherein the specific target detection model is trained based on a training picture, a candidate training target box and a target detection result corresponding to the training picture; and the specific target detection model is used to obtain the target prediction result based on a candidate target feature corresponding to the third target in candidate target features extracted from the to-be-detected picture based on the candidate target box, and an offset of the third target box relative to the first target box.
[0104] The device embodiments described above are only schematic, wherein units shown as separate components can or can not be physically separate, and components shown as units can or can not be physical units, i.e., can be located in one place or distributed on a plurality of network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment scheme. Those skilled in the art can understand and implement without creative labor.
[0105] Those skilled in the art can clearly understand the implementation of the various embodiments by means of software and necessary general hardware platforms through the above description of the embodiments, and of course, the embodiments can also be implemented by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, and the computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of the various embodiments or some parts of the embodiments.
[0106] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for some technical features thereof; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present application.
Claims
1. A stepwise regression based target detection method characterized by, The method comprises: obtaining a to-be-detected picture, wherein the to-be-detected picture comprises a target picture and a target box corresponding to the target picture, and the target box comprises a first target box and a second target box with a number of pixels greater than the first target box; performing scale transformation on the first target box according to a preset ratio to obtain a third target box with a scale size of the second target box; obtaining a candidate target box according to the third target box and the second target box, and inputting the to-be-detected picture and the candidate target box into a specific target detection model to obtain a target prediction result output by the specific target detection model; wherein the specific target detection model is trained based on a training picture, a candidate training target box and a target detection result corresponding to the training picture; the specific target detection model is used to obtain a target prediction result based on a candidate target feature extracted from the to-be-detected picture based on the candidate target box and an offset of the third target box relative to the first target box; the specific target detection model comprises: a feature extraction layer, which extracts a candidate target feature from the to-be-detected picture based on the candidate target box; a scale transformation prediction layer, which obtains a scale transformation prediction result based on the candidate target box and the candidate target feature, wherein the scale transformation prediction result comprises a candidate target feature corresponding to the third target box after the scale transformation and a candidate target feature corresponding to the second target box without the scale transformation; a hierarchical regression layer, which obtains a predicted offset of a candidate target feature corresponding to the third target box relative to a to-be-detected target feature corresponding to the first target box based on the scale transformation prediction result and the to-be-detected picture; a target prediction layer, which obtains a target prediction result corresponding to the first target box based on the scale transformation prediction result, the candidate target feature and the predicted offset.
2. The level-based regression based object detection method of claim 1, wherein, the target prediction layer comprises: an initial target feature corresponding to the third target box is obtained based on the scale transformation prediction result and the candidate target feature; a to-be-detected target feature corresponding to the first target box is obtained based on the to-be-predicted target feature and the predicted offset; a prediction box coordinate regression and a prediction box category classification are performed based on the to-be-predicted target feature to obtain a target prediction result.
3. The level-based regression based object detection method of claim 2, wherein, the initial target feature corresponding to the third target box is obtained based on the scale transformation prediction result and the candidate target feature, and comprises: the initial target feature corresponding to the third target box is obtained by selecting, from the candidate target feature, a candidate target feature corresponding to the third target box after the scale transformation based on the scale transformation prediction result.
4. The level-based regression based object detection method of claim 1, wherein, training the specific target detection model comprises: obtaining a training picture, wherein the training picture comprises a target training picture and a training target box corresponding to the target training picture, and the training target box comprises a first training target box and a second training target box with a number of pixels greater than the first training target box; performing a scale transformation on the first training target frame to obtain a third training target frame having a scale size of the second training target frame; obtaining a candidate training target frame according to the third training target frame and the second training target frame, and adding a corresponding label to the candidate training target frame according to whether the candidate training target frame has undergone the scale transformation; obtaining an actual offset according to the third training frame and the first training frame; using the training picture and the candidate training target frame as input data for training, using a target detection result corresponding to the training picture as a label, and using a deep learning method to train the specific target detection model to obtain a target prediction result corresponding to a first target frame of a to-be-detected picture.
5. The level-based regression based target detection method of claim 4, wherein, The training of the specific target detection model further includes: constructing a loss function according to the actual offset, the training prediction offset, the label added to the candidate training target frame, and the scale transformation training result, and determining that the training is complete according to convergence of the loss function. 6.A target detection device based on hierarchical regression, characterized in that, a data acquisition module acquires a to-be-detected picture, wherein the to-be-detected picture includes a target picture and a target frame corresponding to the target picture, and the target frame includes a first target frame and a second target frame having a larger number of pixels than the first target frame; a scale transformation module performs a scale transformation on the first target frame according to a preset scale to obtain a third target frame having a scale size of the second target frame; a target detection module inputs the to-be-detected picture and the candidate target frame into a specific target detection model to obtain a target prediction result output by the specific target detection model; The specific target detection model is trained based on a training picture, a candidate training target frame, and a target detection result corresponding to the training picture. The specific target detection model is used to obtain a target prediction result based on a candidate target feature corresponding to the third target frame in a candidate target feature extracted from the to-be-detected picture based on the candidate target frame, and an offset of the third target frame relative to the first target frame. The specific target detection model includes: a feature extraction layer that extracts a candidate target feature from the to-be-detected picture based on the candidate target frame; a scale transformation prediction layer that obtains a scale transformation prediction result based on the candidate target frame and the candidate target feature, wherein the scale transformation prediction result includes a candidate target feature corresponding to the third target frame after the scale transformation and a candidate target feature corresponding to the second target frame without the scale transformation; a hierarchical regression layer that obtains a prediction offset of a candidate target feature corresponding to the third target frame relative to a to-be-detected target feature corresponding to the first target frame based on the scale transformation prediction result and the to-be-detected picture; a target prediction layer that obtains a target prediction result corresponding to the first target frame based on the scale transformation prediction result, the candidate target feature, and the prediction offset.
7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the steps of the target detection method based on hierarchical regression according to any one of claims 1 to 5 when executing the program.
8. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, which is executed by a processor, implements the steps of the target detection method based on stepwise regression according to any one of claims 1 to 5.
9. A computer program product comprising a computer program, characterized in that, The computer program, which is executed by a processor, implements the steps of the target detection method based on stepwise regression according to any one of claims 1 to 5.