Target detection model training method and device, equipment and readable storage medium

Abstract

This invention relates to the field of deep learning technology, and more particularly to a method, apparatus, device, and readable storage medium for training an object detection model. The method includes: acquiring a sample dataset of labeled ground truth bounding boxes; constructing an auxiliary training branch for supervising and reinforcing the anchor box regression positions, and adding the auxiliary training branch to a preset object detection model to obtain a target detection model to be trained, wherein the anchor boxes are preset densely sampled boxes in the preset object detection model; and training the target detection model to be trained using the sample dataset to obtain a trained object detection model. This invention, by constructing an auxiliary training branch for supervising and reinforcing the regression branch, ensures that the regression positions of anchor boxes matching the same target do not deviate significantly, thereby effectively improving the regression quality of the object detection network and solving the regression instability problem caused by anchor box switching. This makes the predicted box positions matching the same target tend to be similar, thus improving the stability of the object detection boxes.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and in particular to a method, apparatus, device, and readable storage medium for training an object detection model. Background Technology

[0002] Object detection tasks mainly include detecting the presence of a target of interest, semantic classification of the target, locating the target's position, and determining the spatial extent of the target. During object detection, factors such as noise from the image acquisition equipment, weather, and lighting conditions can affect the accuracy of various perception tasks based on object detection. This can lead to instability in the aspect ratio or center coordinates of the detection bounding box, causing the detection box for the same target to be in different positions across multiple consecutive frames. This instability affects the accuracy of these perception tasks. Summary of the Invention

[0003] The main objective of this invention is to provide a method, apparatus, device, and computer-readable storage medium for training an object detection model, which aims to optimize the object detection model and thereby improve the stability of the object detection bounding box.

[0004] To achieve the above objectives, the present invention provides a method for training an object detection model, the method comprising the following steps:

[0005] Obtain the sample dataset with labeled ground truth bounding boxes;

[0006] An auxiliary training branch is constructed to supervise and enhance the anchor box regression position, and the auxiliary training branch is added to a preset target detection model to obtain a target detection model to be trained, wherein the anchor box is a preset dense sampling box in the preset target detection model;

[0007] The trained target detection model is obtained by training the target detection model using the sample dataset.

[0008] Optionally, the step of training the target detection model to be trained using the sample dataset to obtain the trained target detection model includes:

[0009] Each sample image from the sample dataset is input into the target detection model to be trained, and the classification vector output by the classification branch of the target detection model to be trained and the regression vector output by the regression branch of the target detection model to be trained are obtained.

[0010] The classification vector and the regression vector are input into the model loss function of the target detection model to be trained to obtain the first loss;

[0011] The regression vector is input into the auxiliary training branch to obtain the target prediction box corresponding to the ground truth box, and the offset degree of each target prediction box relative to the reference position is determined. The offset degree is used as the second loss, wherein the coordinate data of the reference position is calculated based on the coordinate data of each target prediction box.

[0012] Based on the gradient descent optimizer and the first and second losses, the parameters of the target detection model to be trained are optimized through backpropagation to obtain the trained target detection model.

[0013] Optionally, the step of inputting the regression vector into the auxiliary training branch to obtain the target prediction box corresponding to the ground truth box includes:

[0014] The regression vector is input into the auxiliary training branch, and the auxiliary training branch combines the regression vector and the anchor box to decode and obtain multiple dense prediction boxes.

[0015] Based on the area intersection-union ratio between the ground truth bounding box and each of the dense prediction bounding boxes, a target prediction bounding box that matches the ground truth bounding box is determined from each of the dense prediction bounding boxes.

[0016] Optionally, the step of determining the target prediction box matching the ground truth box from among the dense prediction boxes based on the area intersection-over-union ratio between the ground truth box and each of the dense prediction boxes includes:

[0017] Calculate the area intersection-union ratio between each of the dense predicted bounding boxes and the ground truth bounding boxes;

[0018] The dense prediction boxes whose area intersection-union ratio with the ground truth box is within a preset range are determined as the target prediction boxes that match the ground truth box.

[0019] Optionally, the step of inputting the regression vector into the auxiliary training branch, and decoding the regression vector and the preset anchor box in the auxiliary training branch to obtain dense prediction boxes includes:

[0020] The regression vector is input into the auxiliary training branch, and the auxiliary training branch combines the regression vector and the preset anchor box to decode and obtain the corner coordinates of multiple dense prediction boxes.

[0021] The coordinates of the first corner point and the second corner point in the coordinates of the first corner point are respectively used as the coordinate data of each of the dense prediction boxes, wherein the first corner point and the second corner point are diagonal points to each other.

[0022] Optionally, the step of inputting the regression vector into the auxiliary training branch, and decoding the regression vector and the preset anchor box in the auxiliary training branch to obtain the corner coordinates of each dense prediction box includes:

[0023] The regression vector is input into the auxiliary training branch, and the auxiliary training branch combines the regression vector and the preset anchor box to decode and obtain the center point coordinates, width and height of each dense prediction box;

[0024] Based on the center point coordinates, width, and height of each of the dense prediction boxes, the corner point coordinates of each of the dense prediction boxes are calculated.

[0025] Optionally, the step of determining the degree of offset of each of the target prediction boxes relative to the reference position includes:

[0026] Calculate the average of the coordinate data of each target prediction box to obtain the coordinate data of the reference position;

[0027] Calculate the variance between the coordinate data of each target prediction box and the coordinate data of the reference position to obtain the degree of offset of each target prediction box relative to the reference position.

[0028] To achieve the above objectives, the present invention also provides a target detection model training device, the target detection model training device comprising:

[0029] The acquisition module is used to acquire a sample dataset of labeled ground truth bounding boxes;

[0030] A construction module is used to construct an auxiliary training branch for supervising and reinforcing the anchor box regression position, and to add the auxiliary training branch to a preset target detection model to obtain a target detection model to be trained, wherein the anchor box is a preset dense sampling box in the preset target detection model;

[0031] The training module is used to train the target detection model to be trained using the sample dataset to obtain the trained target detection model.

[0032] To achieve the above objectives, the present invention also provides an object detection model training device, the object detection model training device comprising: a memory, a processor, and an object detection model training program stored in the memory and executable on the processor, wherein the object detection model training program, when executed by the processor, implements the steps of the object detection model training method as described above.

[0033] Furthermore, to achieve the above objectives, the present invention also proposes a computer-readable storage medium storing a target detection model training program, which, when executed by a processor, implements the steps of the target detection model training method described above.

[0034] In this invention, a sample dataset of labeled ground truth bounding boxes is obtained; an auxiliary training branch is constructed to supervise and strengthen the anchor box regression positions, and this auxiliary training branch is added to a preset object detection model to obtain the object detection model to be trained. Here, the anchor boxes are pre-defined densely sampled boxes in the preset object detection model; the object detection model to be trained is then trained using the sample dataset to obtain the trained object detection model. This invention, by constructing an auxiliary training branch to supervise and strengthen the regression branch, ensures that the regression positions of anchor boxes matching the same target do not deviate significantly, thereby effectively improving the regression quality of the object detection network and solving the regression instability problem caused by anchor box switching. This makes the predicted box positions matching the same target tend to be similar, thus improving the stability of the object detection boxes. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the hardware operating environment involved in the embodiments of the present invention;

[0036] Figure 2 This is a flowchart illustrating the first embodiment of the target detection model training method of the present invention;

[0037] Figure 3 This is a schematic diagram of the structure of a target detection model according to an embodiment of the present invention;

[0038] Figure 4 This is a schematic diagram of the structure of an auxiliary training branch according to an embodiment of the present invention;

[0039] Figure 5 This is a schematic diagram of the functional modules of a preferred embodiment of the target detection model training device of the present invention.

[0040] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0041] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0042] like Figure 1 As shown, Figure 1 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of the present invention.

[0043] It should be noted that the target detection model training device in this embodiment of the invention can be a smartphone, a personal computer, a server, or other devices, and no specific limitations are imposed here.

[0044] like Figure 1 As shown, the target detection model training device may include: a processor 1001, such as a CPU; a network interface 1004; a user interface 1003; a memory 1005; and a communication bus 1002. The communication bus 1002 is used to establish communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or stable non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0045] Those skilled in the art will understand that Figure 1 The device structure shown does not constitute a limitation on the training device for the object detection model. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0046] like Figure 1 As shown, the memory 1005, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an object detection model training program. The operating system is a program that manages and controls the device's hardware and software resources, supporting the operation of the object detection model training program and other software or programs. Figure 1 In the device shown, the user interface 1003 is mainly used for data communication with the client; the network interface 1004 is mainly used for establishing a communication connection with the server; and the processor 1001 can be used to call the target detection model training program stored in the memory 1005 and perform the following operations:

[0047] Obtain the sample dataset with labeled ground truth bounding boxes;

[0048] An auxiliary training branch is constructed to supervise and enhance the anchor box regression position, and the auxiliary training branch is added to a preset target detection model to obtain a target detection model to be trained, wherein the anchor box is a preset dense sampling box in the preset target detection model;

[0049] The trained target detection model is obtained by training the target detection model using the sample dataset.

[0050] Further, the step of training the target detection model to be trained using the sample dataset to obtain the trained target detection model includes:

[0051] Each sample image from the sample dataset is input into the target detection model to be trained, and the classification vector output by the classification branch of the target detection model to be trained and the regression vector output by the regression branch of the target detection model to be trained are obtained.

[0052] The classification vector and the regression vector are input into the model loss function of the target detection model to be trained to obtain the first loss;

[0053] The regression vector is input into the auxiliary training branch to obtain the target prediction box corresponding to the ground truth box, and the offset degree of each target prediction box relative to the reference position is determined. The offset degree is used as the second loss, wherein the coordinate data of the reference position is calculated based on the coordinate data of each target prediction box.

[0054] Based on the gradient descent optimizer and the first and second losses, the parameters of the target detection model to be trained are optimized through backpropagation to obtain the trained target detection model.

[0055] Further, the step of inputting the regression vector into the auxiliary training branch to obtain the target prediction box corresponding to the ground truth box includes:

[0056] The regression vector is input into the auxiliary training branch, and the auxiliary training branch combines the regression vector and the anchor box to decode and obtain multiple dense prediction boxes.

[0057] Based on the area intersection-union ratio between the ground truth bounding box and each of the dense prediction bounding boxes, a target prediction bounding box that matches the ground truth bounding box is determined from each of the dense prediction bounding boxes.

[0058] Further, the step of determining the target prediction box matching the ground truth box from among the dense prediction boxes based on the area intersection-over-union ratio between the ground truth box and each of the dense prediction boxes includes:

[0059] Calculate the area intersection-union ratio between each of the dense predicted bounding boxes and the ground truth bounding boxes;

[0060] The dense prediction boxes whose area intersection-union ratio with the ground truth box is within a preset range are determined as the target prediction boxes that match the ground truth box.

[0061] Further, the step of inputting the regression vector into the auxiliary training branch, and decoding the regression vector and the preset anchor box in the auxiliary training branch to obtain dense prediction boxes includes:

[0062] The regression vector is input into the auxiliary training branch, and the auxiliary training branch combines the regression vector and the preset anchor box to decode and obtain the corner coordinates of multiple dense prediction boxes.

[0063] The coordinates of the first corner point and the second corner point in the coordinates of the first corner point are respectively used as the coordinate data of each of the dense prediction boxes, wherein the first corner point and the second corner point are diagonal points to each other.

[0064] Further, the step of inputting the regression vector into the auxiliary training branch, and decoding the regression vector and the preset anchor box in the auxiliary training branch to obtain the corner coordinates of each dense prediction box includes:

[0065] The regression vector is input into the auxiliary training branch, and the auxiliary training branch combines the regression vector and the preset anchor box to decode and obtain the center point coordinates, width and height of each dense prediction box;

[0066] Based on the center point coordinates, width, and height of each of the dense prediction boxes, the corner point coordinates of each of the dense prediction boxes are calculated.

[0067] Further, the step of determining the degree of offset of each of the target prediction boxes relative to the reference position includes:

[0068] Calculate the average of the coordinate data of each target prediction box to obtain the coordinate data of the reference position;

[0069] Calculate the variance between the coordinate data of each target prediction box and the coordinate data of the reference position to obtain the degree of offset of each target prediction box relative to the reference position.

[0070] Based on the above structure, various embodiments of the object detection model training method are proposed.

[0071] Reference Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the target detection model training method of the present invention.

[0072] This invention provides an embodiment of a method for training an object detection model. It should be noted that although the flowchart shows a logical order, in some cases, the steps shown or described may be executed in a different order. In this embodiment, the executing entity of the object detection model training method can be a personal computer, smartphone, or other device; this is not limited in this embodiment. For ease of description, the execution entity is omitted from the following description of each embodiment. In this embodiment, the object detection model training method includes:

[0073] Step S10: Obtain the sample dataset with labeled ground truth bounding boxes;

[0074] During target detection, factors such as noise from the image acquisition device, weather, and lighting can cause instability in the aspect ratio and center coordinates of the detection bounding box. This can lead to situations where, even if the target's position remains unchanged, the detection bounding box may shift or fail to change smoothly across multiple consecutive frames. In this embodiment, an auxiliary training branch for training the stability of the detection bounding box is added to the target detection model to optimize its structure and improve its stability.

[0075] Specifically, in this embodiment, the images used to train the model are referred to as sample images. A sample dataset is built based on multiple sample images to train the object detection model. Each sample image carries a label, that is, the ground truth box (GT) of the selected target is marked on the sample image. The specific image annotation method is not limited here.

[0076] Furthermore, in one feasible implementation, in order to improve the accuracy of the target detection model, images based on multiple scenes can be collected as sample images. For example, images of different weather scenes such as cloudy, rainy, snowy, and sunny can be collected, as well as scene images at different times. The specific settings can be made according to actual needs, which will not be elaborated here.

[0077] Step S20: Construct an auxiliary training branch for supervising and reinforcing the anchor box regression position, and add the auxiliary training branch to the preset target detection model to obtain the target detection model to be trained, wherein the anchor box is a preset dense sampling box in the preset target detection model;

[0078] In this embodiment, an auxiliary training branch is constructed to regress the regression positions of each anchor box, where the anchor boxes are pre-defined dense sampling boxes in the preset object detection model. The regression method used by this auxiliary training branch is not limited here; for example, it can be regression based on a stability loss function or linear regression.

[0079] The auxiliary training branch is added to a pre-set target detection model, and the resulting target detection model is called the target detection model to be trained. In this specific implementation, the type and structure of the pre-set target detection model are not limited. Specifically, in this embodiment, the auxiliary training branch needs to train the model based on the anchor boxes of the regression branch; therefore, the auxiliary training branch needs to be connected to the output layer of the regression branch.

[0080] Exemplarily, in one feasible implementation, reference is made to Figure 3 It adopts a single-stage object detection network based on anchor points as the base network, such as Figure 3 As shown, the network backbone performs feature extraction, and the extracted features are fused through the feature pyramid to obtain feature layers with a stride of 1 / 8 to 1 / 128. Each feature layer leads to a shared detection head for classification and regression tasks. In this embodiment, the output layers of the auxiliary training branch and the regression branch are connected.

[0081] Step S30: Train the target detection model to be trained using the sample dataset to obtain the trained target detection model.

[0082] In this embodiment, after obtaining the target detection model to be trained with the embedded auxiliary training branch, the target detection model to be trained is trained using the sample dataset to obtain the trained target detection model.

[0083] Furthermore, in one feasible implementation, preprocessing the sample images in the sample dataset increases the number and types of samples in the pseudo-database, improves sample diversity, and thus enhances the generalization ability of the neural network. Specifically, multiple sample images are acquired, and each sample image is preprocessed to obtain a processed image. The preprocessing includes angle rotation, size scaling, brightness adjustment, image denoising, and image augmentation. Ground truth bounding boxes are labeled for target objects in each processed image, and a sample dataset is established based on each labeled processed image.

[0084] Further, in one feasible embodiment, step S30 includes:

[0085] Step S301: Input each sample image of the sample dataset into the target detection model to be trained to obtain the classification vector output by the classification branch of the target detection model to be trained and the regression vector output by the regression branch of the target detection model to be trained.

[0086] In this embodiment, each sample image from the sample dataset is input into the target detection model to be trained, resulting in the classification vector output by the classification branch and the regression vector output by the regression branch. Specifically, the structure of the classification and regression branches is not limited, nor is the process of obtaining the classification and regression vectors.

[0087] Step S302: Input the classification vector and the regression vector into the model loss function of the target detection model to be trained to obtain the first loss;

[0088] In this embodiment, the classification vector and regression vector are input into the model loss function of the target detection model to be trained to obtain the first loss. Specifically, the model loss function is not limited here. For example, in a feasible embodiment, the model loss function can be composed of a classification loss function and a regression loss function. The classification loss function and the regression loss function can be Cross Entropy loss and Smooth L1 loss (a piecewise loss function).

[0089] Step S303: Input the regression vector into the auxiliary training branch to obtain the target prediction box corresponding to the ground truth box, and determine the degree of offset of each target prediction box relative to the reference position. Use the degree of offset as the second loss, wherein the coordinate data of the reference position is calculated based on the coordinate data of each target prediction box.

[0090] The regression vector is input into the auxiliary training branch to obtain the predicted bounding boxes corresponding to the ground truth bounding boxes (hereinafter referred to as target predicted boxes for distinction), and the offset of each target predicted box relative to the reference position is determined, with the offset being used as the second loss. The coordinate data of the reference position is calculated based on the coordinate data of each target predicted box. Specifically, the coordinate data of the reference position can be the mean coordinate data of each anchor box or the median coordinate data of each anchor box; there is no restriction here, and it can be set according to actual needs.

[0091] Step S304: Based on the gradient descent optimizer and the first loss and the second loss, the parameters of the target detection model to be trained are optimized through backpropagation to obtain the trained target detection model.

[0092] In this embodiment, the parameters of the target detection model to be trained are optimized by backpropagation based on the gradient descent optimizer and the first and second losses, so as to obtain the trained target detection model.

[0093] In this embodiment, a sample dataset with labeled ground truth bounding boxes is obtained; an auxiliary training branch is constructed to supervise and enhance the anchor box regression position, and the auxiliary training branch is added to a preset target detection model to obtain a target detection model to be trained, wherein the anchor box is a preset dense sampling box in the preset target detection model; the target detection model to be trained is trained using the sample dataset to obtain a trained target detection model.

[0094] This embodiment constructs an auxiliary training branch for supervising and reinforcing the regression branch, ensuring that the regression positions of anchor boxes matching the same target do not deviate significantly. This effectively improves the regression quality of the target detection network and solves the problem of regression instability caused by anchor box switching, making the predicted box positions of the same target tend to be the same, thereby improving the stability of the target detection boxes.

[0095] Furthermore, based on the first embodiment described above, a second embodiment of the target detection model training method of the present invention is proposed. In this embodiment, step S303 includes:

[0096] Step S3031: Input the regression vector into the auxiliary training branch, and decode the regression vector and anchor boxes in the auxiliary training branch to obtain multiple dense prediction boxes;

[0097] In this embodiment, the regression vector is input into the auxiliary training branch, where it is combined with the anchor boxes to decode and obtain multiple dense prediction boxes. The specific decoding process can refer to the conventional decoding process, and will not be elaborated here.

[0098] Step S3032: Based on the area intersection-union ratio between the ground truth bounding box and each of the dense prediction bounding boxes, determine the target prediction bounding box that matches the ground truth bounding box from each of the dense prediction bounding boxes.

[0099] In this embodiment, the target prediction box that matches the real box is determined from each dense prediction box based on the area intersection-union ratio between the ground truth box and each dense prediction box.

[0100] Furthermore, in one feasible implementation, if the target detection model is a multi-target detection model, that is, each sample image is labeled with multiple ground truth boxes, it is necessary to match the corresponding target prediction box for each ground truth box.

[0101] In this embodiment, regression vectors are input into the auxiliary training branch, where they are combined with anchor boxes to decode and obtain multiple dense prediction boxes. Based on the area intersection-over-union (IoU) ratio between the ground truth boxes and each dense prediction box, the target prediction box matching the ground truth box is determined from each dense prediction box. This embodiment effectively determines the regression position of anchor boxes matching the same target, avoiding interference from the regression positions of non-target-matching anchor boxes in the auxiliary training branch, thus improving the accuracy of the auxiliary training branch and consequently enhancing the accuracy of the target detection model.

[0102] Further, in one feasible embodiment, step S3032 includes:

[0103] Step S30321: Calculate the area intersection-union ratio between each of the dense prediction boxes and the ground truth boxes;

[0104] In this embodiment, the area intersection-union ratio (IUGR) between each dense predicted bounding box and the ground truth bounding box is calculated. The calculation method for the IUGR can refer to the conventional calculation method, and will not be elaborated here.

[0105] Step S30322: The dense prediction boxes whose area intersection-union ratio with the ground truth box is within a preset range are determined as the target prediction boxes that match the ground truth box.

[0106] After calculating the area intersection-union ratio (IUU) of each dense prediction box with the ground truth box, the dense prediction boxes whose IUU with the ground truth box are within a preset range are determined as the target prediction boxes that match the ground truth boxes.

[0107] Further, in one feasible embodiment, step S3031 includes:

[0108] Step S30311: Input the regression vector into the auxiliary training branch, and decode the regression vector and the preset anchor box in the auxiliary training branch to obtain the corner coordinates of each of the multiple dense prediction boxes;

[0109] In this embodiment, the regression vector is input into the auxiliary training branch, and the corner coordinates of multiple dense prediction boxes are obtained by decoding the regression vector and the preset anchor box in the auxiliary training branch.

[0110] Step S30312: The first corner coordinate of the first corner point and the second corner coordinate of the second corner point are respectively used as the coordinate data of each of the dense prediction boxes, wherein the first corner point and the second corner point are diagonal points of each other.

[0111] In this embodiment, the coordinates of a pair of diagonal points are used as the coordinate data of the target prediction box. This embodiment can restrict the regression positions of each anchor box to tend to be the same, while also restricting the size of each anchor box to tend to be the same. This ensures that there is no large deviation between the regression positions of anchor boxes matching the same target, thereby effectively improving the regression quality of the target detection network and solving the problem of regression instability caused by anchor box switching. This makes the positions of the prediction boxes matching the same target tend to be the same, thereby improving the stability of the target detection boxes.

[0112] Specifically, the two corner points in a pair of diagonal points are referred to as the first corner point and the second corner point, respectively. For any dense prediction box in each dense prediction box, the coordinates of the first corner point of the first corner point and the second corner point of the second corner point in the corner coordinates of the dense prediction box are used as the coordinate data of the dense prediction box.

[0113] Further, in one feasible embodiment, step S30311 includes:

[0114] Step S303111: Input the regression vector into the auxiliary training branch, and decode the regression vector and the preset anchor box in the auxiliary training branch to obtain the center point coordinates, width and height of each dense prediction box;

[0115] In this embodiment, the regression vector is input into the auxiliary training branch, and the center point coordinates, width and height of each dense prediction box are obtained by decoding the regression vector and the preset anchor box in the auxiliary training branch.

[0116] Step S303112: Based on the center point coordinates, width, and height of each of the dense prediction boxes, calculate the corner coordinates of each of the dense prediction boxes.

[0117] Based on the center point coordinates, width, and height of each dense prediction bounding box, the corner point coordinates of each dense prediction bounding box are calculated. The specific calculation formula is not detailed here.

[0118] Further, in one feasible embodiment, step S30 includes:

[0119] Step S3033: Calculate the average value of the coordinate data of each target prediction box to obtain the coordinate data of the reference position;

[0120] In this embodiment, the average position of the target prediction box is used as the reference position. Specifically, the average value of the coordinate data of each dense prediction box is calculated to obtain the coordinate data of the reference position.

[0121] Step S3033: Calculate the variance between the coordinate data of each target prediction box and the coordinate data of the reference position to obtain the degree of offset of each target prediction box relative to the reference position.

[0122] In this embodiment, the variance between the coordinate data of each target prediction box and the coordinate data of the reference position is calculated to obtain the degree of offset of each target prediction box relative to the reference position.

[0123] Variance can be used to measure the dispersion of a set of data. The smaller the variance, the smaller the dispersion, and the smaller the offset of each target prediction box relative to the reference position, meaning the positions of each target prediction box are more concentrated and tend to be the same. Conversely, the larger the variance, the greater the dispersion, and the greater the offset of each target prediction box relative to the reference position, meaning the positions of each target prediction box are more dispersed. This embodiment optimizes the target detection model under training by using gradient backpropagation based on the offset between target prediction boxes. This makes the positions of the prediction boxes matching the same target tend to be the same during the target detection process. When performing target detection on multiple consecutive frames of images, the detection boxes of each frame are determined after applying Non-Maximum Suppression (NMS) based on the confidence level. Since the positions of the target prediction boxes of the same target in the same frame tend to be the same, the positions of the detection boxes in each frame also tend to be the same, thereby improving the stability of the target detection boxes.

[0124] Further, in a feasible implementation, the target detection model to be trained can be a multi-target detection model, that is, multiple ground truth bounding boxes are labeled in a sample image. In this implementation, each ground truth bounding box is matched with multiple target prediction bounding boxes. For convenience, the multiple target prediction bounding boxes matched by a ground truth bounding box are referred to as a prediction box group, and each anchor box in a prediction box group corresponds to the same ground truth bounding box. In this implementation, the average coordinate data of each target prediction bounding box in a prediction box group is determined as the coordinate data of the reference position of the prediction box group. The variance between the coordinate data of the target prediction bounding boxes in each prediction box group and the coordinate data of the reference position in the prediction box group is calculated to obtain the degree of offset of each target prediction bounding box in each prediction box group relative to the reference position. In this implementation, the average value of the offset of each prediction box group is used as the second loss. The formula for calculating the second loss in this implementation can be as follows:

[0125]

[0126] Where m represents the number of predicted boxes matched by each ground truth bounding box, j represents the nth predicted bounding box, n represents the number of ground truth bounding boxes labeled in the sample image, and k represents the nth ground truth bounding box. i represents the nth coordinate, e iLet represent the mean of the coordinates, where i = 0, 1, 2, and 3, representing the x-coordinate of the first corner point, the y-coordinate of the first corner point, the x-coordinate of the second corner point, and the y-coordinate of the second corner point, respectively.

[0127] In this embodiment, regression vectors are input into the auxiliary training branch, where they are combined with anchor boxes to decode and obtain multiple dense prediction boxes. Based on the area intersection-over-union (IoU) ratio between the ground truth boxes and each dense prediction box, the target prediction box matching the ground truth box is determined from each dense prediction box. This embodiment effectively determines the regression position of anchor boxes matching the same target, avoiding interference from the regression positions of non-target-matching anchor boxes in the auxiliary training branch, thus improving the accuracy of the auxiliary training branch and consequently enhancing the accuracy of the target detection model.

[0128] Furthermore, in one feasible implementation, refer to Figure 4 The process of calculating the second loss (i.e., stability loss) can be as follows: Obtain a regression vector (i.e., regression vector) with output dimensions H*W*(A*4) from the regression branch, where A represents the number of anchors at each pixel. Input the regression vector into the auxiliary training branch, where the auxiliary training branch combines the regression vector and anchor boxes to decode and obtain the coordinate data of the dense prediction boxes (i.e., the decoder). Figure 4 As shown, the sample image is labeled with two ground truth boxes (gt1 and gt2). After decoding, each ground truth box needs to be matched with its corresponding target prediction box. Specifically, the target prediction box corresponding to each ground truth box is determined by performing area intersection-union (IoU) matching (i.e., IoU filter) between the decoded dense prediction boxes (i.e., DecodedBoxes) and the ground truth boxes. The target prediction boxes corresponding to the same ground truth box are called a prediction box group. The offset within each prediction box group is calculated, and the average of the offsets of each prediction box group is used as the second loss. That is, the formula for calculating the second loss can be as follows:

[0129]

[0130] Where m represents the number of predicted boxes matched by each ground truth bounding box, j represents the nth predicted bounding box, n represents the number of ground truth bounding boxes labeled in the sample image, and k represents the nth ground truth bounding box. i represents the nth coordinate, e i Let represent the mean of the coordinates, where i = 0, 1, 2, and 3, representing the x-coordinate of the first corner point, the y-coordinate of the first corner point, the x-coordinate of the second corner point, and the y-coordinate of the second corner point, respectively.

[0131] Furthermore, embodiments of the present invention also propose a target detection model training device, referring to... Figure 5The target detection model training device includes:

[0132] Module 10 is used to acquire a sample dataset of labeled ground truth bounding boxes;

[0133] The construction module 20 is used to construct an auxiliary training branch for supervising and reinforcing the anchor box regression position, and to add the auxiliary training branch to a preset target detection model to obtain a target detection model to be trained, wherein the anchor box is a preset dense sampling box in the preset target detection model;

[0134] The training module 30 is used to train the target detection model to be trained using the sample dataset to obtain the trained target detection model.

[0135] Furthermore, the training module 30 is also used for:

[0136] Each sample image from the sample dataset is input into the target detection model to be trained, and the classification vector output by the classification branch of the target detection model to be trained and the regression vector output by the regression branch of the target detection model to be trained are obtained.

[0137] The classification vector and the regression vector are input into the model loss function of the target detection model to be trained to obtain the first loss;

[0138] The regression vector is input into the auxiliary training branch to obtain the target prediction box corresponding to the ground truth box, and the offset degree of each target prediction box relative to the reference position is determined. The offset degree is used as the second loss, wherein the coordinate data of the reference position is calculated based on the coordinate data of each target prediction box.

[0139] Based on the gradient descent optimizer and the first and second losses, the parameters of the target detection model to be trained are optimized through backpropagation to obtain the trained target detection model.

[0140] Furthermore, the training module 30 is also used for:

[0141] The regression vector is input into the auxiliary training branch, and the auxiliary training branch combines the regression vector and the anchor box to decode and obtain multiple dense prediction boxes.

[0142] Based on the area intersection-union ratio between the ground truth bounding box and each of the dense prediction bounding boxes, a target prediction bounding box that matches the ground truth bounding box is determined from each of the dense prediction bounding boxes.

[0143] Furthermore, the training module 30 is also used for:

[0144] Calculate the area intersection-union ratio between each of the dense predicted bounding boxes and the ground truth bounding boxes;

[0145] The dense prediction boxes whose area intersection-union ratio with the ground truth box is within a preset range are determined as the target prediction boxes that match the ground truth box.

[0146] Furthermore, the training module 30 is also used for:

[0147] The regression vector is input into the auxiliary training branch, and the auxiliary training branch combines the regression vector and the preset anchor box to decode and obtain the corner coordinates of multiple dense prediction boxes.

[0148] The coordinates of the first corner point and the second corner point in the coordinates of the first corner point are respectively used as the coordinate data of each of the dense prediction boxes, wherein the first corner point and the second corner point are diagonal points to each other.

[0149] Furthermore, the training module 30 is also used for:

[0150] The regression vector is input into the auxiliary training branch, and the auxiliary training branch combines the regression vector and the preset anchor box to decode and obtain the center point coordinates, width and height of each dense prediction box;

[0151] Based on the center point coordinates, width, and height of each of the dense prediction boxes, the corner point coordinates of each of the dense prediction boxes are calculated.

[0152] Furthermore, the training module 30 is also used for:

[0153] Calculate the average of the coordinate data of each target prediction box to obtain the coordinate data of the reference position;

[0154] Calculate the variance between the coordinate data of each target prediction box and the coordinate data of the reference position to obtain the degree of offset of each target prediction box relative to the reference position.

[0155] All embodiments of the target detection model training device of the present invention can refer to the various embodiments of the target detection model training method of the present invention, and will not be repeated here.

[0156] Furthermore, embodiments of the present invention also propose a computer-readable storage medium storing a target detection model training program, wherein the target detection model training program, when executed by a processor, implements the steps of the target detection model training method described below.

[0157] The various embodiments of the target detection model training device and computer-readable storage medium of the present invention can be referred to the various embodiments of the target detection model training method of the present invention, and will not be repeated here.

[0158] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0159] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0160] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0161] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for training an object detection model, characterized in that, The object detection model training method includes the following steps: Obtain the sample dataset with labeled ground truth bounding boxes; An auxiliary training branch is constructed to supervise and enhance the anchor box regression position. The auxiliary training branch is connected to the regression branch output layer of the preset target detection model. The auxiliary training branch is added to the preset target detection model to obtain the target detection model to be trained. The anchor box is a preset dense sampling box in the preset target detection model. Each sample image from the sample dataset is input into the target detection model to be trained, and the classification vector output by the classification branch of the target detection model to be trained and the regression vector output by the regression branch of the target detection model to be trained are obtained. The classification vector and the regression vector are input into the model loss function of the target detection model to be trained to obtain the first loss; The regression vector is input into the auxiliary training branch to obtain the target prediction box corresponding to the ground truth box, and the offset degree of each target prediction box relative to the reference position is determined. The offset degree is used as the second loss, wherein the coordinate data of the reference position is calculated based on the coordinate data of each target prediction box. Based on the gradient descent optimizer and the first and second losses, the parameters of the target detection model to be trained are optimized through backpropagation to obtain the trained target detection model.

2. The target detection model training method as described in claim 1, characterized in that, The step of inputting the regression vector into the auxiliary training branch to obtain the target prediction box corresponding to the ground truth box includes: The regression vector is input into the auxiliary training branch, and the auxiliary training branch combines the regression vector and the anchor box to decode and obtain multiple dense prediction boxes. Based on the area intersection-union ratio between the ground truth bounding box and each of the dense prediction bounding boxes, a target prediction bounding box that matches the ground truth bounding box is determined from each of the dense prediction bounding boxes.

3. The target detection model training method as described in claim 2, characterized in that, The step of determining the target prediction box that matches the ground truth box from among the dense prediction boxes based on the area intersection-union ratio between the ground truth box and each of the dense prediction boxes includes: Calculate the area intersection-union ratio between each of the dense predicted bounding boxes and the ground truth bounding boxes; The dense prediction boxes whose area intersection-union ratio with the ground truth box is within a preset range are determined as the target prediction boxes that match the ground truth box.

4. The target detection model training method as described in claim 2, characterized in that, The step of inputting the regression vector into the auxiliary training branch, and decoding the regression vector and the preset anchor box in the auxiliary training branch to obtain dense prediction boxes includes: The regression vector is input into the auxiliary training branch, and the auxiliary training branch combines the regression vector and the preset anchor box to decode and obtain the corner coordinates of multiple dense prediction boxes. The coordinates of the first corner point and the second corner point in the coordinates of the first corner point are respectively used as the coordinate data of each of the dense prediction boxes, wherein the first corner point and the second corner point are diagonal points to each other.

5. The target detection model training method as described in claim 4, characterized in that, The step of inputting the regression vector into the auxiliary training branch, and decoding the regression vector and the preset anchor box in the auxiliary training branch to obtain the corner coordinates of each dense prediction box includes: The regression vector is input into the auxiliary training branch, and the auxiliary training branch combines the regression vector and the preset anchor box to decode and obtain the center point coordinates, width and height of each dense prediction box; Based on the center point coordinates, width, and height of each of the dense prediction boxes, the corner point coordinates of each of the dense prediction boxes are calculated.

6. The target detection model training method according to any one of claims 1 to 5, characterized in that, The step of determining the offset of each of the target prediction boxes relative to the reference position includes: Calculate the average of the coordinate data of each target prediction box to obtain the coordinate data of the reference position; Calculate the variance between the coordinate data of each target prediction box and the coordinate data of the reference position to obtain the degree of offset of each target prediction box relative to the reference position.

7. A target detection model training device, characterized in that, The target detection model training device includes: The acquisition module is used to acquire a sample dataset of labeled ground truth bounding boxes; A construction module is used to construct an auxiliary training branch for supervising and reinforcing the anchor box regression position. The auxiliary training branch is connected to the regression branch output layer of the preset target detection model, and the auxiliary training branch is added to the preset target detection model to obtain the target detection model to be trained. The anchor box is a preset dense sampling box in the preset target detection model. A training module is used to train the target detection model to be trained using the sample dataset to obtain a trained target detection model. Specifically, the training module inputs each sample image from the sample dataset into the target detection model to obtain a classification vector output by the classification branch and a regression vector output by the regression branch. The classification vector and regression vector are then input into the model loss function of the target detection model to obtain a first loss. The regression vector is input into the auxiliary training branch to obtain the predicted target boxes corresponding to the ground truth boxes, and the offset of each predicted target box relative to a reference position is determined. This offset is used as a second loss, where the coordinate data of the reference position is calculated based on the coordinate data of each predicted target box. Finally, based on a gradient descent optimizer and the first and second losses, the parameters of the target detection model are optimized through backpropagation to obtain a trained target detection model.

8. A target detection model training device, characterized in that, The target detection model training device includes: a memory, a processor, and a target detection model training program stored in the memory and executable on the processor. When the target detection model training program is executed by the processor, it implements the steps of the target detection model training method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a target detection model training program, which, when executed by a processor, implements the steps of the target detection model training method as described in any one of claims 1 to 6.

Images