Online object detection model training method based on intermediate position encoding

Abstract

The present invention discloses an online target detection model training method based on intermediate position coding. The method first introduces an intermediate position encoder to re-encode the input label, and gradually reduces the coding weight from the center point to the boundary point in a linear attenuation manner. Then, the image batches to be trained are input into the backbone network to extract features. The prediction results are output by a shared detection head, the loss between the prediction results and the true value is calculated, and the results are back-propagated to the feature space and the label space respectively, so that the parameters in the space are gradient optimized. The method uses a joint optimization method to simultaneously update the parameters of different spaces. Finally, the loss is measured by the intermediate distance, and the difference between the feature space and the label space is reduced, so that the model parameters tend to a stable distribution. The target detection model method disclosed by the present invention solves the problem that the parameters are not easy to converge due to the random initialization of network weights.

Description

Technical Field

[0001] The present invention relates to technical fields such as computer vision and pattern recognition, and mainly to a target detection model training method based on deep learning, which has wide applications in scientific research experiments, smart cities, and industrial detection. Background Art

[0002] Object detection is one of the three basic application tasks in the field of computer vision. It is currently widely used in security monitoring, autonomous driving, and industrial product inspection. However, the object detection model has the problems of large number of parameters, complex model, and difficulty in convergence when training large models. Therefore, a reasonable object detection model training method has high practical value and broad application prospects.

[0003] In the past decade, the research on deep learning model training methods has made a series of progress, but the research on target detection model training methods is still at the stage of adding pre-trained models to initialize model weights, and the research field of online target detection training methods is still blank. At present, target detection model training methods can be mainly divided into two categories: pre-trained model-based methods and knowledge distillation-based methods. Pre-trained model-based methods use the method of initializing network parameters to speed up model training by using model weights that have been trained on other training sets. Some early pre-trained model-based methods initialize target detection model parameters through image classification models. Due to the different distribution of task objectives and dataset images, parameter initialization methods may lead to negative optimization problems. Later, with the development of parameter initialization technology, He et al. proposed a parameter initialization method based on Gaussian distribution in 2019 and made great progress.

[0004] Methods based on knowledge distillation usually use a teacher model that is more complex and has more parameters than the original model, and the target detection model as the student model. The model training is completed through online learning of relevant knowledge through knowledge distillation. The teacher-student network uses transfer learning to allow the student network to learn the knowledge learned by the teacher network model through knowledge distillation with a model with fewer parameters. This method is currently a mainstream online model training method in the industry. However, this method still has major drawbacks, because the teacher network and the student network need feature matching, an inappropriate teacher network may make the training process unstable, and a large network model requires more hyperparameters, which is not allowed for model optimization. Recently, some researchers have adopted a cyclic teacher-student network to solve the feature matching problem of the model. In 2020, Li et al. regarded network design as a feature space search problem and used NAS to construct the overall network model. Experiments have shown that feature alignment in transfer learning can be better achieved through online learning.

[0005] Network model pre-training technology has made great progress, but in real application scenarios, the size of the dataset may not meet the needs of model training. Therefore, in order to solve this problem, an online object detection model training method based on intermediate position encoding is proposed. Summary of the invention

[0006] Purpose of the invention: In actual application scenarios, the training of target detection models is not only limited by the size of the data set, but also by the selection of the knowledge distillation teacher network. Therefore, the present invention hopes to solve the problem of parameter convergence caused by randomly initialized network weights through online learning, using intermediate position encoding as a feature alignment method, so that target detection technology can be truly implemented and save more resources for social development.

[0007] 1. An online target detection model training method based on intermediate position coding, characterized in that it comprises the following steps:

[0008] Step 1.1: Input the training image data and the real image label into the backbone network and the autoencoder module respectively, and encode the data in the autoencoder module. The defined autoencoder optimizes the label encoding function to complete the feature alignment in the label space;

[0009] Step 1.2: Use the label feature obtained by the encoder as the true value and the result predicted by the detection head to calculate the detection loss, and at the same time pass the prediction result into the label space to obtain the label space loss;

[0010] Step 1.3: Through the joint optimization of the loss functions of detection loss and label space loss, the parameters are updated using the deep learning back-propagation strategy to complete the neural network weight learning;

[0011] Step 1.4: Use the trained label encoding function to map the label to the feature space, and use the intermediate supervision loss to assist the object detection model training;

[0012] Step 1.5: After the label space loss is stable, the back propagation part from the detection head to the label encoding is removed, and the overall network structure only updates the parameters through the detection loss;

[0013] 2. The training data processing encoding method in step 1.1 is as follows:

[0014] Step 2.1: Input image and corresponding true label file;

[0015] Step 2.2: Each image contains a different number of instances, each of which may have a different class label and bounding box. For bounding boxes of different sizes, a tensor of shape CxHxW is used, where C represents the type of instance, H represents the length of the feature map, and W represents the width of the feature map;

[0016] Step 2.3: For a dataset with a total of N types of items, for the C-th type of object, fill the bounding box area in the C-dimensional channel with positive values, the value of the center point of the bounding box is 1, the value of the boundary position is 0.5, and the weight decay adopts a linear decay method;

[0017] Step 2.4: During training, the bounding boxes are augmented by multiplying them by a distance weight between 0 and 1. If two bounding boxes of the same type overlap, the larger value of the bounding box is calculated to fill the intersection area.

[0018] 3. The method for defining the autoencoder label space loss function in step 1.2 is:

[0019] Step 3.1: Initialize the encoder using pre-trained weights;

[0020] Step 3.2: Define the auxiliary autoencoder. An encoding function is required to map the true label to the latent space. The entire optimization process needs to exist in both the objective and the constraint. It can be regarded as a hierarchical optimization problem, which can be transformed into a joint optimization solution through quadratic convex optimization. The objective function is:

[0021]

[0022] Among them, θ d represents the label space, θ f represents the feature space, L det represents the loss function between the generated label and the real label, λ represents the weight hyperparameter, L dis Represents the detection loss.

[0023] Step 3.3: Update the encoder module parameters by back-propagation, including the distance loss L between the generated position encoding and the true label det and the difference loss of the detection head L dis , respectively:

[0024]

[0025] Among them, X r represents the true label value, X f Represents the generated position encoding value, and N represents the total number of samples.

[0026]

[0027] Among them, X p represents the predicted value, X r Represents the true value.

[0028] 4. The method for joint optimization of the loss functions of the three stages defined in step 1.4 is:

[0029] Step 4.1: After learning the label encoding function in requirement 3, this part is the intermediate supervision used to improve the training of target detection;

[0030] Step 4.2: In order to learn the information contained in the real scene, the detection head is set to a shared detection head, and the results of the label space and feature space are detected at the same time. The loss function of this part is the detection loss, and the loss function is:

[0031]

[0032] in, The label value representing the real sample includes the set of coordinates of the center point of the real bounding box and the width and height corresponding to the real box, and x represents the set of coordinates of the center point of the predicted bounding box and the width and height corresponding to the box.

[0033] Step 4.3: The label autoencoder needs to act on the detector network by sharing the detection head to exchange information. To minimize the label space and the loss function of information exchange is:

[0034]

[0035] Among them, BN represents batch normalization, A represents feature adaptation network, and x f represents the predicted value, x r Represents the true value.

[0036] Step 4.4: Use the balance coefficients α, β and γ to balance the multi-task paradigm of the three loss terms. The final overall loss function is:

[0037] L=αL det +βL IOU +γL message

[0038] 5. The intermediate supervision method of the self-label encoder in step 4.3 is:

[0039] Step 5.1: Output the encoding map of the feature space through the self-label encoder. This part is used as the intermediate supervision to assist the training of the target detector. To ensure the effect of intermediate supervision, the intermediate auxiliary loss L is introduced to balance the label encoding space and feature space. message ;

[0040] Step 5.2: Compared with the backbone network, it is simpler and converges faster. det When it tends to be stable, it can be considered that the label autoencoder has learned the predetermined label encoding function and no further learning is needed. In order to reduce the complexity of the network, when the loss function of the label autoencoder tends to be stable, it is necessary to prevent the gradient flow from backpropagating to the label autoencoder. At this time, the overall loss function is:

[0041] L=L IOU +0.5xL message

[0042] When the detector loss function becomes stable, the object detection network training is completed.

[0043] Beneficial results of the present invention:

[0044] On this basis, combined with the relevant technologies of transfer learning and knowledge distillation, a solution for online training of target detection models is proposed, which greatly improves the possibility of implementing target detection technology in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a core structure diagram of the online target detection model training method based on intermediate position coding described in the present invention;

[0046] Figure 2 It is a schematic diagram of the structure and composition of the online target detection model;

[0047] Figure 3 It is a schematic diagram of the detector structure with intermediate supervision;

[0048] Figure 4 It is a schematic diagram of the autoencoder structure;

[0049] Figure 5 This is a schematic diagram of the components of the autoencoder network; DETAILED DESCRIPTION

[0050] The present invention will be further described below in conjunction with the accompanying drawings.

[0051] like Figure 1 As shown, the online target detection training method based on intermediate position coding described in the present invention has the following specific process:

[0052] Step 1: Input the image and the corresponding true label, and use the autoencoder module to automatically encode the label. The autoencoder module model structure is as follows: Figure 4 As shown:

[0053] Step 1.1: To enhance the receptive field of the encoder module, a convolutional layer is used to extract features from the original image, where the convolution kernel size is 7x7 and the stride is 2. Then a linear rectification function is used as the activation function, and the number of output channels is 128;

[0054] Step 1.2: Use 6 residual network blocks to further extract image features. The specific structure is as follows: Figure 2 As shown in the figure. The first and second residual network blocks use two convolution kernels of size 1x1, stride 1 and one convolution kernel of size 3x3, stride 2. The convolution kernel size of the third residual network block is the same as the above two convolution kernels, except that the stride of the 3x3 convolution kernel is 1. The size of the fourth and sixth residual blocks is the same as residual blocks 1 and 2, and the size of the fifth residual block is the same as residual block 3. The residual block structure is shown in the figure. Figure 5 As shown, the encoder module finally outputs a feature vector with 2048 channels.

[0055] Step 2: Input the label position encoding obtained in step 1 and the features output by the backbone network into the detection head, and update the parameters through back propagation so that the label position encoding module learns the label encoding function. The joint label encoding structure is as follows: Figure 2 As shown;

[0056] Step 2.1: Input the training images into the backbone network and the real labels into the encoder module. The basic idea is to learn an optimized label encoding function through the defined auxiliary automatic encoding module. Minimizing the label encoding function requires the joint objective and loss function at the same time, which is a hierarchical optimization problem. Combined with the idea of ​​joint optimization, the label autoencoder loss function is defined as L1, and the classification loss and position loss of the detection head are simplified to L2. L3 is to minimize the difference between the label encoding space and the feature space;

[0057] Step 2.2: Under the premise of training the encoding in the first step, the auto-encoding module and the backbone network share the detection head at the same time. The detection result of the detection head is divided into two parts: label prediction and image prediction. The label prediction part is propagated to the label auto-encoding module through the supervision of the L1 loss function, and the image prediction part is propagated to the backbone network through the supervision of the L2 loss. The label position encoding module and the feature space difference use the L3 supervision loss. To ensure the training of the overall network structure, the L3 loss is back-propagated only to the backbone network, not to the target encoder module;

[0058] Step 3: Select the trained label position encoding module and use it as intermediate supervision to improve the training of the target detector, where the detection model training structure is as follows: Figure 4 As shown:

[0059] Step 3.1: Cancel the label space prediction part of the detection head, so that the detection head only detects the spatial features extracted by the backbone network.d Auxiliary loss to directly supervise the detection backbone network;

[0060] Step 3.2: Use the trained label encoder to output the encoding map of the feature space, which is used as intermediate supervision to improve the overall object detector training. To ensure the effect of intermediate supervision, an auxiliary loss L is introduced a To directly supervise the backbone network. At this point, the autoencoder has learned the predetermined label encoding function, so it is necessary to prevent the gradient flow from backpropagating to the automatic label encoder. After the target detection network training is completed, the target encoder can be removed;

[0061] In general, the object detection model is trained on the target dataset COCO, with images and labels as input, of size 448x448. The number of autoencoder training rounds is 10, the contour cardinality in training is set to 32, and for each batch, adaptive moment estimation is selected as the optimizer, cross entropy loss and center loss are used as loss functions. The learning rate is set to 1e -4 The training was performed for 8000 iterations.

[0062] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present invention. They are not intended to limit the scope of protection of the present invention. Any equivalent implementation methods or changes that do not deviate from the technical spirit of the present invention should be included in the scope of protection of the present invention.

Claims

1. An online target detection model training method based on intermediate position coding, characterized in that: The following steps are involved: Step 1.1: Input the training image data and the real image label into the backbone network and the intermediate position residual encoding module respectively, encode the data in the intermediate position residual encoder module, optimize the label encoding function through the intermediate position residual encoder module, and complete the feature alignment of the label space; Step 1.2: Input images and corresponding true label files. Each image contains a different number of instances, and each instance may have a different class label and bounding box. For bounding boxes of different sizes, a tensor of shape CxHxW is used, where C represents the type of instance, H represents the length of the feature map, and W represents the width of the feature map. Step 1.3: For a dataset with a total of N types of items, among which the C-th type of objects, fill the bounding box area in the C-dimensional channel with positive values, the value of the center point of the bounding box is 1, the value of the boundary position is 0.5, and the value decay adopts a linear decay method; Step 1.4: During training, the bounding boxes are weighted by multiplying them by a distance weight between 0 and 1. If two bounding boxes of the same type overlap, the larger value of the bounding box is calculated to fill the intersection area. Step 1.5: The encoder extracts the original image features through a convolutional layer, then activates it with a linear rectification function, and uses 6 residual network blocks to expand the receptive field, and finally outputs a feature vector of 2048 channels; Step 1.6: Use the label feature obtained by the encoder as the true value and the result predicted by the detection head to calculate the detection loss, and at the same time pass the prediction result into the label space to obtain the label space loss; Step 1.7: Through the joint optimization of the loss functions of detection loss and label space loss, the parameters are updated using the deep learning back-propagation strategy to complete the neural network weight learning; Step 1.8: Use the trained label encoding function to map the label to the feature space, and use the intermediate supervision loss to assist the object detection model training; Step 1.9: After the label space loss is stabilized, the back propagation part from the detection head to the label encoding is removed, and the overall network structure is only updated and iterated by the detection loss.

2. The online target detection model training method based on intermediate position coding according to claim 1, characterized in that: The label space loss method defined in step 1.6 is: Step 2.1: Initialize the encoder with pre-trained weights; Step 2.2: Define the auxiliary autoencoder. An encoding function is required to map the true label to the latent space. The entire optimization process needs to exist in both the objective and the constraint. It can be regarded as a hierarchical optimization problem, which can be transformed into a joint optimization solution through quadratic convex optimization. The objective function is: Among them, θ d represents the label space, θ f represents the feature space, L det represents the loss function between the generated label and the real label, λ represents the weight hyperparameter, L dis Indicates detection loss; Step 2.3: Update the encoder module parameters by back-propagation, including the loss L between the generated position encoding and the true label det and the detection head loss L dis , respectively: Among them, X r represents the true label value, X f represents the generated positional encoding value, and N represents the total number of samples; Among them, X p represents the predicted value, X r Represents the true value.

3. The online target detection model training method based on intermediate position coding according to claim 1, characterized in that: The intermediate supervision loss assisted target detection model method defined in step 1.8 is: Step 3.1: After learning the label encoding function in requirement 3, this part is the intermediate supervision used to improve the training of target detection; in order to learn the information contained in the real scene, the detection head is set to a shared detection head, and the results of detecting the label space and feature space at the same time are detected. The loss function of this part is the detection loss, and the loss function is: in, The label value of the real sample includes the coordinates of the center point of the real bounding box and the set of the width and height corresponding to the real box, and x represents the set of the predicted coordinates of the center point of the bounding box and the width and height corresponding to the box; Step 3.2: The label autoencoder needs to act on the detector network by sharing the detection head to exchange information. To minimize the label space and the loss function of information exchange is: Among them, BN represents batch normalization, A represents feature adaptation network, and x f represents the predicted value, x r represents the true value; Step 3.3: Using the multi-task paradigm with three loss terms, the final overall loss function is: L=αL det +βL IOU +γL message Among them, α, β, and γ are the balance coefficients of the three loss terms.

4. The online target detection model training method based on intermediate position coding according to claim 3, characterized in that: The intermediate supervision method of the label autoencoder in step 3.1 is: Step 4.1: Output the encoding mapping of the feature space through the label autoencoder. This part is used as the intermediate supervision to assist the training of the target detector. To ensure the effect of intermediate supervision, the intermediate auxiliary loss L is introduced to balance the label encoding space and feature space. message ; Step 4.2: Compared with the backbone network, it is simpler and converges faster. det When it tends to be stable, it can be considered that the label autoencoder has learned the predetermined label encoding function and no further learning is needed. In order to reduce the complexity of the network, when the loss function of the label autoencoder tends to be stable, it is necessary to prevent the gradient flow from backpropagating to the label autoencoder. At this time, the overall loss function is: L=L IOU +0.5×L message When the detector loss function becomes stable, the object detection network training is completed.

Images