Combined target detection method

Abstract

The invention discloses a combined target detection method, which comprises the following steps of: S1, acquiring a picture training sample, acquiring a plurality of targets from the picture training sample, and carrying out relevance combination labeling on the targets with relevance in the plurality of acquired targets; s2, performing iou value calculation on one target and a combined target thereof; if a plurality of iou values exist when one target is associated with a plurality of targets, selecting the maximum iou value as a training iou value; s3, calculating the length and direction of a target center vector between the two targets according to the relevance combination of the selected maximum iou value; and S4, performing model training on the target detection network based on the training iou value and the length and direction of the related target center vector to obtain a trained target detection model. Under the current mainstream target detection training framework, complex labeling is not needed, and the detection performance of the model on the combined target in the scene is improved through a method of improving the training process and optimizing the loss function.

Description

Technical Field

[0001] The invention relates to a combined target detection method. Background Art

[0002] In conventional target detection scenarios, there are usually some cases where the detection target has a "combination" nature. For example, when a person sits on a sofa, the person and the sofa form a combination, and when a person rides a bicycle, the person and the bicycle form a combination. However, combined targets often have problems with being blocked or deformed. For example, when a person rides a bicycle, the person often blocks part of the bicycle, and the person's movements are often different from those of normal people walking or running. In this case, the detection rate of people and cars will decrease. More commonly, only one detection frame can be output for people and cars, and no other detection frame can be output. When faced with more stringent detection requirements, such as in road monitoring and construction monitoring, if the target is missed, it may cause more serious consequences. Summary of the invention

[0003] The purpose of the present invention is to provide a combined target detection method to solve the problems raised in the above background technology. To achieve the above purpose, the present invention provides the following technical solutions:

[0004] A combined target detection method, characterized in that it comprises the following steps:

[0005] S1: Obtain a picture training sample, obtain multiple targets from the picture training sample, and perform correlation combination annotation on the associated multiple targets;

[0006] S2: Calculate the iou value of a target and its combined target; if a target is associated with multiple targets, there are multiple iou values, and select the target with the largest iou value with the first target as the combined target;

[0007] S3: Calculate the length and direction of the target center vector between the two targets by selecting the correlation combination with the maximum iou value;

[0008] S4: Perform model training on the target detection network based on the training iou value and the length and direction of the relevant target center vector to obtain a trained target detection model.

[0009] Furthermore, when multiple targets are associated and combined, the targets that may be combined are annotated into a separate annotation file.

[0010] Furthermore, in the associative combination, including target frame 1 and target frame 2, the center point of target frame 1 is taken as the starting point and the center point of target frame 2 is taken as the end point, and the length and direction of the target center vector are calculated, wherein the lengths in the x and y directions are expressed as the ratio of the center point distance in the maximum circumscribed rectangle of the two target frames, and the lengths in the x and y directions of the vector are expressed as the ratio of the center point distance in the maximum circumscribed rectangle of the two target frames, the maximum circumscribed rectangles of the two target frames are w and h respectively, the length of the vector in the x direction is w1, and the length in the y direction is h1, then the lengths are expressed as x direction: w1 / w, y direction h1 / h, and the angles between the two are expressed as:

[0011] Further, the target detection network is trained based on the training iou value and the length and direction of the relevant target center vector to obtain the trained target detection model, including the following steps:

[0012] S41: Obtain the length and direction of the relevant target center vector through model forward propagation, send it to the loss function model, and then update the target detection network weight by performing gradient back propagation on the derivative of the loss function to obtain the combined loss function;

[0013] The combined objective loss function is as follows:

[0014]

[0015] Among them, ρ represents the square of the Euclidean distance between the center points of two boxes, and ε is a small constant to avoid the denominator being 0. pred and gt are the predicted target center vector and the real target center vector, w and h are the maximum width and height of the bounding rectangle, and ωi is a parameter in the model;

[0016] S42: adjusting the ratio of the combined target loss and the conventional target loss in the training model based on the dynamic combined target loss weight;

[0017] Furthermore, the conventional target loss is in the form of: L_general = a*(L_cls+L_iou+L_dfl), and its combined target loss is: L_comb = b*L, where L is the combined loss function L, a, b are dynamic weights;

[0018] During the training process, according to the setting of the total training epoch, the training is divided into three stages with 1 / 3 of the total epoch number. In each stage, the goals of the training model are different. According to these goals, the dynamic loss weight is adjusted to promote the convergence of the model in each stage:

[0019] In each stage, the default dynamic weight of L_general is a=1, and the dynamic weight b of L_comb is adjusted according to the specific effect.

[0020] Furthermore, in the first stage, the training goal of the model is to focus on the detection effect of training independent targets, and the dynamic weight b of L_comb is set to 0;

[0021] In the second stage, the training goal of the model is to focus on promoting the convergence speed of the combined target, so that the network can sufficiently consider the loss of L_comb, but cannot affect the convergence of the normal target detection loss. The dynamic weight b of L_comb is set to 0.1*(L_comb / L_general);

[0022] In the third stage, the training goal of the model is to focus on achieving final convergence, and the dynamic weight b of L_comb is set to 0.75*(count_comb / count_target).

[0023] Beneficial effects:

[0024] Compared with traditional target detection, the present invention has the following advantages:

[0025] The current mainstream target detection framework of the present invention improves the detection performance of the model for combined targets in this scenario by improving the training process and optimizing the loss function without complex labeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a training sample graph;

[0027] Figure 2 It is a yolo series target detection network diagram. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] The present invention provides a combined target detection method, comprising the following steps:

[0030] S1: Obtain image training samples, obtain multiple targets from the image training samples, and perform associated combination annotations on the associated multiple targets; in conventional target detection data sets, there will be annotations for the detection target positions, i.e., x, y, w, h, where xy is the coordinate point of the upper left corner, and wh is the width and height of the target.

[0031] As an example: the acquired image training samples are used to obtain multiple targets, and the combined targets that may be generated from the multiple targets are annotated into a separate annotation file. The annotation file is the annotation information shared by all training samples, and there is no need to annotate each sample independently. For example, see Figure 1 In the scenario where a person rides an ebike, the two targets usually have a certain combination relationship. However, if two independent targets are trained, because part of the person or the ebike may be occluded, usually only one target category will be detected. In order to better handle such samples, the two labels person and ebike can be written in one line in the association annotation file, and a true flag can be added after them, which means that the person and the ebike may be combined in this training, that is, describing the situation where a person rides an ebike or a person pushes an ebike.

[0032] S2: Calculate the iou value for a target and its combined target; if a target is associated with multiple targets, there are multiple iou values, and select the maximum iou value as the training iou value;

[0033] As an example: if the iou of two electric vehicles and one person is greater than 0, only the target association and iou value of the person with the largest iou and the electric vehicle are retained;

[0034] S3: Calculate the length and direction of the target center vector between the two targets by selecting the correlation combination with the maximum iou value;

[0035] As an example: the label order is defined in advance as people and electric vehicles. Then, in the associated targets, we use the center point of the person target frame as the starting point and the center point of the electric vehicle target frame as the end point to calculate the length and direction of the target center vector. In order to enable the model to be optimized later and correctly complete the derivation process of back propagation, the representation of the vector length and direction is defined: the length of the vector in the x and y directions is represented by the ratio of the center point distance in the maximum circumscribed rectangle of the two target frames. The maximum circumscribed rectangles of the two targets are w and h. The length of the vector in the x direction is w1 and the length in the y direction is h1. The lengths are expressed as w1 / w in the x direction and h1 / h in the y direction, respectively. The angle between the two is expressed as:

[0036] S4: Perform model training on the target detection network based on the training iou value and the length and direction of the relevant target center vector to obtain a trained target detection model;

[0037] Specifically, the steps include:

[0038] S41: Obtain the length and direction of the relevant target center vector through model forward propagation, send it to the loss function model, and then update the target detection network weight by performing gradient back propagation on the derivative of the loss function to obtain the combined loss function;

[0039] As an example: At present, the commonly used YOLO series target detection network structure is as follows Figure 2 As shown in the figure, the information that needs to be analyzed is finally obtained from the yellow feature map below, which is analyzed through the operator and sent to the loss function. Then, the network weights are updated by performing gradient back propagation on the derivative of the loss function.

[0040] The loss function consists of three parts: associated iou, associated center direction, and associated center distance. The formula is:

[0041] L=αL IoU +βL Dir +γL Dist +λR

[0042] Among them, LIoU is the association IoU loss, LDir is the association center direction loss, LDist is the association center distance loss, R is the regularization term; α, β, γ, λ are the corresponding weight coefficients.

[0043] To provide a more stable gradient, an improved IoU loss is used as follows:

[0044]

[0045] ρ represents the square of the Euclidean distance between the center points of two boxes, and ∈ is a small constant to avoid the denominator being zero.

[0046] The angular error is used to measure the direction difference, and the cosine similarity is used as follows:

[0047] L Dir =1-cos(θ pred -θ gt )

[0048] The normalized vector length is used as the distance loss, as follows:

[0049]

[0050] in pred and gt are the predicted target center vector and the true target center vector, respectively, and w and h are the maximum width and height of the bounding rectangle.

[0051] L2 regularization is introduced to control the complexity of the model, as follows:

[0052]

[0053] where ωi are parameters in the model.

[0054] Putting the above together, the final combined objective loss function is as follows:

[0055]

[0056] S42: adjusting the ratio of the combined target loss and the conventional target loss in the training model based on the dynamic combined target loss weight;

[0057] As an example: when two or more targets are determined to be combined targets, a dynamic combined target loss weight is needed to adjust the ratio of the combined target loss to the regular target loss. For a regular target detection model, the regular target loss is in the form of:

[0058] L_general=a*(L_cls+L_iou+L_dfl), and its combined target loss L_comb=b*L, where L is the combined loss function L, a and b are dynamic weights; during the training process, according to the setting of the total training epoch, the training is divided into three stages with 1 / 3 of the total epoch number. In each stage, the goals of the training model are different. According to these goals, we need to adjust the dynamic loss weight to promote the convergence of the model at each stage:

[0059] In each stage, the default dynamic weight of L_general is a=1, and the weight a can be adjusted according to the specific effect during training. In this patent, we default to adjusting the dynamic weight b of L_comb while fixing a.

[0060] The specific training process is as follows:

[0061] In the first stage, the training goal of the model is to focus on the detection effect of independent targets. Therefore, in this stage, we set the dynamic weight b of L_comb to 0 and ignore the problem of combined loss. Because in this stage, if the network itself does not have a good independent target detection ability, the combined loss value will be very large and cannot be balanced with the independent loss value to achieve the effect of synchronous convergence.

[0062] In the second stage, the dynamic weight b is set to 0.1*(L_comb / L_general). In this stage, what we need to do is to promote the convergence speed of the combined target, so that the network can sufficiently consider the loss of L_comb, but not affect the convergence of the normal target detection loss. Therefore, when L_general is large enough, we consider appropriately reducing the weight of L_comb so that the network can prioritize the weight of target detection. However, when the loss of L_general is small, we tend to increase the weight of L_comb loss, so that the model can increase the intensity of training the combined target on the basis of good performance of L_general, so that the combined target training method designed by us can play its due effect.

[0063] In the third stage, the model is about to reach final convergence. At this time, we select the value of the dynamic weight b according to the proportion of the number of combined targets in the training set in the detection targets, 0.75*(count_comb / count_target), so that the overall loss reaches the final balanced convergence state.

[0064] The above is only a preferred embodiment of the present invention, and does not limit the present invention in any form. The protection scope of the present invention shall be subject to the protection scope of the claims. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modifications to equivalent embodiments of equivalent changes using the technical content disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the scope of the technical solution of the present invention.

Claims

1. A combined target detection method, characterized in that: The following steps are involved: S1: Obtain a picture training sample, obtain multiple targets from the picture training sample, and perform correlation combination annotation on the associated multiple targets; S2: Calculate the iou value of a target and its combined target; if a target is associated with multiple targets, there are multiple iou values, and select the target with the largest iou value with the first target as the combined target; S3: Calculate the length and direction of the target center vector between the two targets by selecting the correlation combination with the maximum iou value; S4: Perform model training on the target detection network based on the training iou value and the length and direction of the relevant target center vector to obtain a trained target detection model.

2. The combined target detection method according to claim 1, characterized in that: When associative combined annotation is performed on multiple targets that are associated, the targets that may be combined will be annotated into a separate annotation file.

3. The combined target detection method according to claim 2, characterized in that: In the associative combination, including target frame 1 and target frame 2, the center point of target frame 1 is taken as the starting point and the center point of target frame 2 is taken as the end point. The length and direction of the target center vector are calculated. The lengths in the x and y directions are expressed as the ratio of the center point distance to the maximum circumscribed rectangle of the two target frames. The lengths in the x and y directions of the vector are expressed as the ratio of the center point distance to the maximum circumscribed rectangle of the two target frames. The maximum circumscribed rectangles of the two target frames are w and h respectively. The length of the vector in the x direction is w1 and the length in the y direction is h1. The lengths are expressed as w1 / w in the x direction and h1 / h in the y direction respectively. The angles between the two are expressed as:

4. The combined target detection method according to claim 1, characterized in that: The target detection network model training based on the training iou value and the length and direction of the relevant target center vector to obtain the trained target detection model includes the following steps: S41: Obtain the length and direction of the relevant target center vector through model forward propagation, send it to the loss function model, and then update the target detection network weight by performing gradient back propagation on the derivative of the loss function to obtain the combined loss function; The combined objective loss function is as follows: Among them, ρ represents the square of the Euclidean distance between the center points of two boxes, and ε is a small constant to avoid the denominator being 0. pred and gt are the predicted target center vector and the real target center vector, w and h are the maximum width and height of the bounding rectangle, and ωi is a parameter in the model; S42: Adjust the ratio of combined target loss and conventional target loss in the training model based on the dynamic combined target loss weight.

5. The combined target detection method according to claim 4, characterized in that: The conventional target loss is of the form: L_general = a*(L_cls+L_iou+L_dfl), its combined target loss: L_comb = b*L, where L is the combined loss function L, a, b are dynamic weights; During the training process, according to the setting of the total training epoch, the training is divided into three stages with 1 / 3 of the total epoch number. In each stage, the goals of the training model are different. According to these goals, the dynamic loss weight is adjusted to promote the convergence of the model in each stage: In each stage, the default dynamic weight of L_general is a=1, and the dynamic weight b of L_comb is adjusted according to the specific effect.

6. The combined target detection method according to claim 5, characterized in that: In the first stage, the training goal of the model is to focus on the detection effect of training independent targets, and the dynamic weight b of L_comb is set to 0; In the second stage, the training goal of the model is to focus on promoting the convergence speed of the combined target, so that the network can sufficiently consider the loss of L_comb, but cannot affect the convergence of the normal target detection loss. The dynamic weight b of L_comb is set to 0.1*(L_comb / L_general); In the third stage, the training goal of the model is to focus on achieving final convergence, and the dynamic weight b of L_comb is set to 0.75*(count_comb / count_target).

Images