A method for calculating the secondary loss function in vehicle detection training
By introducing a secondary loss function calculation method in MTCNN cascade detection, the loss value is calculated using cross entropy and log-likelihood functions combined with 2-norm, the problem of low false detection and recall in vehicle detection is solved, and the detection is improved Accuracy and training convergence.
Patent Information
- Application Number
- CN202010400157.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-05-13
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2040-05-13
AI Technical Summary
MTCNN cascade detection has problems with low false detection and low recall rates in vehicle detection, especially the difficulty of converging the target with a length-to-width ratio close to 1 in calculation of the first-order loss function, resulting in low accuracy and recall rates.
A secondary loss function calculation method in vehicle detection training is adopted, the loss value is calculated through cross entropy and log-likelihood functions, and the fine-tuning loss is calculated in combination with 2-norm, and the loss value ratio is adjusted to improve detection accuracy.
Improves the accuracy and recall of vehicle detection, ensuring convergence in training, simple and cost-saving methods.
Smart Images

Figure CN113673668B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of neural networks, and in particular to a method for calculating a secondary loss function in vehicle detection training. Background Art
[0002] In today's society, neural network technology in the field of artificial intelligence is developing rapidly. Among them, MTCNN technology is also one of the more popular technologies in recent years. MTCNN, Multi-task convolutional neural network, puts face area detection and face key point detection together, and can be generally divided into three-layer network structures: P-Net, R-Net, and O-Net. Multi-task neural network model for face detection tasks, this model mainly uses three cascaded networks, using the idea of candidate boxes plus classifiers for fast and efficient face detection. The three cascaded networks are P-Net for quickly generating candidate windows, R-Net for high-precision candidate window filtering and selection, and O-Net for generating the final bounding box and face key points.
[0003] However, MTCNN cascade detection has the following defects:
[0004] 1. There are certain false detections, and the recall rate and accuracy rate are relatively low.
[0005] 2. In particular, the first-level loss function calculation used in the prior art can easily converge for targets with a length-to-width ratio close to 1, but cannot converge for vehicles with a large length-to-width ratio, resulting in low accuracy and recall rates.
[0006] In addition, the prior art also includes the following commonly used technical terms:
[0007] 1. Network structure cascade: The method of detecting by connecting several detectors in series is called cascade.
[0008] 2. Convolution kernel: The convolution kernel is a matrix used for image processing and a parameter for performing operations with the original image. The convolution kernel is usually composed of a column matrix (for example, a 3*3 matrix), and each square in the area has a weight value. The matrix shape is generally 1×1, 3×3, 5×5, 7×7, 1×3, 3×1, 2×2, 1×5, 5×1, …
[0009] 3. Convolution: Place the center of the convolution kernel on the pixel to be calculated, calculate the product of each element in the kernel and the image pixel value it covers and sum them up. The resulting structure is the new pixel value at that position. This process is called convolution.
[0010] 4. Activation function: a function that processes the result of convolution.
[0011] 5. Feature map: The result obtained after the input data is calculated through convolution is called a feature map. The result generated after the data is fully connected is also called a feature map. The size of the feature map is generally expressed as length × width × depth, or 1 × depth
[0012] 6. Step size: The length of the movement of the center position of the convolution kernel on the coordinate.
[0013] 7. Non-aligned processing at both ends: When an image or data is processed by a convolution kernel of size 3×3, if it is not enough for one convolution kernel to process, it will lead to insufficient data on both sides. At this time, the data on both sides or one side is discarded. This phenomenon is called non-aligned processing at both ends.
[0014] 8. Loss calculation cascade: It means calculating the loss value at a certain node in the network structure and weighting the loss value into the overall loss. This method of calculating the loss value is called loss calculation cascade.
[0015] 9. Loss function is also called cost function. It is the objective function of neural network optimization. The process of neural network training or optimization is the process of minimizing the loss function (the smaller the loss function value, the closer the corresponding predicted result is to the actual result value. Summary of the invention
[0016] In order to solve the problems existing in the above-mentioned prior art, the purpose of the present invention is to achieve through this application: solving the problem of calculating the first-level loss function when the vehicle shape is arbitrary, as well as the phenomenon of low accuracy and recall rate in cascade detection, and training convergence.
[0017] Specifically, the present invention provides a method for calculating a secondary loss function in vehicle detection training, the method comprising the following steps:
[0018] S1, make samples and label samples:
[0019] S1.1, the minimum enclosing rectangle of the vehicle is used as the labeling target, and all vehicles in each image are labeled;
[0020] S1.2, according to the labeled classification samples: there are four categories in total, vehicles are classified into three categories, all three categories of vehicles are positive samples, and one category of negative samples, negative samples are no vehicles;
[0021] S2, using the secondary loss function to train sample annotation:
[0022] S2.1, the first-level loss function uses the target four-classification and the coordinate two-point four-value fine-tuning, the first-level loss value is calculated by cross entropy, and the loss value in fine-tuning is calculated by 2-norm;
[0023] S2.2, the second-level loss function uses the binary classification to determine whether it is the target and the fine-tuning of the coordinates of two points and four values. The loss function value in the second-level classification is calculated by the log-likelihood function, and the loss value in the fine-tuning is calculated by the 2-norm;
[0024] S3, calculation of the secondary loss function: When calculating the entire secondary loss function, the first-level loss value accounts for 0.65, and the second-level classification loss value accounts for 0.35; in each level, the classification loss value accounts for 0.4, and the coordinate fine-tuning loss value accounts for 0.6.
[0025] The step S1.2 further comprises:
[0026] The first level of the negative sample is marked as [0,0,0], and the second level loss is marked as 0;
[0027] The vehicles are classified into three types of vehicle targets: Let the length of the vehicle target be w and the width be h;
[0028] The first type of vehicle target is when the aspect ratio When , the first-level loss is marked as [1,0,0], and the second-level loss is marked as 1;
[0029] The second type of vehicle target is when the aspect ratio When , the first-level loss is marked as [0,1,0], and the second-level loss is marked as 1;
[0030] The third type of vehicle target is when the aspect ratio When , the first-level loss is labeled as [0,0,1] and the second-level loss is labeled as 1.
[0031] The 2-norm calculation in step S2 is the square root of the sum of the squares of the absolute values of the vector elements: Suppose the fine-tuning output value of the four values of the two coordinate points is [x1, x2, x3, x4], x i ∈[x1,x2,x3,x4], N =4,1≤i≤4.
[0032] The cross entropy calculation described in step S2.1 is performed by the cross entropy cost function Obtained, where n is the number of training data, this sum covers all training inputs x, and y is the expected output.
[0033] The log-likelihood function calculation in step S2.2 is performed by the log-likelihood function C=-Σ k y k log a k Get, where a k represents the output value of the kth neuron, y k Represents the true value corresponding to the kth neuron, which can be 0 or 1.
[0034] In the cross entropy calculation or log-likelihood function calculation in step S2, when the classification outputs the correct class result, that is, the value after the output layer uses the softmax function, the softmax function is in, represents the input of the jth neuron in the Lth layer (usually the last layer), represents the output of the jth neuron in the Lth layer, and e represents a natural constant. Represents the sum of the inputs of all neurons in the Lth layer.
[0035] Therefore, the advantages of the present application are: the method of the present invention improves the accuracy of vehicle detection training through the calculation method of the secondary loss function, the method is simple and cost-saving. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.
[0037] Figure 1 is a flow chart of the method of the present invention.
[0038] Figure 2 Schematic diagram of the first type of vehicle target produced by the sample in the method of the present invention.
[0039] Figure 3 Schematic diagram of the second type of vehicle target produced by the sample in the method of the present invention.
[0040] Figure 4 Schematic diagram of the third type of vehicle target produced by the sample in the method of the present invention. DETAILED DESCRIPTION
[0041] In order to more clearly understand the technical content and advantages of the present invention, the present invention is now further described in detail in conjunction with the accompanying drawings.
[0042] like Figure 1 As shown, the present invention relates to a method for calculating a secondary loss function in vehicle detection training, the method comprising the following steps:
[0043] S1, make samples and label samples:
[0044] S1.1, the minimum enclosing rectangle of the vehicle is used as the labeling target, and all vehicles in each image are labeled;
[0045] S1.2, according to the labeled classification samples: there are four categories in total, vehicles are classified into three categories, all three categories of vehicles are positive samples, and one category of negative samples, negative samples are no vehicles;
[0046] S2, using the secondary loss function to train sample annotation:
[0047] S2.1, the first-level loss function uses the target four-classification and the coordinate two-point four-value fine-tuning, the first-level loss value is calculated by cross entropy, and the loss value in fine-tuning is calculated by 2-norm;
[0048] S2.2, the second-level loss function uses the binary classification to determine whether it is the target and the fine-tuning of the coordinates of two points and four values. The loss function value in the second-level classification is calculated by the log-likelihood function, and the loss value in the fine-tuning is calculated by the 2-norm;
[0049] S3, calculation of the secondary loss function: When calculating the entire secondary loss function, the first-level loss value accounts for 0.65, and the second-level classification loss value accounts for 0.35; in each level, the classification loss value accounts for 0.4, and the coordinate fine-tuning loss value accounts for 0.6.
[0050] The step S1.2 further comprises:
[0051] The first level of the negative sample is marked as [0,0,0], and the second level loss is marked as 0;
[0052] The vehicles are classified into three types of vehicle targets, such as Figure 2-4 As shown:
[0053] The first type of vehicle target is when the aspect ratio When , the first-level loss is marked as [1,0,0], and the second-level loss is marked as 1;
[0054] The second type of vehicle target is when the aspect ratio When , the first-level loss is marked as [0,1,0], and the second-level loss is marked as 1;
[0055] The third type of vehicle target is when the aspect ratio When , the first-level loss is labeled as [0,0,1] and the second-level loss is labeled as 1.
[0056] The 2-norm calculation in step S2 is the square root of the sum of the squares of the absolute values of the vector elements: Suppose the fine-tuning output value of the four values of the two coordinate points is [x1, x2, x3, x4], x i ∈[x1,x2,x3,x4], N=4, 1≤i≤4. 2-norm: Norm is a function with the concept of "length".
[0057] The cross entropy calculation described in step S2.1 is performed by the cross entropy cost function Obtained, where n is the number of training data, this sum covers all training inputs x, and y is the expected output. The cross-entropy cost function comes from the concept of entropy in information theory. It is a commonly used cost function in neural network classification problems (such as image classification). The cross-entropy cost function has a good explanation for the classification problem: when the classification output of the correct class result (the value after the output layer uses the softmax function) is close to 1, the label corresponding to the correct class is 1, that is, y = 1. It can be obtained that the first item in C is close to 0 and the second item is equal to 0. For the incorrect class, a is close to 0, y = 0, then the first item in C is 0, and the second item is close to 0. Therefore, C is close to 0 in the end; when the difference between the classification output of the correct class result and 1 is greater, the value of C in the above formula is greater.
[0058] The log-likelihood function calculation in step S2.2 is performed by the log-likelihood function C=-Σ k y k log a k Get, where a k represents the output value of the kth neuron, y k Indicates the true value corresponding to the kth neuron, which can be 0 or 1. The log-likelihood function is similar to the cross-entropy cost function, but only considers the loss of the correct class and does not consider the loss of the wrong class. Like the cross-entropy cost function, the log-likelihood also has a good explanation for classification: when the output value a of the correct class (the output layer only uses the value after softmax) is close to 1, y=1, and C is close to 0; when the output value a is farther from a, the C value is larger.
[0059] In the cross entropy calculation or log-likelihood function calculation in step S2, when the classification outputs the correct class result, that is, the value after the output layer uses the softmax function, the softmax function is in, represents the input of the jth neuron in the Lth layer (usually the last layer), represents the output of the jth neuron in the Lth layer, and e represents a natural constant. It represents the sum of the inputs of all neurons in the Lth layer. The most obvious feature of the softmax function is that it takes the ratio of each neuron's input to the sum of all neurons' inputs in the current layer as the output of the neuron. This makes the output easier to interpret: the larger the output value of a neuron, the more likely it is that the category corresponding to the neuron is the true category.
[0060] This method also involves a corresponding network structure, which is as follows:
[0061] 1) First level network:
[0062] The first layer input data is 47×47×1, grayscale image, the convolution kernel size is 3×3, the step size is 2, the output depth is 16, and the output result is the feature map (1) 23×23×16.
[0063] The second layer input data feature map (1) is 23×23×16, the convolution kernel size is 3×3, the step size is 2, the output depth is 16, and the output result is feature map (2) 11×11×16.
[0064] The third layer input data feature map (2) is 11×11×16, the convolution kernel size is 3×3, the step size is 2, the output depth is 16, and the output result is feature map (3) 5×5×16.
[0065] The fourth layer input data feature map (3) 5×5×16, the convolution kernel size is 3×3, the step size is 1, the output depth is 16, and the output result is feature map (6) 3×3×16.
[0066] The fifth layer inputs the data feature map (3) of 5×5×16. Remove the upper and lower values of the feature map in the width direction to obtain the feature map (4) of 5×3×16.
[0067] The sixth layer inputs the data feature map (3) of 5×5×16. By removing the values on the left and right ends of the feature map in the height direction, we get the feature map (5) of 3×5×16.
[0068] The seventh layer input data feature map (4) 5×3×16, the convolution kernel size is 3×1, the step size is 1, the output depth is 16, and the output result is feature map (7) 3×3×16.
[0069] The eighth layer input data feature map (5) 3×5×16, the convolution kernel size is 1×3, the step size is 1, the output depth is 16, and the output result is the feature map (8) 3×3×16.
[0070] The ninth layer input data feature map (6) is 3×3×16, the convolution kernel size is 3×3, the step size is 1, the output depth is 1 and 4, and the output result is the feature Figure 1 ×1×1 and 1×1×4.
[0071] The tenth layer input data feature map (7) is 3×3×16, the convolution kernel size is 3×3, the step size is 1, the output depth is 1 and 4, and the output result is the feature Figure 1 ×1×1 and 1×1×4.
[0072] The eleventh layer input data feature map (8) is 3×3×16, the convolution kernel size is 3×3, the step size is 1, the output depth is 1 and 4, and the output result is the feature Figure 1 ×1×1 and 1×1×4.
[0073] The twelfth layer combines the results of the ninth, tenth, and eleventh layers into feature maps (9) 1×1×3 and feature maps (10) 1×1×12. The twelfth layer inputs feature maps (9) 1×1×3 and feature maps (10) 1×1×12, the convolution kernel sizes are 1×1 and 1×1, the step size is 1, the output depths are 1 and 4, and the output results are feature maps (11) 1×1×1 and feature maps (12) 1×1×4. All convolutions are processed using non-aligned processing at both ends. Among them, feature maps (9) 1×1×3 and feature maps (10) 1×1×12 are used as the predicted values calculated by the first-level loss function of the first-level network, and the loss function value is calculated based on the predicted values and the labeled true values. Feature maps (11) 1×1×1 and feature maps (12) 1×1×4 are used as the predicted values calculated by the second-level loss function, and the loss function value is calculated based on the predicted values and the labeled true values.
[0074] 2) Second level network:
[0075] The initial layer input data is 49×49×1, grayscale image, the convolution kernel size is 3×3, the step size is 1, the output depth is 16, and the output result is the feature map (0) 47×47×16;
[0076] The first layer input data feature value (0) is 47×47×16, the convolution kernel size is 3×3, the step size is 2, the output depth is 32, and the output result is the feature map (1) 23×23×32;
[0077] The second layer input data feature map (1) is 23×23×32, the convolution kernel size is 3×3, the step size is 2, the output depth is 64, and the output result is feature map (2) 11×11×64;
[0078] The third layer input data feature map (2) is 11×11×64, the convolution kernel size is 3×3, the step size is 2, the output depth is 64, and the output result is feature map (3) 5×5×64;
[0079] The fourth layer input data feature map (3) 5×5×64, the convolution kernel size is 3×3, the step size is 1, the output depth is 64, and the output result is the feature map (6) 3×3×64;
[0080] The fifth layer inputs the data feature map (3) of 5×5×64, removes the upper and lower values of the feature map in the width direction, and obtains the feature map (4) of 5×3×64;
[0081] The sixth layer inputs the data feature map (3) of 5×5×64, removes the left and right values of the feature map in the height direction, and obtains the feature map (5) of 3×5×64;
[0082] The seventh layer input data feature map (4) 5×3×64, the convolution kernel size is 3×1, the step size is 1, the output depth is 64, and the output result is the feature map (7) 3×3×64;
[0083] The eighth layer input data feature map (5) 3×5×64, the convolution kernel size is 1×3, the step size is 1, the output depth is 64, and the output result is the feature map (8) 3×3×64;
[0084] The ninth layer input data feature map (6) is 3×3×64, the convolution kernel size is 3×3, the step size is 1, the output depth is 1 and 4, and the output result is the feature Figure 1 ×1×1 and 1×1×4;
[0085] The tenth layer input data feature map (7) is 3×3×64, the convolution kernel size is 3×3, the step size is 1, the output depth is 1 and 4, and the output result is the feature Figure 1 ×1×1 and 1×1×4;
[0086] The eleventh layer input data feature map (8) is 3×3×64, the convolution kernel size is 3×3, the step size is 1, the output depth is 1 and 4, and the output result is the feature Figure 1 ×1×1 and 1×1×4;
[0087] The twelfth layer combines the results of the ninth, tenth, and eleventh layers into feature maps (9) 1×1×3 and feature maps (10) 1×1×12;
[0088] The twelfth layer inputs feature map (9) 1×1×3 and feature map (10) 1×1×12, the convolution kernel sizes are 1×1 and 1×1, the step size is 1, the output depth is 1 and 4, and the output results are feature map (11) 1×1×1 and feature map (12) 1×1×4;
[0089] All convolutions are processed using non-aligned ends.
[0090] Among them, feature map (9) 1×1×3 and feature map (10) 1×1×12 are used as the predicted values for the first-level loss function calculation of the second-level network, and the loss function value is calculated based on the predicted values and the labeled true values; feature map (11) 1×1×1 and feature map (12) 1×1×4 are used as the predicted values for the second-level loss function calculation, and the loss function value is calculated based on the predicted values and the labeled true values.
[0091] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A method for calculating a secondary loss function in vehicle detection training, characterized in that: The method comprises the following steps: S1, make samples and label samples: S1.1, the minimum enclosing rectangle of the vehicle is used as the labeling target, and all vehicles in each image are labeled; S1.2, according to the labeled classification samples: there are four categories in total, vehicles are classified into three categories, all three categories of vehicles are positive samples, and one category of negative samples, negative samples are no vehicles; The step S1.2 further comprises: The first level of the negative sample is marked as [0,0,0], and the second level loss is marked as 0; The vehicles are classified into three types of vehicle targets: Let the length of the vehicle target be w and the width be h; The first type of vehicle target is when the aspect ratio When , the first-level loss is marked as [1,0,0], and the second-level loss is marked as 1; The second type of vehicle target is when the aspect ratio When , the first-level loss is marked as [0,1,0], and the second-level loss is marked as 1; The third type of vehicle target is when the aspect ratio When , the first-level loss is marked as [0,0,1], and the second-level loss is marked as 1; S2, using the secondary loss function to train sample annotation: S2.1, the first-level loss function uses the target four-classification and the coordinate two-point four-value fine-tuning, the first-level loss value is calculated by cross entropy, and the loss value in fine-tuning is calculated by 2-norm; S2.2, the second-level loss function uses the binary classification to determine whether it is the target and the fine-tuning of the coordinates of two points and four values. The loss function value in the second-level classification is calculated by the log-likelihood function, and the loss value in the fine-tuning is calculated by the 2-norm; S3, calculation of the secondary loss function: When calculating the entire secondary loss function, the first-level loss value accounts for 0.65, and the second-level classification loss value accounts for 0.35; in each level, the classification loss value accounts for 0.4, and the coordinate fine-tuning loss value accounts for 0.
6.
2. The method for calculating the secondary loss function in vehicle detection training according to claim 1, characterized in that: The 2-norm calculation in step S2 is the square root of the sum of the squares of the absolute values of the vector elements: Suppose the fine-tuning output value of the four values of the two coordinate points is [x1, x2, x3, x4, x i ∈[x1,x2,x3,x4], N=4, 1≤i≤4.
3. The method for calculating the secondary loss function in vehicle detection training according to claim 1, characterized in that: The cross entropy calculation described in step S2.1 is performed by the cross entropy cost function Obtained, where n is the number of training data, this sum covers all training inputs x, and y is the expected output.
4. The method for calculating the secondary loss function in vehicle detection training according to claim 1, characterized in that: The log-likelihood function calculation in step S2.2 is performed by the log-likelihood function C=-∑ k y k log a k Get, where a k represents the output value of the kth neuron, y k Represents the true value corresponding to the kth neuron, which can be 0 or 1.
5. The method for calculating the secondary loss function in vehicle detection training according to claim 1, characterized in that: In the cross entropy calculation or log-likelihood function calculation in S2, when the classification outputs the correct class result, that is, the value after the output layer uses the softmax function, the softmax function is in, represents the input of the jth neuron in the Lth layer, represents the output of the jth neuron in the Lth layer, e represents a natural constant, Represents the sum of the inputs of all neurons in the Lth layer.
