Vehicle detection model training method, device, storage medium and apparatus
By supervising the training of student models using a teacher model and optimizing the vehicle detection model using a distillation loss function, the problem of balancing accuracy and speed in vehicle detection models on the terminal is solved, achieving high-precision and high-speed vehicle detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING 360 INTELLIGENT TECHNOLOGY CO LTD
- Filing Date
- 2020-10-22
- Publication Date
- 2026-06-09
AI Technical Summary
In existing technologies, vehicle detection models struggle to balance accuracy and speed at vehicle terminals, particularly in driving image detection where insufficient accuracy and slow speed are prevalent.
The transfer learning approach is adopted, in which the training process of the student model is supervised by the teacher model. The loss parameters between the student model and the teacher model are calculated using a preset distillation loss function, and the training parameters of the student detection model are updated. The student model is optimized by combining the loss parameters of feature maps and heatmaps to improve its detection accuracy and speed.
While maintaining the high speed of the student model, the detection accuracy of the vehicle detection model has been significantly improved, ensuring that the vehicle terminal can achieve both high accuracy and high speed in driving image recognition.
Smart Images
Figure CN114463573B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of vehicle assisted driving technology, and in particular to a vehicle detection model training method, equipment, storage medium and device. Background Technology
[0002] Among automotive assistance functions, collision warning is the most common. The first step in collision warning is vehicle detection. To accurately determine the position of the vehicle ahead, it's necessary to predict its location in the video feed in a timely and precise manner. Therefore, improving the accuracy and speed of vehicle image detection at the vehicle's terminal is a technical challenge that requires continuous breakthroughs in assisted driving.
[0003] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention
[0004] The main objective of this invention is to provide a vehicle detection model training method, equipment, storage medium, and device, which aims to improve the accuracy and speed of vehicle image detection at vehicle terminals.
[0005] To achieve the above objectives, the present invention provides a vehicle detection model training method, which includes the following steps:
[0006] The vehicle sample images are identified using a pre-set student detection model to obtain the first feature;
[0007] The vehicle sample images are identified using a pre-set teacher detection model to obtain a second feature;
[0008] Calculate the loss parameters between the first feature and the second feature according to a preset distillation loss function;
[0009] The training parameters in the preset student detection model are updated according to the loss parameters to train the student detection model and obtain the vehicle detection model.
[0010] Optionally, the first feature includes a first output feature map and a first distribution heatmap, and the second feature includes a second output feature map and a second distribution heatmap. The step of calculating the loss parameter between the first output feature and the second output feature according to a preset distillation loss function includes:
[0011] Calculate the feature map loss parameters between the first output feature map and the second output feature map according to the preset distillation loss function;
[0012] The heat map loss parameters between the first distribution heat map and the second distribution heat map are calculated based on the preset distillation loss function;
[0013] The loss parameters are determined based on the feature map loss parameters and the heat map loss parameters.
[0014] Optionally, determining the loss parameters based on the feature map loss parameters and the heatmap loss parameters includes:
[0015] Obtain preset feature marker information from the vehicle sample image;
[0016] The true loss parameter between the second distribution heatmap and the preset feature labeling information is calculated based on the preset true loss function;
[0017] The loss parameters are determined based on the feature map loss parameters, the heatmap loss parameters, and the actual loss parameters.
[0018] Optionally, before calculating the feature map loss parameters between the first output feature map and the second output feature map according to the preset distillation loss function, the method further includes:
[0019] Select the corresponding distillation temperature coefficient from the first preset coefficient table based on the actual loss parameters;
[0020] A preset distillation loss function is constructed based on the preset error function and the distillation temperature coefficient.
[0021] Optionally, determining the loss parameter based on the feature map loss parameter, the heatmap loss parameter, and the true loss parameter includes:
[0022] Select the corresponding distillation correction coefficient from the second preset coefficient table based on the actual loss parameters;
[0023] The feature map loss parameter and the thermogram loss parameter are corrected according to the distillation correction coefficient to obtain the distillation loss parameter;
[0024] The loss parameters are determined based on the distillation loss parameters and the actual loss parameters.
[0025] Optionally, the distribution heatmap includes a classification heatmap, a regression box width and height heatmap, and a center offset heatmap; the step of calculating the heatmap loss parameter between the first distribution heatmap and the second distribution heatmap according to the preset distillation loss function includes:
[0026] The classification loss parameter between the first classification heatmap and the second classification heatmap is calculated based on the preset distillation loss function;
[0027] The regression box loss parameters between the first regression box width and height heatmap and the second regression box width and height heatmap are calculated based on the preset distillation loss function.
[0028] The center offset loss parameter between the first center offset heatmap and the second center offset heatmap is calculated based on the preset distillation loss function;
[0029] The heatmap loss parameters are determined based on the classification loss parameters, the regression box loss parameters, and the center offset loss parameters.
[0030] Optionally, determining the heatmap loss parameter based on the classification loss parameter, the regression box loss parameter, and the center offset loss parameter includes:
[0031] Obtain the weight ratios corresponding to the classification loss parameter, the regression box loss parameter, and the center offset loss parameter; wherein the weight ratios corresponding to the regression box loss parameter and the center offset loss parameter are both less than the weight ratios corresponding to the classification loss parameter.
[0032] The heatmap loss parameters are obtained by calculating the classification loss parameters, the regression box loss parameters, and the center offset loss parameters based on the weight ratio.
[0033] Optionally, the step of identifying vehicle sample images using a preset student detection model to obtain the first feature includes:
[0034] Vehicle sample images are identified using a preset student detection model, and the output features of each output layer in the preset student detection model are obtained.
[0035] When the number of output features of each output layer in the preset student detection model is greater than the preset feature number threshold, the output features of each output layer in the preset student detection model are taken as the first feature.
[0036] Optionally, the step of identifying the vehicle sample image using a preset teacher detection model to obtain the second feature includes:
[0037] Vehicle sample images are identified using a pre-set teacher detection model, and the output features of each output layer in the teacher detection model are obtained.
[0038] When the number of output features of each output layer in the teacher detection model is greater than the preset feature number threshold, the output features of each output layer in the teacher detection model are used as the second feature.
[0039] Optionally, before calculating the loss parameter between the first feature and the second feature according to the preset distillation loss function, the method further includes:
[0040] Determine whether the number of features of the first feature is equal to the number of features of the second feature;
[0041] When the number of features of the first feature is not equal to the number of features of the second feature, the same number of output features are selected from the output features of each output layer in the teacher detection model according to the number of features of the first feature, and the adjusted second feature is obtained.
[0042] Accordingly, the step of calculating the loss parameter between the first feature and the second feature according to the preset distillation loss function includes:
[0043] The loss parameters between the first feature and the adjusted second feature are calculated based on a preset distillation loss function.
[0044] Optionally, before obtaining the first feature by recognizing the vehicle sample image using a preset student detection model, the method further includes:
[0045] Obtain a first target detection network and a second target detection network; wherein the number of parameters in the first target detection network is less than the number of parameters in the second target detection network;
[0046] A vehicle image training set is obtained, and the first target detection network and the second target detection network are trained using the vehicle image training set to obtain the trained first target detection network and the trained second target detection network.
[0047] When the trained first object detection network satisfies the first preset convergence condition, the trained first object detection network is used as the preset student detection model.
[0048] When the second target detection network after training satisfies the second preset convergence condition, the trained second target detection network is used as the preset teacher detection model.
[0049] Optionally, updating the training parameters in the preset student detection model according to the loss parameters to train the student detection model and obtain a vehicle detection model includes:
[0050] Update the training parameters in the preset student detection model according to the loss parameters to obtain the adjusted student detection model;
[0051] Obtain the loss parameters corresponding to the adjusted student detection model;
[0052] When the loss parameters corresponding to the adjusted student detection model are within a preset range, the adjusted student detection model is used as the vehicle detection model.
[0053] Optionally, after updating the training parameters in the preset student detection model according to the loss parameters to train the student detection model and obtain the vehicle detection model, the method further includes:
[0054] The vehicle image to be detected is acquired, and the vehicle detection model is used to detect the vehicle image to obtain vehicle driving parameters.
[0055] The vehicle driving parameters are compared with preset thresholds, and an alarm is triggered when the vehicle driving parameters reach the preset thresholds.
[0056] Furthermore, to achieve the above objectives, the present invention also proposes a vehicle detection model training device, the vehicle detection model training device comprising:
[0057] The feature extraction module is used to identify vehicle sample images using a preset student detection model to obtain the first feature;
[0058] The feature extraction module is also used to identify the vehicle sample image through a preset teacher detection model to obtain a second feature;
[0059] The loss calculation module is used to calculate the loss parameters between the first feature and the second feature according to a preset distillation loss function;
[0060] The parameter adjustment module is used to update the training parameters in the preset student detection model according to the loss parameters, so as to train the student detection model and obtain the vehicle detection model.
[0061] Optionally, the first feature includes a first output feature map and a first distribution heatmap, and the second feature includes a second output feature map and a second distribution heatmap;
[0062] The loss calculation module is also used to calculate the feature map loss parameters between the first output feature map and the second output feature map according to a preset distillation loss function;
[0063] The loss calculation module is also used to calculate the heat map loss parameters between the first distribution heat map and the second distribution heat map according to the preset distillation loss function;
[0064] The loss calculation module is also used to determine the loss parameters based on the feature map loss parameters and the heat map loss parameters.
[0065] Optionally, the feature extraction module is further configured to identify vehicle sample images using a preset student detection model and obtain the output features of each output layer in the preset student detection model.
[0066] The feature extraction module is further configured to use the output feature as the first feature when the number of features in the output feature is greater than a preset feature number threshold.
[0067] Optionally, the feature extraction module is further configured to identify vehicle sample images using a preset teacher detection model and obtain the output features of each output layer in the teacher detection model;
[0068] The feature extraction module is further configured to use the output feature as a second feature when the number of features in the output feature is greater than the preset feature number threshold.
[0069] Optionally, the parameter adjustment module is further configured to update the training parameters in the preset student detection model according to the loss parameters, so as to obtain the adjusted student detection model;
[0070] The parameter adjustment module is also used to obtain the loss parameters corresponding to the adjusted student detection model;
[0071] The parameter adjustment module is also used to use the adjusted student detection model as a vehicle detection model when the loss parameter corresponding to the adjusted student detection model is within a preset range.
[0072] Furthermore, to achieve the above objectives, the present invention also proposes a vehicle detection model training device, which includes: a memory, a processor, and a vehicle detection model training program stored in the memory and executable on the processor. When the vehicle detection model training program is executed by the processor, it implements the steps of the vehicle detection model training method described above.
[0073] Furthermore, to achieve the above objectives, the present invention also proposes a storage medium storing a vehicle detection model training program, wherein the vehicle detection model training program, when executed by a processor, implements the steps of the vehicle detection model training method described above.
[0074] In this invention, during the training of the vehicle detection model mounted on the vehicle terminal, a teacher model supervises the training process of the student model to achieve transfer learning. After the student model converges, the final vehicle detection model is obtained. Since the teacher model has higher detection accuracy than the student model, the detection accuracy of the student model is improved. At the same time, the student model's inherent speed advantage is retained, allowing the final vehicle detection model to balance speed and accuracy. This results in the vehicle terminal achieving both high speed and accuracy when identifying vehicles using the vehicle detection model. Attached Figure Description
[0075] Figure 1This is a schematic diagram of the structure of a vehicle detection model training device in the hardware operating environment involved in the embodiments of the present invention;
[0076] Figure 2 This is a flowchart illustrating the first embodiment of the vehicle detection model training method of the present invention;
[0077] Figure 3 This is a flowchart illustrating the second embodiment of the vehicle detection model training method of the present invention;
[0078] Figure 4 This is a flowchart illustrating the third embodiment of the vehicle detection model training method of the present invention;
[0079] Figure 5 This is a flowchart illustrating the fourth embodiment of the vehicle detection model training method of the present invention;
[0080] Figure 6 This is a structural block diagram of the first embodiment of the vehicle detection model training device of the present invention.
[0081] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0082] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0083] Reference Figure 1 , Figure 1 This is a schematic diagram of the vehicle detection model training device structure in the hardware operating environment involved in the embodiments of the present invention.
[0084] like Figure 1As shown, the vehicle detection model training device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to establish communication between these components. The user interface 1003 may include a display screen, and optionally, it may also include a standard wired interface or a wireless interface. In this invention, the wired interface of the user interface 1003 may be a USB interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed random access memory (RAM) or a non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.
[0085] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the vehicle detection model training device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.
[0086] like Figure 1 As shown, the memory 1005, which is identified as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a vehicle detection model training program.
[0087] exist Figure 1 In the vehicle detection model training device shown, the network interface 1004 is mainly used to connect to the backend server and communicate data with the backend server; the user interface 1003 is mainly used to connect to the user device; the vehicle detection model training device calls the vehicle detection model training program stored in the memory 1005 through the processor 1001 and executes the vehicle detection model training method provided in this embodiment of the invention.
[0088] Based on the above hardware structure, an embodiment of the vehicle detection model training method of the present invention is proposed.
[0089] Reference Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the vehicle detection model training method of the present invention, which presents the first embodiment of the vehicle detection model training method of the present invention.
[0090] In the first embodiment, the vehicle detection model training method includes the following steps:
[0091] Step S10: Recognize the vehicle sample image using a preset student detection model to obtain the first feature.
[0092] It should be understood that the execution entity in this embodiment is the vehicle detection model training device, which has functions such as image processing, data analysis, and program execution. The vehicle detection model training device can be a computer, server, or other computer equipment, or a vehicle terminal equipped with such a computer equipment. Of course, other devices with similar functions can also be used, and this embodiment does not limit them.
[0093] Understandably, considering the real-time requirements of driver assistance functions, vehicle detection models are typically computed on the local vehicle terminal. However, due to limitations in local computing power, vehicle detection models usually employ lightweight networks, which may result in lower accuracy.
[0094] It should be noted that the preset student detection model can be a lightweight neural network with a small number of parameters, which ensures that the converged vehicle detection model can be computed on the local vehicle terminal.
[0095] It should be noted that vehicle sample images can include images of vehicles on the road, which can be obtained by capturing images from a vehicle's dashcam. Of course, other types of vehicle images can also be used. The process of the preset student detection model recognizing vehicle sample images can include features extraction and classification of the vehicle images. The first feature can include the extracted feature map or classification results, etc.
[0096] Step S20: The vehicle sample image is identified using a preset teacher detection model to obtain the second feature.
[0097] Understandably, to ensure the effective supervised training of the pre-set teacher detection model on the pre-set student detection model, the pre-set teacher detection model can have high accuracy, such as 95%. This also results in a large number of parameters in the pre-set teacher detection model. During supervised training of the pre-set student detection model, the parameters of the teacher detection model are kept constant.
[0098] It should be noted that the process by which the pre-defined teacher detection model identifies vehicle sample images may include features extraction and classification of the vehicle images. This second feature may include the extracted feature map or classification results.
[0099] It should be noted that, to improve training speed, neural networks can be pre-trained separately to obtain a pre-converged preset student detection model and a preset teacher detection model. In specific implementation, a first target detection network and a second target detection network are first obtained; wherein the number of parameters in the first target detection network is less than the number of parameters in the second target detection network; a vehicle image training set is obtained, and the first target detection network and the second target detection network are trained using the vehicle image training set respectively to obtain the trained first target detection network and the trained second target detection network; when the trained first target detection network meets a first preset convergence condition, the trained first target detection network is used as the preset student detection model; when the trained second target detection network meets a second preset convergence condition, the trained second target detection network is used as the preset teacher detection model.
[0100] It should be noted that the preset student detection model and the preset teacher detection model can adopt anchor-free (object detection) series detection models. In specific implementations, the preset student detection model can use the MobileNet-V2 network, and the preset teacher detection model can use the ResNet152 network. Of course, other networks can also be used, and this implementation method does not limit them.
[0101] Understandably, to ensure the effective supervised training of the preset student detection model by the preset teacher detection model, the prediction accuracy of the preset teacher detection model can be greater than that of the preset student detection model. For example, the prediction accuracy of the preset teacher detection model can be 95%, and the prediction accuracy of the preset student detection model can be 70%. The first and second preset convergence conditions can be prediction accuracy or the number of iterations. Of course, the preset convergence conditions can be freely set according to user needs, and this implementation does not impose any restrictions on them.
[0102] Step S30: Calculate the loss parameter between the first feature and the second feature according to the preset distillation loss function.
[0103] It should be noted that calculating the loss parameter between the first feature and the second feature according to the preset distillation loss function can be achieved by calculating the difference between the first feature and the second feature and using the difference as the loss parameter. For example, the loss parameter can be obtained by subtracting the eigenvalue matrix corresponding to the first feature from the eigenvalue matrix corresponding to the second feature. Of course, there are other methods as well, and this embodiment does not limit them.
[0104] Understandably, before calculating the difference between the first feature and the second feature, a distillation operation needs to be performed on the first feature and / or the second feature. The distillation operation refers to adjusting the entropy of the feature distribution using a preset distillation temperature coefficient. The specific adjustment effect depends on the selected distillation temperature coefficient.
[0105] Step S40: Update the training parameters in the preset student detection model according to the loss parameters to train the student detection model and obtain the vehicle detection model.
[0106] Understandably, the loss function can reflect the gap between the model's prediction and the actual result, and thus adjust the training parameters in the model according to this gap; where the training parameters can be weights, etc.
[0107] It should be noted that after updating the training parameters in the preset student detection model, training is repeated in the above manner. By continuously updating the training parameters, the preset student detection model converges, thereby obtaining the vehicle detection model. In specific implementation, the training parameters in the preset student detection model can be updated according to the loss parameters to obtain an adjusted student detection model; the loss parameters corresponding to the adjusted student detection model can be obtained; when the loss parameters corresponding to the adjusted student detection model are within a preset range, the adjusted student detection model is used as the vehicle detection model. Of course, the convergence condition can also be set to the number of iterations, and the vehicle detection model is obtained when the number of iterations is reached.
[0108] Understandably, after obtaining the vehicle detection model, the vehicle terminal can use this model to identify real-time driving footage, providing basic information for vehicle warnings. In specific implementation, an image of the vehicle to be detected is acquired, and the vehicle detection model is used to detect the image to obtain vehicle driving parameters. These parameters are then compared with a preset threshold, and an alarm is triggered when the parameters reach the preset threshold.
[0109] It should be noted that vehicle driving parameters can include information such as vehicle type, vehicle speed, and distance to other vehicles. The vehicle terminal can also determine its own distance from the vehicle in front based on the vehicle driving parameters, predict the probability of collision based on its own vehicle speed, and issue an alarm when the probability of collision exceeds a threshold.
[0110] In the first embodiment, when training the vehicle detection model mounted on the vehicle terminal, the training process of the student model is supervised by a teacher model to achieve transfer learning. After the student model converges, the vehicle detection model is obtained. Since the teacher model has higher detection accuracy than the student model, the detection accuracy of the student model is improved. At the same time, the student model's inherent speed advantage is retained, allowing the final vehicle detection model to balance speed and accuracy. This results in the vehicle terminal achieving both high speed and accuracy when identifying vehicles using the vehicle detection model.
[0111] Reference Figure 3 , Figure 3 This is a flowchart illustrating the second embodiment of the vehicle detection model training method of the present invention. Based on the first embodiment described above, a second embodiment of the vehicle detection model training method of the present invention is proposed.
[0112] In the second embodiment, the first feature includes a first output feature map and a first distribution heatmap, and the second feature includes a second output feature map and a second distribution heatmap. Step S30 includes:
[0113] Step S301: Calculate the feature map loss parameters between the first output feature map and the second output feature map according to the preset distillation loss function.
[0114] It is understandable that neural networks typically have multiple output layers, each with a corresponding output feature map. This embodiment, when calculating the loss parameters for the preset teacher detection model and the preset student detection model, includes calculating the difference between their output feature maps, thereby better reflecting the gap between the preset teacher detection model and the preset student detection model.
[0115] Step S302: Calculate the heat map loss parameters between the first distribution heat map and the second distribution heat map according to the preset distillation loss function.
[0116] It is understandable that the heatmap represents the final detection result of the detection model, reflecting the distribution of the detection results. This implementation method, when calculating the loss parameters for the preset teacher detection model and the preset student detection model, also includes calculating the difference between their output heatmaps, thereby better reflecting the gap between the preset teacher detection model and the preset student detection model.
[0117] Step S303: Determine the loss parameters based on the feature map loss parameters and the heat map loss parameters.
[0118] It should be noted that determining the loss parameters based on the feature map loss parameters and the heatmap loss parameters can be achieved by adding the feature map loss parameters and the heatmap loss parameters together. This embodiment considers both the feature map and the heatmap when calculating the distillation loss between the preset teacher detection model and the preset student detection model, enabling the preset student detection model to converge faster and achieve higher accuracy.
[0119] In a specific implementation, step S303 includes: acquiring preset feature label information of the vehicle sample image; calculating the true loss parameter between the second distribution heatmap and the preset feature label information according to a preset true loss function; and determining the loss parameter according to the feature map loss parameter, the heatmap loss parameter, and the true loss parameter.
[0120] Understandably, since the teacher detection model itself is not absolutely accurate, errors in the teacher detection model are incorporated into the final loss parameters to prevent them from affecting the accuracy of the pre-set student detection model. This ensures the accuracy of the pre-set student detection model. The pre-set feature labeling information is manually labeled information.
[0121] It should be noted that, in order to accelerate the convergence speed and accuracy of the preset student detection model, an appropriate distillation coefficient is selected when constructing the distillation function, taking into account the magnitude of the true loss parameter. In specific implementation, the corresponding distillation temperature coefficient is selected from the first preset coefficient table according to the true loss parameter; the preset distillation loss function is constructed based on the preset error function and the distillation temperature coefficient.
[0122] It should be noted that the larger the actual loss parameter, the larger the distillation temperature coefficient can be. The distillation temperature coefficient can be freely set according to user needs, such as 0.005, 1, or 20. The preset error function can be a difference function, an average error function, or a mean square error function; this implementation does not impose any restrictions on this.
[0123] Simultaneously, corresponding distillation correction coefficients can be determined based on the actual loss parameters (feature map loss parameters and heat map loss parameters). In specific implementation, the corresponding distillation correction coefficients are selected from the second preset coefficient table based on the actual loss parameters; the feature map loss parameters and the heat map loss parameters are corrected according to the distillation correction coefficients to obtain the distillation loss parameters; and the loss parameters are determined based on the distillation loss parameters and the actual loss parameters.
[0124] It should be noted that the larger the true loss parameter, the larger the distillation correction coefficient can be. To avoid the influence of errors in the pre-set teacher detection model, the distillation loss parameter can be set to less than 1. Different distillation correction coefficients can be determined for the feature map loss parameter and the heatmap loss parameter, or they can be the same. For example, the distillation correction coefficient for the feature map loss parameter can be 0.3, and the distillation correction coefficient for the heatmap loss parameter can be 0.5; or both can be 0.3.
[0125] In the second embodiment, the first feature includes a first output feature map and a first distribution heatmap, and the second feature includes a second output feature map and a second distribution heatmap. This embodiment considers both feature maps and heatmaps, enabling the preset student detection model to converge faster and with higher accuracy. Furthermore, different distillation temperature coefficients and distillation correction coefficients are set according to the true loss parameters of the preset student detection model, thereby improving the convergence speed and accuracy of the preset student model.
[0126] Reference Figure 4 , Figure 4 This is a flowchart illustrating a third embodiment of the vehicle detection model training method of the present invention. Based on the first and second embodiments described above, a third embodiment of the vehicle detection model training method of the present invention is proposed. This embodiment is described based on the second embodiment.
[0127] In the third embodiment, the distribution heatmap includes a classification heatmap, a regression box width and height heatmap, and a center offset heatmap; step S302 includes:
[0128] Step S3021: Calculate the classification loss parameter between the first classification heatmap and the second classification heatmap according to the preset distillation loss function.
[0129] Understandably, classification heatmaps typically include the probability of a feature belonging to a specific type. For example, feature one might have a 70% probability of belonging to a vehicle, a 20% probability of belonging to a person, and a 10% probability of belonging to a building. Heatmaps provide a more comprehensive reflection of the detection model's performance. In this embodiment, the method specifically utilizes classification heatmaps when calculating the distillation loss parameters between the preset teacher detection model and the preset student detection model, effectively improving the accuracy of the preset student detection model.
[0130] Step S3022: Calculate the regression box loss parameter between the first regression box width and height heatmap and the second regression box width and height heatmap according to the preset distillation loss function.
[0131] Understandably, the detection model can label features using regression boxes, and the size of the regression boxes reflects the accuracy of the detection model in recognizing the features. In this embodiment, the method also involves a heatmap of the regression box width and height when calculating the distillation loss parameters between the preset teacher detection model and the preset student detection model, which can effectively improve the accuracy of the preset student detection model.
[0132] Step S3023: Calculate the center offset loss parameter between the first center offset heatmap and the second center offset heatmap according to the preset distillation loss function.
[0133] Understandably, the center offset heatmap reflects the central position of a feature, and thus also reflects the accuracy of the detection model in recognizing that feature. In this embodiment, the center offset heatmap is also used when calculating the distillation loss parameter between the preset teacher detection model and the preset student detection model, which can effectively improve the accuracy of the preset student detection model.
[0134] Step S3024: Determine the heatmap loss parameters based on the classification loss parameters, the regression box loss parameters, and the center offset loss parameters.
[0135] It should be noted that determining the heatmap loss parameter based on the classification loss parameter, the regression box loss parameter, and the center offset loss parameter can be achieved by adding the classification loss parameter, the regression box loss parameter, and the center offset loss parameter together to obtain the heatmap loss parameter.
[0136] Understandably, for object detection models, recognizing features is generally not difficult, while classifying features is more challenging. The preset student detection model is more likely to show significant differences from the preset teacher detection model on the classification heatmap. To address these issues, this implementation method amplifies the impact of the classification heatmap.
[0137] In a specific implementation, step S3024 may include: obtaining the weight ratios corresponding to the classification loss parameter, the regression box loss parameter, and the center offset loss parameter; wherein the weight ratios corresponding to the regression box loss parameter and the center offset loss parameter are both less than the weight ratio corresponding to the classification loss parameter; and calculating the classification loss parameter, the regression box loss parameter, and the center offset loss parameter based on the weight ratios to obtain the heatmap loss parameter.
[0138] It is understandable that by setting a higher weight ratio for the classification loss parameters, the proportion of the classification loss parameters in the heatmap loss parameters is increased, thereby making the pre-set student detection model pay more attention to the classification heatmap during the training process, thus improving the accuracy of the converged vehicle detection model.
[0139] In the third embodiment, the distribution heatmap includes a classification heatmap, a regression box width and height heatmap, and a center offset heatmap. The heatmap loss parameters have a wider coverage, which is beneficial for improving the accuracy of the converged vehicle detection model. Simultaneously, considering the difficulty of the classification problem, a higher proportion is set for the classification loss parameters, causing the pre-set student detection model to focus more on the classification heatmap during training, further improving the accuracy of the converged vehicle detection model.
[0140] Reference Figure 5 , Figure 5 This is a flowchart illustrating the fourth embodiment of the vehicle detection model training method of the present invention. Based on the first, second, and third embodiments described above, a fourth embodiment of the vehicle detection model training method of the present invention is proposed. This embodiment is described based on the first embodiment.
[0141] In the fourth embodiment, step S10 includes:
[0142] Step S101: Recognize the vehicle sample image using a preset student detection model, and obtain the output features of each output layer in the preset student detection model.
[0143] Understandably, neural networks typically have multiple output layers, each with a corresponding output feature map. To improve the calculation effect of distillation loss on feature maps, this implementation acquires the output features of each output layer in the preset student detection model to obtain more feature maps.
[0144] Step S102: When the number of output features of each output layer in the preset student detection model is greater than the preset feature number threshold, the output features of each output layer in the preset student detection model are taken as the first feature.
[0145] It should be noted that the preset feature quantity threshold determines the number of feature maps; a larger preset feature quantity threshold results in more selected feature maps, and vice versa. The preset feature quantity threshold can be freely set according to the user's needs. This implementation does not impose any restrictions on this.
[0146] In the fourth embodiment, step S20 includes:
[0147] Step S201: Recognize the vehicle sample image using a preset teacher detection model, and obtain the output features of each output layer in the teacher detection model.
[0148] Understandably, the output features of the pre-set teacher detection model need to correspond with those of the pre-set student detection model in order to calculate the distillation loss.
[0149] Step S202: When the number of output features of each output layer in the teacher detection model is greater than the preset feature number threshold, the output features of each output layer in the teacher detection model are used as the second feature.
[0150] Understandably, setting the same preset feature quantity threshold for the preset teacher detection model and the preset student model can ensure that the number of features in the selected feature maps is more matched and easier to calculate.
[0151] In a specific implementation, before calculating the loss parameter between the first feature and the second feature according to the preset distillation loss function, the method further includes: determining whether the number of features of the first feature is equal to the number of features of the second feature; when the number of features of the first feature is not equal to the number of features of the second feature, selecting output features with the same number of features from the output features of each output layer in the teacher detection model according to the number of features of the first feature, and obtaining the adjusted second feature; correspondingly, calculating the loss parameter between the first feature and the second feature according to the preset distillation loss function includes: calculating the loss parameter between the first feature and the adjusted second feature according to the preset distillation loss function.
[0152] It should be noted that due to differences in the number and dimensions of output layers between the preset teacher detection model and the preset student detection model, there may be a mismatch between the selected first and second features, requiring further filtering. For example, if the preset feature number threshold is set to 32 dimensions, the preset student detection model may select first features with 32, 48, and 64 dimensions; however, due to output settings limitations of the preset teacher detection model, its default output layer count is only 32 and 64 dimensions. In this case, it is necessary to search the output layers of the convolutional layers in the preset teacher detection model to select the corresponding 48-dimensional feature map. The above values are merely examples; other values are also possible, and this embodiment does not impose any restrictions on them.
[0153] In the fourth embodiment, the output feature maps of the preset student detection model and the preset teacher detection model are selected according to the preset feature quantity threshold to obtain more feature maps, so that the feature map loss parameters can better reflect the differences between the preset student detection model and the preset teacher detection model.
[0154] Furthermore, this embodiment of the invention also proposes a storage medium storing a vehicle detection model training program, which, when executed by a processor, implements the steps of the vehicle detection model training method described above.
[0155] Since this storage medium adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought about by the technical solutions of the above embodiments, which will not be repeated here.
[0156] Furthermore, this invention also proposes a vehicle detection model training device, referring to... Figure 6 , Figure 6 This is a structural block diagram of the first embodiment of the vehicle detection model training device of the present invention.
[0157] In this embodiment, the vehicle detection model training device includes:
[0158] The feature extraction module 10 is used to identify vehicle sample images using a preset student detection model to obtain the first feature.
[0159] The feature extraction module 10 is also used to identify the vehicle sample image through a preset teacher detection model to obtain a second feature.
[0160] The loss calculation module 20 is used to calculate the loss parameters between the first feature and the second feature according to a preset distillation loss function.
[0161] The parameter adjustment module 30 is used to update the training parameters in the preset student detection model according to the loss parameters, so as to train the student detection model and obtain the vehicle detection model.
[0162] In this embodiment, when training the vehicle detection model mounted on the vehicle terminal, a teacher model supervises the training process of the student model to achieve transfer learning. After the student model converges, the vehicle detection model is obtained. Since the teacher model has higher detection accuracy than the student model, the detection accuracy of the student model is improved. At the same time, the student model's inherent speed advantage is retained, allowing the final vehicle detection model to balance speed and accuracy. This results in the vehicle terminal achieving both high speed and accuracy when identifying vehicles using the vehicle detection model.
[0163] In one embodiment, the first feature includes a first output feature map and a first distribution heatmap, and the second feature includes a second output feature map and a second distribution heatmap.
[0164] The loss calculation module 20 is further configured to calculate the feature map loss parameter between the first output feature map and the second output feature map according to a preset distillation loss function; calculate the heat map loss parameter between the first distribution heat map and the second distribution heat map according to the preset distillation loss function; and determine the loss parameter according to the feature map loss parameter and the heat map loss parameter.
[0165] In one embodiment, the loss calculation module 20 is further configured to acquire preset feature label information of the vehicle sample image; calculate the true loss parameter between the second distribution heatmap and the preset feature label information according to a preset true loss function; and determine the loss parameter according to the feature map loss parameter, the heatmap loss parameter and the true loss parameter.
[0166] In one embodiment, the loss calculation module 20 is further configured to select a corresponding distillation temperature coefficient from a first preset coefficient table based on the actual loss parameters; and to construct a preset distillation loss function based on a preset error function and the distillation temperature coefficient.
[0167] In one embodiment, the loss calculation module 20 is further configured to select a corresponding distillation correction coefficient from a second preset coefficient table based on the actual loss parameter; correct the feature map loss parameter and the heat map loss parameter respectively based on the distillation correction coefficient to obtain the distillation loss parameter; and determine the loss parameter based on the distillation loss parameter and the actual loss parameter.
[0168] In one embodiment, the distribution heatmap includes a classification heatmap, a regression box width-height heatmap, and a center offset heatmap. The loss calculation module 20 is further configured to: calculate a classification loss parameter between the first classification heatmap and the second classification heatmap according to the preset distillation loss function; calculate a regression box loss parameter between the first regression box width-height heatmap and the second regression box width-height heatmap according to the preset distillation loss function; calculate a center offset loss parameter between the first center offset heatmap and the second center offset heatmap according to the preset distillation loss function; and determine heatmap loss parameters based on the classification loss parameter, the regression box loss parameter, and the center offset loss parameter.
[0169] In one embodiment, the loss calculation module 20 is further configured to obtain the weight ratios corresponding to the classification loss parameter, the regression box loss parameter, and the center offset loss parameter; wherein the weight ratios corresponding to the regression box loss parameter and the center offset loss parameter are both less than the weight ratio corresponding to the classification loss parameter; and to calculate the classification loss parameter, the regression box loss parameter, and the center offset loss parameter based on the weight ratios to obtain the heatmap loss parameter.
[0170] In one embodiment, the feature extraction module 10 is further configured to identify vehicle sample images using a preset student detection model and obtain the output features of each output layer in the preset student detection model; when the number of features of each output layer in the preset student detection model is greater than a preset feature number threshold, the output features of each output layer in the preset student detection model are used as the first feature.
[0171] In one embodiment, the feature extraction module 10 is further configured to identify vehicle sample images using a preset teacher detection model and obtain the output features of each output layer in the teacher detection model; when the number of features of each output layer in the teacher detection model is greater than the preset feature number threshold, the output features of each output layer in the teacher detection model are used as second features.
[0172] In one embodiment, the vehicle detection model training device further includes a matching module, which is used to determine whether the number of features of the first feature is equal to the number of features of the second feature; when the number of features of the first feature is not equal to the number of features of the second feature, output features with the same number of features are selected from the output features of each output layer in the teacher detection model according to the number of features of the first feature to obtain the adjusted second feature; correspondingly, the loss calculation module 20 is also used to calculate the loss parameter between the first feature and the adjusted second feature according to a preset distillation loss function.
[0173] In one embodiment, the vehicle detection model training device further includes a pre-training module, which is used to acquire a first target detection network and a second target detection network; wherein the number of parameters of the first target detection network is less than the number of parameters of the second target detection network; acquire a vehicle image training set, and train the first target detection network and the second target detection network respectively using the vehicle image training set to obtain a trained first target detection network and a trained second target detection network; when the trained first target detection network satisfies a first preset convergence condition, the trained first target detection network is used as a preset student detection model; when the trained second target detection network satisfies a second preset convergence condition, the trained second target detection network is used as a preset teacher detection model.
[0174] In one embodiment, the parameter adjustment module 30 is further configured to update the training parameters in the preset student detection model according to the loss parameters to obtain the adjusted student detection model; obtain the loss parameters corresponding to the adjusted student detection model; and when the loss parameters corresponding to the adjusted student detection model are within a preset range, use the adjusted student detection model as a vehicle detection model.
[0175] In one embodiment, the vehicle detection model training device further includes a detection module, which is used to acquire an image of a vehicle to be detected, and to detect the image of the vehicle to be detected using the vehicle detection model to obtain vehicle driving parameters; the vehicle driving parameters are compared with a preset threshold, and an alarm is triggered when the vehicle driving parameters reach the preset threshold.
[0176] Other embodiments or specific implementations of the vehicle detection model training device of the present invention can be referred to the above-described method embodiments, and will not be repeated here.
[0177] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.
[0178] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In the unit claims listing several devices, several of these devices may be embodied by the same hardware item. The use of the terms first, second, and third, etc., does not indicate any order and can be interpreted as names.
[0179] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as a read-only memory image (ROM) / random access memory (RAM), magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.
[0180] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.
Claims
1. A method for training a vehicle detection model, characterized in that, The vehicle detection model training method includes the following steps: The vehicle sample images are identified using a pre-set student detection model to obtain the first feature; The vehicle sample images are identified using a pre-set teacher detection model to obtain a second feature; The loss parameter between the first feature and the second feature is calculated according to a preset distillation loss function, wherein the loss parameter includes a true loss parameter, which is used to determine the distillation temperature coefficient and construct the preset distillation loss function; The training parameters in the preset student detection model are updated according to the loss parameters to train the student detection model and obtain the vehicle detection model. The first feature includes a first output feature map and a first distribution heatmap, and the second feature includes a second output feature map and a second distribution heatmap. The step of calculating the loss parameter between the first output feature and the second output feature according to a preset distillation loss function includes: Calculate the feature map loss parameters between the first output feature map and the second output feature map according to the preset distillation loss function; The heat map loss parameters between the first distribution heat map and the second distribution heat map are calculated based on the preset distillation loss function; Obtain preset feature marker information from the vehicle sample image; The true loss parameter between the second distribution heatmap and the preset feature labeling information is calculated based on the preset true loss function; The loss parameters are determined based on the feature map loss parameters, the heatmap loss parameters, and the actual loss parameters.
2. The vehicle detection model training method as described in claim 1, characterized in that, Before calculating the feature map loss parameters between the first output feature map and the second output feature map according to the preset distillation loss function, the method further includes: Select the corresponding distillation temperature coefficient from the first preset coefficient table based on the actual loss parameters; A preset distillation loss function is constructed based on the preset error function and the distillation temperature coefficient.
3. The vehicle detection model training method as described in claim 1, characterized in that, The step of determining the loss parameters based on the feature map loss parameters, the heatmap loss parameters, and the true loss parameters includes: Select the corresponding distillation correction coefficient from the second preset coefficient table based on the actual loss parameters; The feature map loss parameter and the thermogram loss parameter are corrected according to the distillation correction coefficient to obtain the distillation loss parameter; The loss parameters are determined based on the distillation loss parameters and the actual loss parameters.
4. The vehicle detection model training method as described in claim 1, characterized in that, The distribution heatmap includes a classification heatmap, a regression box width and height heatmap, and a center offset heatmap; the step of calculating the heatmap loss parameter between the first distribution heatmap and the second distribution heatmap according to the preset distillation loss function includes: The classification loss parameter between the first classification heatmap and the second classification heatmap is calculated based on the preset distillation loss function; The regression box loss parameters between the first regression box width and height heatmap and the second regression box width and height heatmap are calculated based on the preset distillation loss function. The center offset loss parameter between the first center offset heatmap and the second center offset heatmap is calculated based on the preset distillation loss function; The heatmap loss parameters are determined based on the classification loss parameters, the regression box loss parameters, and the center offset loss parameters.
5. The vehicle detection model training method as described in claim 4, characterized in that, The step of determining the heatmap loss parameters based on the classification loss parameters, the regression box loss parameters, and the center offset loss parameters includes: Obtain the weight ratios corresponding to the classification loss parameter, the regression box loss parameter, and the center offset loss parameter; wherein the weight ratios corresponding to the regression box loss parameter and the center offset loss parameter are both less than the weight ratios corresponding to the classification loss parameter. The heatmap loss parameters are obtained by calculating the classification loss parameters, the regression box loss parameters, and the center offset loss parameters based on the weight ratio.
6. The vehicle detection model training method as described in claim 1, characterized in that, The step of identifying vehicle sample images using a preset student detection model to obtain a first feature includes: Vehicle sample images are identified using a preset student detection model, and the output features of each output layer in the preset student detection model are obtained. When the number of output features of each output layer in the preset student detection model is greater than the preset feature number threshold, the output features of each output layer in the preset student detection model are taken as the first feature.
7. The vehicle detection model training method as described in claim 6, characterized in that, The step of identifying the vehicle sample image using a preset teacher detection model to obtain the second feature includes: Vehicle sample images are identified using a pre-set teacher detection model, and the output features of each output layer in the teacher detection model are obtained. When the number of output features of each output layer in the teacher detection model is greater than the preset feature number threshold, the output features of each output layer in the teacher detection model are used as the second feature.
8. The vehicle detection model training method as described in claim 7, characterized in that, Before calculating the loss parameter between the first feature and the second feature according to the preset distillation loss function, the method further includes: Determine whether the number of features of the first feature is equal to the number of features of the second feature; When the number of features of the first feature is not equal to the number of features of the second feature, the same number of output features are selected from the output features of each output layer in the teacher detection model according to the number of features of the first feature, and the adjusted second feature is obtained. Accordingly, the step of calculating the loss parameter between the first feature and the second feature according to the preset distillation loss function includes: The loss parameters between the first feature and the adjusted second feature are calculated based on a preset distillation loss function.
9. The vehicle detection model training method according to any one of claims 1 to 3, characterized in that, Before obtaining the first feature by recognizing the vehicle sample image through a preset student detection model, the process also includes: Obtain a first target detection network and a second target detection network; wherein the number of parameters in the first target detection network is less than the number of parameters in the second target detection network; A vehicle image training set is obtained, and the first target detection network and the second target detection network are trained using the vehicle image training set to obtain the trained first target detection network and the trained second target detection network. When the trained first object detection network satisfies the first preset convergence condition, the trained first object detection network is used as the preset student detection model. When the second target detection network after training satisfies the second preset convergence condition, the trained second target detection network is used as the preset teacher detection model.
10. The vehicle detection model training method according to any one of claims 1 to 3, characterized in that, The step of updating the training parameters in the preset student detection model according to the loss parameters to train the student detection model and obtain a vehicle detection model includes: Update the training parameters in the preset student detection model according to the loss parameters to obtain the adjusted student detection model; Obtain the loss parameters corresponding to the adjusted student detection model; When the loss parameters corresponding to the adjusted student detection model are within a preset range, the adjusted student detection model is used as the vehicle detection model.
11. The vehicle detection model training method as described in claim 1, characterized in that, After updating the training parameters in the preset student detection model according to the loss parameters to train the student detection model and obtain the vehicle detection model, the method further includes: The vehicle image to be detected is acquired, and the vehicle detection model is used to detect the vehicle image to obtain vehicle driving parameters. The vehicle driving parameters are compared with preset thresholds, and an alarm is triggered when the vehicle driving parameters reach the preset thresholds.
12. A vehicle detection model training device, characterized in that, The vehicle detection model training device includes: The feature extraction module is used to identify vehicle sample images using a preset student detection model to obtain the first feature; The feature extraction module is also used to identify the vehicle sample image through a preset teacher detection model to obtain a second feature; The loss calculation module is used to calculate the loss parameters between the first feature and the second feature according to a preset distillation loss function, wherein the loss parameters include the actual loss parameters, and the actual loss parameters are used to determine the distillation temperature coefficient and construct the preset distillation loss function; The parameter adjustment module is used to update the training parameters in the preset student detection model according to the loss parameters, so as to train the student detection model and obtain the vehicle detection model. The loss calculation module is further configured to: calculate feature map loss parameters between the first output feature map and the second output feature map according to a preset distillation loss function; calculate heatmap loss parameters between the first distribution heatmap and the second distribution heatmap according to the preset distillation loss function; obtain preset feature label information of the vehicle sample image; calculate the true loss parameters between the second distribution heatmap and the preset feature label information according to a preset true loss function; and determine the loss parameters based on the feature map loss parameters, the heatmap loss parameters, and the true loss parameters.
13. The vehicle detection model training device as described in claim 12, characterized in that, The feature extraction module is also used to identify vehicle sample images through a preset teacher detection model and obtain the output features of each output layer in the teacher detection model. The feature extraction module is further configured to use the output feature as a second feature when the number of features in the output feature is greater than a preset feature number threshold.
14. The vehicle detection model training device as described in claim 12, characterized in that, The parameter adjustment module is also used to update the training parameters in the preset student detection model according to the loss parameters, so as to obtain the adjusted student detection model. The parameter adjustment module is also used to obtain the loss parameters corresponding to the adjusted student detection model; The parameter adjustment module is also used to use the adjusted student detection model as a vehicle detection model when the loss parameter corresponding to the adjusted student detection model is within a preset range.
15. A vehicle detection model training device, characterized in that, The vehicle detection model training device includes: a memory, a processor, and a vehicle detection model training program stored in the memory and executable on the processor. When the vehicle detection model training program is executed by the processor, it implements the steps of the vehicle detection model training method as described in any one of claims 1 to 11.
16. A storage medium, characterized in that, The storage medium stores a vehicle detection model training program, which, when executed by a processor, implements the steps of the vehicle detection model training method as described in any one of claims 1 to 11.