Vehicle course angle detection method and device
By constructing a deep learning model with multiple detection heads, the flexibility and accuracy of the existing vehicle heading angle detection algorithm in diverse vehicles and complex environments are solved, and high-precision and real-time vehicle heading angle detection are achieved.
Patent Information
- Application Number
- CN202411512823.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-28
- Publication Date
- 2025-02-11
AI Technical Summary
The existing vehicle heading angle detection algorithms have problems such as insufficient flexibility, complex rules and improved accuracy when dealing with diverse vehicles and complex road environments, and it is difficult to meet the dual requirements of autonomous driving technology for high accuracy and real-time.
Using the vehicle heading angle detection method based on deep learning, the heading angle detection model including ResNet backbone network, FPN feature fusion network, and multiple detection heads is constructed, and the heading angle detection model is constructed through pre-configured 损失函数进行训练,得到训练好的车辆航向角检测模型。 该模型能够对实时车辆图片进行航向角检测,实现高精度和实时性。
It realizes flexible and accurate identification of vehicle heading angles, high detection efficiency, easy deployment, and can meet the dual requirements of autonomous driving technology for high accuracy and real-time.
Smart Images
Figure CN119417797A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of vehicle identification technology, and in particular to a vehicle heading angle detection method and device. Background Art
[0002] At present, vehicle heading angle detection is a core issue in autonomous driving technology. Its purpose is to accurately obtain the vehicle's position information, so as to effectively calculate and predict the vehicle's motion trajectory, ensure driving safety, and avoid collision accidents. However, the diversity of vehicle types, the differences in vehicle shapes, and the complexity of road environments, such as backlight, darkness, occlusion and other special situations, all pose great challenges to the accurate detection of vehicle heading angles. In addition, the vehicle heading angle detection algorithm needs to run in real time in the vehicle system to process continuous real-time data, so it places extremely high demands on the real-time performance of the algorithm. Most of the existing vehicle heading angle detection algorithms are based on image processing technology, which estimates the vehicle's heading angle by extracting features such as the vehicle's color, edge, direction and shape. However, these traditional methods have the problems of insufficient flexibility, complex rules, and accuracy that needs to be improved, making it difficult to meet the dual requirements of autonomous driving technology for high precision and real-time performance. Summary of the invention
[0003] The purpose of the embodiments of the present application is to provide a vehicle heading angle detection method and device, which can flexibly and accurately identify the vehicle heading angle, have high detection efficiency, and are easy to deploy, and can meet the dual requirements of autonomous driving technology for high precision and real-time performance.
[0004] The first aspect of the present application provides a vehicle heading angle detection method, comprising:
[0005] Get labeled training and test data sets;
[0006] Constructing an original heading angle detection model; wherein the original heading angle detection model includes a ResNet backbone network, an FPN feature fusion network, a first detection head, a second detection head, and a third detection head;
[0007] The original heading angle detection model is trained by using the training data set and a pre-configured loss function to obtain a trained vehicle heading angle detection model;
[0008] Performing heading angle prediction on the test data set by using the vehicle heading angle detection model to obtain a test set prediction result;
[0009] Calculate the mean absolute error of the heading angle according to the test set prediction result and the test data set;
[0010] When the heading angle mean absolute error is less than a preset error threshold, the vehicle heading angle detection model is configured on the target vehicle;
[0011] Get real-time vehicle images;
[0012] The vehicle heading angle detection is performed using the vehicle heading angle detection model and the real-time vehicle image to obtain a real-time heading angle detection result.
[0013] In the above implementation process, the method can flexibly and accurately identify the vehicle heading angle, has high detection efficiency, and is easy to deploy, which can meet the dual requirements of high precision and real-time performance of autonomous driving technology.
[0014] Furthermore, the step of obtaining the labeled training data set and test data set includes:
[0015] Collect original sample images containing various vehicle scenes;
[0016] The original sample image is annotated with a vehicle 2D frame, a vehicle, and a heading angle to obtain an annotated sample image;
[0017] A training data set and a test data set are constructed based on the labeled sample images.
[0018] Furthermore, the performing heading angle prediction on the test data set by the vehicle heading angle detection model to obtain a test set prediction result includes:
[0019] Perform feature extraction on the test data set through the ResNet backbone network to obtain image features corresponding to each picture in the test data set;
[0020] Performing feature fusion on the image features through the FPN feature fusion network to obtain image fusion features corresponding to each picture in the test data set;
[0021] Performing vehicle 2D frame detection according to the first detection head and the image fusion feature to obtain a vehicle 2D frame feature corresponding to each image in the test data set;
[0022] Performing heading angle interval detection according to the second detection head and the image fusion feature to obtain a vehicle heading angle interval corresponding to each image in the test data set;
[0023] Performing heading angle detection according to the third detection head and the image fusion feature to obtain a vehicle heading angle feature corresponding to each image in the test data set;
[0024] The vehicle 2D frame features, the vehicle heading angle interval and the vehicle heading angle features are summarized to obtain a test set prediction result.
[0025] Furthermore, the calculating the mean absolute error of the heading angle according to the test set prediction result and the test data set includes:
[0026] Post-processing the test set prediction results to obtain a final prediction result of the heading angle corresponding to each vehicle in the test data set;
[0027] Performing statistics on the labeling information of the training data set to obtain labeling statistical information; wherein the labeling statistical information includes the number of vehicle labels labeled as vehicles, the labeling heading angle corresponding to each of the vehicles, and the labeling vehicle 2D box corresponding to each of the vehicles;
[0028] The prediction results of the test data set are counted according to the final prediction results of the heading angle to obtain prediction statistical information; wherein the prediction statistical information includes the predicted number of vehicles predicted to be vehicles, the predicted heading angle corresponding to each of the vehicles, and the vehicle prediction 2D box corresponding to each of the vehicles;
[0029] Performing intersection-and-union calculation on the annotated vehicle 2D frame and the predicted vehicle 2D frame to obtain an intersection-and-union calculation result;
[0030] Calculate the heading angle error corresponding to each vehicle according to the intersection-to-combination ratio calculation result;
[0031] The mean absolute error of the heading angles of all vehicles is calculated according to the heading angle error and the test data set.
[0032] Furthermore, the post-processing of the test set prediction result to obtain the final prediction result of the heading angle corresponding to each vehicle in the test data set includes:
[0033] Extracting 2D frame features of the vehicle detected by the first detection head;
[0034] Determine the 2D frame coordinate position corresponding to each vehicle in the test data set according to a preset recognition frame threshold and the vehicle 2D frame feature;
[0035] Extracting the vehicle heading angle interval detected by the second detection head;
[0036] Determine the heading angle interval corresponding to each vehicle in the test data set according to the vehicle heading angle interval and the 2D frame coordinate position;
[0037] Extracting the vehicle heading angle feature detected by the third detection head;
[0038] Determine the heading angle value corresponding to each vehicle in the test data set according to the vehicle heading angle feature and the 2D frame coordinate position;
[0039] Calculating a final heading angle corresponding to each vehicle in the test data set according to the heading angle interval and the heading angle value;
[0040] The heading angle interval, the 2D frame coordinate position and the final heading angle are summarized to obtain a final prediction result of the heading angle corresponding to each vehicle in the test data set.
[0041] A second aspect of the present application provides a vehicle heading angle detection device, the vehicle heading angle detection device comprising:
[0042] A first acquisition unit is used to acquire a labeled training data set and a test data set;
[0043] A construction unit, used to construct an original heading angle detection model; wherein the original heading angle detection model includes a ResNet backbone network, an FPN feature fusion network, a first detection head, a second detection head and a third detection head;
[0044] A training unit, used to train the original heading angle detection model using the training data set and a pre-configured loss function to obtain a trained vehicle heading angle detection model;
[0045] A testing unit, used to predict the heading angle of the test data set by using the vehicle heading angle detection model to obtain a test set prediction result;
[0046] A calculation unit, used for calculating the mean absolute error of the heading angle according to the test set prediction result and the test data set;
[0047] A configuration unit, configured to configure the vehicle heading angle detection model to a target vehicle when the heading angle mean absolute error is less than a preset error threshold;
[0048] A second acquisition unit is used to acquire a real-time vehicle image;
[0049] The detection unit is used to perform vehicle heading angle detection through the vehicle heading angle detection model and the real-time vehicle image to obtain a real-time heading angle detection result.
[0050] Furthermore, the first acquiring unit includes:
[0051] The acquisition subunit is used to collect original sample images containing various vehicle scenes;
[0052] The labeling subunit is used to label the original sample image with a vehicle 2D frame, a vehicle, and a heading angle to obtain a labeled sample image;
[0053] The construction subunit is used to construct a training data set and a test data set based on the labeled sample images.
[0054] Furthermore, the testing unit comprises:
[0055] An extraction subunit, configured to perform feature extraction on the test data set through the ResNet backbone network to obtain image features corresponding to each picture in the test data set;
[0056] A fusion subunit, used to perform feature fusion on the image features through the FPN feature fusion network to obtain image fusion features corresponding to each picture in the test data set;
[0057] A first detection subunit is used to perform vehicle 2D frame detection according to the first detection head and the image fusion feature, and obtain a vehicle 2D frame feature corresponding to each image in the test data set;
[0058] A second detection subunit is used to perform heading angle interval detection according to the second detection head and the image fusion feature to obtain a vehicle heading angle interval corresponding to each image in the test data set;
[0059] A third detection subunit is used to perform heading angle detection according to the third detection head and the image fusion feature to obtain a vehicle heading angle feature corresponding to each image in the test data set;
[0060] The summarizing subunit is used to summarize the vehicle 2D frame features, the vehicle heading angle interval and the vehicle heading angle features to obtain a test set prediction result.
[0061] Furthermore, the calculation unit comprises:
[0062] A post-processing subunit, used to post-process the test set prediction result to obtain a final prediction result of the heading angle corresponding to each vehicle in the test data set;
[0063] A statistical subunit, configured to perform statistics on the labeling information of the training data set to obtain labeling statistical information; wherein the labeling statistical information includes the number of vehicle labels labeled as vehicles, the labeling heading angle corresponding to each of the vehicles, and the labeling vehicle 2D box corresponding to each of the vehicles;
[0064] The statistical subunit is further used to perform statistics on the prediction results of the test data set according to the final prediction results of the heading angle to obtain prediction statistical information; wherein the prediction statistical information includes the predicted number of vehicles predicted to be vehicles, the predicted heading angle corresponding to each of the vehicles, and the vehicle prediction 2D box corresponding to each of the vehicles;
[0065] A calculation subunit, used for performing intersection-and-union calculation on the annotated vehicle 2D frame and the predicted vehicle 2D frame to obtain an intersection-and-union calculation result;
[0066] The calculation subunit is further used to calculate the heading angle error corresponding to each vehicle according to the intersection-to-combination ratio calculation result;
[0067] The calculation subunit is further used to calculate the average absolute error of the heading angles of all vehicles according to the heading angle error and the test data set.
[0068] Furthermore, the post-processing subunit includes:
[0069] An extraction module, used to extract the 2D frame features of the vehicle detected by the first detection head;
[0070] A determination module, used to determine the 2D frame coordinate position corresponding to each vehicle in the test data set according to a preset recognition frame threshold and the vehicle 2D frame feature;
[0071] An extraction module, used to extract the vehicle heading angle interval detected by the second detection head;
[0072] The determination module is further used to determine the heading angle interval corresponding to each vehicle in the test data set according to the vehicle heading angle interval and the 2D frame coordinate position;
[0073] The extraction module is further used to extract the vehicle heading angle feature detected by the third detection head;
[0074] The determination module is further used to determine the heading angle value corresponding to each vehicle in the test data set according to the vehicle heading angle feature and the 2D frame coordinate position;
[0075] A calculation module, used for calculating a final heading angle corresponding to each vehicle in the test data set according to the heading angle interval and the heading angle value;
[0076] A summarizing module is used to summarize the heading angle interval, the 2D frame coordinate position and the final heading angle to obtain a final prediction result of the heading angle corresponding to each vehicle in the test data set.
[0077] A third aspect of the present application provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the vehicle heading angle detection method described in any one of the first aspects of the present application.
[0078] A fourth aspect of the present application provides a computer-readable storage medium storing computer program instructions. When the computer program instructions are read and executed by a processor, the vehicle heading angle detection method described in any one of the first aspect of the present application is executed.
[0079] The beneficial effects of the present application are: being able to implement the entire process of vehicle heading angle feature detection, post-processing, and detection result evaluation based on deep learning, thereby achieving effective recognition of the vehicle heading angle and effective evaluation of the recognition results;
[0080] It is also possible to achieve more convenient expansion effects by specifying the detection model, so that it can be jointly trained with other tasks (such as drivable space detection, parking space detection, etc.), thus facilitating the lightweight deployment of multi-task models;
[0081] It can also reduce the difficulty of deployment based on simpler and more effective post-processing methods;
[0082] Focal-Loss can also be used as the loss, which is more suitable for solving the problem of imbalance in the number of difficult and easy samples. BRIEF DESCRIPTION OF THE DRAWINGS
[0083] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0084] Figure 1 A schematic diagram of a flow chart of a vehicle heading angle detection method provided in an embodiment of the present application;
[0085] Figure 2 A schematic diagram of a flow chart of another vehicle heading angle detection method provided in an embodiment of the present application;
[0086] Figure 3 A schematic diagram of an exemplary process of a vehicle heading angle detection method provided in an embodiment of the present application;
[0087] Figure 4 A schematic diagram of the structure of a vehicle heading angle detection device provided in an embodiment of the present application;
[0088] Figure 5 A schematic diagram of the structure of another vehicle heading angle detection device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0089] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.
[0090] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.
[0091] Example 1
[0092] Please see Figure 1 , Figure 1 A schematic flow chart of a vehicle heading angle detection method provided in this embodiment. The vehicle heading angle detection method includes:
[0093] S101, obtain labeled training data set and test data set.
[0094] S102, constructing an original heading angle detection model; wherein the original heading angle detection model includes a ResNet backbone network, an FPN feature fusion network, a first detection head, a second detection head and a third detection head.
[0095] S103, training the original heading angle detection model using a training data set and a pre-configured loss function to obtain a trained vehicle heading angle detection model.
[0096] S104. Predicting the heading angle of the test data set by using the vehicle heading angle detection model to obtain a prediction result of the test set.
[0097] S105. Calculate the mean absolute error of the heading angle according to the test set prediction result and the test data set.
[0098] S106: When the heading angle mean absolute error is less than a preset error threshold, the vehicle heading angle detection model is configured on the target vehicle.
[0099] S107: Acquire real-time vehicle images.
[0100] S108, performing vehicle heading angle detection using a vehicle heading angle detection model and a real-time vehicle image to obtain a real-time heading angle detection result.
[0101] In this embodiment, the vehicle heading angle detection method is specifically a full-process method of vehicle heading angle feature detection, post-processing, and detection result evaluation based on deep learning.
[0102] In this embodiment, the execution subject of the method may be a computing device such as a computer or a server, and no limitation is made in this embodiment.
[0103] In this embodiment, the execution subject of the method may also be a smart device such as a smart phone, a tablet computer, etc., which is not limited in this embodiment.
[0104] It can be seen that the vehicle heading angle detection method described in this embodiment can flexibly and accurately identify the vehicle heading angle, has high detection efficiency, and is easy to deploy, and can meet the dual requirements of autonomous driving technology for high precision and real-time performance.
[0105] Example 2
[0106] Please see Figure 2 , Figure 2 A schematic flow chart of a vehicle heading angle detection method provided in this embodiment. The vehicle heading angle detection method includes:
[0107] S201, collecting original sample images containing various vehicle scenes.
[0108] S202: annotate the original sample image with a vehicle 2D frame, a vehicle, and a heading angle to obtain an annotated sample image.
[0109] S203: Construct a training data set and a test data set based on the labeled sample images.
[0110] In this embodiment, the method can collect pictures containing various types of vehicle scenes and annotate the vehicles (label the vehicle category and 2D frame) and their heading angles (with the camera optical center as the far point, the front is the positive direction of the z-axis, the horizontal left is the positive direction of the x-axis, and the vertical downward is the positive direction of the y-axis), and stipulate that the counterclockwise direction from the positive semi-axis to the negative semi-axis of the z-axis corresponds to 0 to 180°, and the clockwise direction from the positive semi-axis to the negative semi-axis of the z-axis corresponds to 0 to -180°, so as to obtain training and test images.
[0111] S204, constructing an original heading angle detection model; wherein the original heading angle detection model includes a ResNet backbone network, an FPN feature fusion network, a first detection head, a second detection head and a third detection head.
[0112] In this embodiment, the process of constructing the original heading angle detection model is the process of building a lane feature extraction network model. In this process, ResNet is first used as the backbone network of the network structure for feature extraction, and then the FPN network is used for feature fusion.
[0113] Specifically, the method first uses a detection head to learn whether there is a vehicle and the width and height of the 2D box at the center of the annotated position in the image, marked as detection head 1 (3 channels). Secondly, the vehicle heading angle is divided into 4 intervals: 0 to 90° is marked as interval 1; 90° to 180° is marked as interval 2; 0 to -90° is marked as interval 3; -90° to -180° is marked as interval 4. Then, a detection head is added at the same time to determine which of the 4 intervals the vehicle heading angle is in (marked as detection head 2 (4 channels)), and another detection head is added for the sin and cos values of the heading angle (marked as detection 3 (2 channels)).
[0114] It can be seen that this network structure is convenient for joint training with other drivable space detection tasks, parking space detection tasks, and other tasks, thereby making the deployment of multi-task models more lightweight based on this easy scalability.
[0115] S205. Train the original heading angle detection model using a training data set and a pre-configured loss function to obtain a trained vehicle heading angle detection model.
[0116] In this embodiment, the method can input the image data obtained in S203 into the network model constructed in S204 to extract image feature information. Then, the Focal-Loss loss function is used to train and learn whether there is a vehicle in the image and which interval the vehicle heading angle belongs to (this loss function is more suitable for solving the problem of unbalanced number of difficult and easy samples). Finally, L1Loss is used to make the absolute value error of the vehicle's 2D box size, heading angle sin value and cos value close to 0, so as to train and learn to obtain the feature vector of the vehicle heading angle.
[0117] S206. Predicting the heading angle of the test data set by using the vehicle heading angle detection model to obtain a prediction result of the test set.
[0118] As an optional implementation, the heading angle of the test data set is predicted by the vehicle heading angle detection model to obtain the test set prediction result, including:
[0119] The ResNet backbone network is used to extract features from the test data set to obtain the image features corresponding to each picture in the test data set.
[0120] The image features are fused through the FPN feature fusion network to obtain the image fusion features corresponding to each picture in the test data set;
[0121] Perform vehicle 2D frame detection based on the first detection head and the image fusion features to obtain the vehicle 2D frame features corresponding to each image in the test data set;
[0122] Perform heading angle interval detection based on the second detection head and image fusion features to obtain the vehicle heading angle interval corresponding to each image in the test data set;
[0123] Perform heading angle detection based on the third detection head and image fusion features to obtain vehicle heading angle features corresponding to each image in the test data set;
[0124] The vehicle 2D frame features, vehicle heading angle interval and vehicle heading angle features are summarized to obtain the test set prediction results.
[0125] S207: Extract the 2D frame features of the vehicle detected by the first detection head.
[0126] S208 . Determine the 2D frame coordinate position corresponding to each vehicle in the test data set according to a preset recognition frame threshold and the vehicle 2D frame feature.
[0127] In this embodiment, the method determines whether the vehicle detection value in the vehicle 2D frame feature is greater than a preset threshold. If it is greater (that is, there is a vehicle), its coordinate position INDEX is recorded, and the value of the corresponding coordinate position in the vehicle 2D frame feature is extracted, which are recorded as W and H of the 2D frame respectively.
[0128] S209: extracting the vehicle heading angle interval detected by the second detection head.
[0129] S210 , determining a heading angle interval corresponding to each vehicle in the test data set according to the vehicle heading angle interval and the 2D frame coordinate position.
[0130] In this embodiment, the method extracts the values of the four channels corresponding to the coordinate position of the second detection head according to the coordinate position INDEX, and selects the maximum value. The channel where the maximum value is located represents the interval C where the heading angle is located.
[0131] S211. Extracting vehicle heading angle features detected by the third detection head.
[0132] S212. Determine the heading angle value corresponding to each vehicle in the test data set according to the vehicle heading angle feature and the 2D frame coordinate position.
[0133] In this embodiment, the method extracts the values a and b of the two channels of the coordinate position corresponding to the third detection head according to the coordinate position INDEX, and determines the angle value M=atan2(a / b)*180 / PI.
[0134] S213. Calculate the final heading angle corresponding to each vehicle in the test data set according to the heading angle interval and the heading angle value.
[0135] In this embodiment, the method can be combined with the above steps to calculate the final heading angle Yaw, as follows:
[0136] If interval C is 1, then Yaw = M;
[0137] If interval C is 2, then Yaw=M+90°;
[0138] If interval C is 3, then Yaw = -M;
[0139] If interval C is 4, then Yaw = -M-90°.
[0140] S214, summarizing the heading angle interval, the 2D frame coordinate position, and the final heading angle to obtain a final prediction result of the heading angle corresponding to each vehicle in the test data set.
[0141] S215. Count the annotation information of the training data set to obtain annotation statistical information; wherein the annotation statistical information includes the number of vehicle annotations marked as vehicles, the labeled heading angle corresponding to each vehicle, and the labeled vehicle 2D box corresponding to each vehicle.
[0142] In this embodiment, the method can count the number of objects p_gt marked as vehicles in the test set and their corresponding heading angles y_gt and 2D boxes bbox_gt.
[0143] S216. Count the prediction results of the test data set according to the final prediction results of the heading angle to obtain prediction statistical information; wherein the prediction statistical information includes the predicted number of vehicles predicted to be vehicles, the predicted heading angle corresponding to each vehicle, and the vehicle prediction 2D box corresponding to each vehicle.
[0144] In this embodiment, the method can also count the number of objects p_pred predicted to be vehicles in the test set and their corresponding heading angles y_pred and 2D boxes bbox_pred.
[0145] S217, performing intersection-and-union calculation on the annotated vehicle 2D frame and the predicted vehicle 2D frame to obtain an intersection-and-union calculation result.
[0146] S218. Calculate the heading angle error corresponding to each vehicle according to the intersection-to-combination ratio calculation result.
[0147] In this embodiment, the method can determine that the predicted vehicle and the labeled vehicle are in a corresponding relationship based on the intersection and union ratio of bbox_gt and bbox_pred being greater than a threshold d_thresh (preset value), then the first vehicle heading angle error E1 = y_pred-y_gt, and E2, E3, etc. can be obtained similarly.
[0148] S219. Calculate the mean absolute error of the heading angles of all vehicles based on the heading angle error and the test data set.
[0149] In this embodiment, the number of predicted vehicles and labeled vehicles that have corresponding relationships is n.
[0150] In this embodiment, the method can calculate the average absolute error E=(E1+E2+E3+…+En) / n of the heading angles of all vehicles in the test set.
[0151] S220: When the mean absolute error of the heading angle is less than a preset error threshold, the vehicle heading angle detection model is configured on the target vehicle.
[0152] S221. Obtain real-time vehicle images.
[0153] S222. Perform vehicle heading angle detection using a vehicle heading angle detection model and a real-time vehicle image to obtain a real-time heading angle detection result.
[0154] Please see Figure 3 , Figure 3 A schematic diagram of an example flow chart of a vehicle heading angle detection method is shown.
[0155] In this embodiment, the execution subject of the method may be a computing device such as a computer or a server, and no limitation is made in this embodiment.
[0156] In this embodiment, the execution subject of the method may also be a smart device such as a smart phone, a tablet computer, etc., which is not limited in this embodiment.
[0157] It can be seen that the vehicle heading angle detection method described in this embodiment can flexibly and accurately identify the vehicle heading angle, has high detection efficiency, and is easy to deploy, and can meet the dual requirements of autonomous driving technology for high precision and real-time performance.
[0158] Example 3
[0159] Please see Figure 4 , Figure 4 FIG. 1 is a schematic diagram of the structure of a vehicle heading angle detection device provided in this embodiment. Figure 4 As shown, the vehicle heading angle detection device comprises:
[0160] A first acquisition unit 310 is used to acquire a labeled training data set and a test data set;
[0161] A construction unit 320 is used to construct an original heading angle detection model; wherein the original heading angle detection model includes a ResNet backbone network, an FPN feature fusion network, a first detection head, a second detection head, and a third detection head;
[0162] The training unit 330 is used to train the original heading angle detection model through a training data set and a pre-configured loss function to obtain a trained vehicle heading angle detection model;
[0163] A testing unit 340 is used to predict the heading angle of a test data set by using a vehicle heading angle detection model to obtain a prediction result of the test set;
[0164] A calculation unit 350 is used to calculate the mean absolute error of the heading angle according to the test set prediction result and the test data set;
[0165] A configuration unit 360, configured to configure a vehicle heading angle detection model to a target vehicle when the heading angle mean absolute error is less than a preset error threshold;
[0166] A second acquisition unit 370 is used to acquire a real-time vehicle image;
[0167] The detection unit 380 is used to detect the vehicle heading angle through a vehicle heading angle detection model and a real-time vehicle image to obtain a real-time heading angle detection result.
[0168] In this embodiment, the explanation of the vehicle heading angle detection device can refer to the description in Embodiment 1 or Embodiment 2, and will not be further elaborated in this embodiment.
[0169] It can be seen that the vehicle heading angle detection device described in this embodiment can flexibly and accurately identify the vehicle heading angle, has high detection efficiency, and is easy to deploy, and can meet the dual requirements of autonomous driving technology for high precision and real-time performance.
[0170] Example 4
[0171] Please see Figure 5 , Figure 5 FIG. 1 is a schematic diagram of the structure of a vehicle heading angle detection device provided in this embodiment. Figure 5 As shown, the vehicle heading angle detection device comprises:
[0172] A first acquisition unit 310 is used to acquire a labeled training data set and a test data set;
[0173] A construction unit 320 is used to construct an original heading angle detection model; wherein the original heading angle detection model includes a ResNet backbone network, an FPN feature fusion network, a first detection head, a second detection head, and a third detection head;
[0174] The training unit 330 is used to train the original heading angle detection model through a training data set and a pre-configured loss function to obtain a trained vehicle heading angle detection model;
[0175] A testing unit 340 is used to predict the heading angle of a test data set by using a vehicle heading angle detection model to obtain a prediction result of the test set;
[0176] A calculation unit 350 is used to calculate the mean absolute error of the heading angle according to the test set prediction result and the test data set;
[0177] A configuration unit 360, configured to configure a vehicle heading angle detection model to a target vehicle when the heading angle mean absolute error is less than a preset error threshold;
[0178] A second acquisition unit 370 is used to acquire a real-time vehicle image;
[0179] The detection unit 380 is used to detect the vehicle heading angle through a vehicle heading angle detection model and a real-time vehicle image to obtain a real-time heading angle detection result.
[0180] As an optional implementation manner, the first acquiring unit 310 includes:
[0181] The acquisition subunit 311 is used to acquire original sample images containing various vehicle scenes;
[0182] The labeling subunit 312 is used to label the original sample image with a vehicle 2D frame, a vehicle, and a heading angle to obtain a labeled sample image;
[0183] The construction subunit 313 is used to construct a training data set and a test data set according to the labeled sample images.
[0184] As an optional implementation, the testing unit 340 includes:
[0185] An extraction subunit 341 is used to extract features from the test data set through a ResNet backbone network to obtain image features corresponding to each image in the test data set;
[0186] A fusion subunit 342 is used to perform feature fusion on the image features through an FPN feature fusion network to obtain an image fusion feature corresponding to each image in the test data set;
[0187] The first detection subunit 343 is used to perform vehicle 2D frame detection according to the first detection head and the image fusion feature, and obtain the vehicle 2D frame feature corresponding to each image in the test data set;
[0188] The second detection subunit 344 is used to perform heading angle interval detection according to the second detection head and the image fusion feature to obtain the vehicle heading angle interval corresponding to each image in the test data set;
[0189] The third detection subunit 345 is used to perform heading angle detection according to the third detection head and the image fusion feature to obtain the vehicle heading angle feature corresponding to each image in the test data set;
[0190] The summarizing subunit 346 is used to summarize the vehicle 2D frame features, the vehicle heading angle interval and the vehicle heading angle features to obtain the test set prediction results.
[0191] As an optional implementation, the calculation unit 350 includes:
[0192] A post-processing subunit 351 is used to post-process the prediction result of the test set to obtain a final prediction result of the heading angle corresponding to each vehicle in the test data set;
[0193] A statistical subunit 352 is used to count the annotation information of the training data set to obtain annotation statistical information; wherein the annotation statistical information includes the number of vehicle annotations marked as vehicles, the labeled heading angle corresponding to each vehicle, and the labeled vehicle 2D box corresponding to each vehicle;
[0194] The statistical subunit 352 is further used to perform statistics on the prediction results of the test data set according to the final prediction results of the heading angle to obtain prediction statistical information; wherein the prediction statistical information includes the predicted number of vehicles predicted to be vehicles, the predicted heading angle corresponding to each vehicle, and the vehicle prediction 2D box corresponding to each vehicle;
[0195] A calculation subunit 353 is used to perform an intersection-and-union ratio calculation on the annotated vehicle 2D frame and the predicted vehicle 2D frame to obtain an intersection-and-union ratio calculation result;
[0196] The calculation subunit 353 is further used to calculate the heading angle error corresponding to each vehicle according to the intersection-and-combination ratio calculation result;
[0197] The calculation subunit 353 is further used to calculate the average absolute error of the heading angles of all vehicles according to the heading angle error and the test data set.
[0198] As an optional implementation, the post-processing subunit 351 includes:
[0199] An extraction module, used to extract the 2D frame features of the vehicle detected by the first detection head;
[0200] A determination module is used to determine the 2D frame coordinate position corresponding to each vehicle in the test data set according to a preset recognition frame threshold and the vehicle 2D frame feature;
[0201] An extraction module, used to extract the vehicle heading angle interval detected by the second detection head;
[0202] The determination module is further used to determine the heading angle interval corresponding to each vehicle in the test data set according to the vehicle heading angle interval and the 2D frame coordinate position;
[0203] The extraction module is also used to extract the vehicle heading angle feature detected by the third detection head;
[0204] The determination module is also used to determine the heading angle value corresponding to each vehicle in the test data set according to the vehicle heading angle feature and the 2D frame coordinate position;
[0205] A calculation module, used to calculate the final heading angle corresponding to each vehicle in the test data set according to the heading angle interval and the heading angle value;
[0206] The summary module is used to summarize the heading angle interval, 2D box coordinate position and final heading angle to obtain the final prediction result of the heading angle corresponding to each vehicle in the test data set.
[0207] In this embodiment, the explanation of the vehicle heading angle detection device can refer to the description in Embodiment 1 or Embodiment 2, and will not be further elaborated in this embodiment.
[0208] It can be seen that the vehicle heading angle detection device described in this embodiment can flexibly and accurately identify the vehicle heading angle, has high detection efficiency, and is easy to deploy, and can meet the dual requirements of autonomous driving technology for high precision and real-time performance.
[0209] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the vehicle heading angle detection method in Embodiment 1 or Embodiment 2 of the present application.
[0210] An embodiment of the present application provides a computer-readable storage medium storing computer program instructions. When the computer program instructions are read and executed by a processor, the vehicle heading angle detection method in Embodiment 1 or Embodiment 2 of the present application is executed.
[0211] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.
[0212] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
[0213] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0214] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.
[0215] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
[0216] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
Claims
1. A vehicle heading angle detection method, characterized in that: include: Get labeled training and test data sets; Constructing an original heading angle detection model; wherein the original heading angle detection model includes a ResNet backbone network, an FPN feature fusion network, a first detection head, a second detection head, and a third detection head; The original heading angle detection model is trained by using the training data set and a pre-configured loss function to obtain a trained vehicle heading angle detection model; Performing heading angle prediction on the test data set by using the vehicle heading angle detection model to obtain a test set prediction result; Calculate the mean absolute error of the heading angle according to the test set prediction result and the test data set; When the heading angle mean absolute error is less than a preset error threshold, the vehicle heading angle detection model is configured on the target vehicle; Get real-time vehicle images; The vehicle heading angle detection is performed using the vehicle heading angle detection model and the real-time vehicle image to obtain a real-time heading angle detection result.
2. The vehicle heading angle detection method according to claim 1, characterized in that: The step of obtaining the labeled training data set and test data set includes: Collect original sample images containing various vehicle scenes; The original sample image is annotated with a vehicle 2D frame, a vehicle, and a heading angle to obtain an annotated sample image; A training data set and a test data set are constructed based on the labeled sample images.
3. The vehicle heading angle detection method according to claim 1, characterized in that: The method of performing heading angle prediction on the test data set by using the vehicle heading angle detection model to obtain a test set prediction result includes: Perform feature extraction on the test data set through the ResNet backbone network to obtain image features corresponding to each picture in the test data set; Performing feature fusion on the image features through the FPN feature fusion network to obtain image fusion features corresponding to each picture in the test data set; Performing vehicle 2D frame detection according to the first detection head and the image fusion feature to obtain a vehicle 2D frame feature corresponding to each image in the test data set; Performing heading angle interval detection according to the second detection head and the image fusion feature to obtain a vehicle heading angle interval corresponding to each image in the test data set; Performing heading angle detection according to the third detection head and the image fusion feature to obtain a vehicle heading angle feature corresponding to each image in the test data set; The vehicle 2D frame features, the vehicle heading angle interval and the vehicle heading angle features are summarized to obtain a test set prediction result.
4. The vehicle heading angle detection method according to claim 1, characterized in that: The calculating the mean absolute error of the heading angle according to the test set prediction result and the test data set includes: Post-processing the test set prediction results to obtain a final prediction result of the heading angle corresponding to each vehicle in the test data set; Performing statistics on the labeling information of the training data set to obtain labeling statistical information; wherein the labeling statistical information includes the number of vehicle labels labeled as vehicles, the labeling heading angle corresponding to each of the vehicles, and the labeling vehicle 2D box corresponding to each of the vehicles; The prediction results of the test data set are counted according to the final prediction results of the heading angle to obtain prediction statistical information; wherein the prediction statistical information includes the predicted number of vehicles predicted to be vehicles, the predicted heading angle corresponding to each of the vehicles, and the vehicle prediction 2D box corresponding to each of the vehicles; Performing intersection-and-union calculation on the annotated vehicle 2D frame and the predicted vehicle 2D frame to obtain an intersection-and-union calculation result; Calculate the heading angle error corresponding to each vehicle according to the intersection-to-combination ratio calculation result; The mean absolute error of the heading angles of all vehicles is calculated according to the heading angle error and the test data set.
5. The vehicle heading angle detection method according to claim 4, characterized in that: The post-processing of the test set prediction results to obtain a final prediction result of the heading angle corresponding to each vehicle in the test data set includes: Extracting 2D frame features of the vehicle detected by the first detection head; Determine the 2D frame coordinate position corresponding to each vehicle in the test data set according to a preset recognition frame threshold and the vehicle 2D frame feature; Extracting the vehicle heading angle interval detected by the second detection head; Determine the heading angle interval corresponding to each vehicle in the test data set according to the vehicle heading angle interval and the 2D frame coordinate position; Extracting the vehicle heading angle feature detected by the third detection head; Determine the heading angle value corresponding to each vehicle in the test data set according to the vehicle heading angle feature and the 2D frame coordinate position; Calculating a final heading angle corresponding to each vehicle in the test data set according to the heading angle interval and the heading angle value; The heading angle interval, the 2D frame coordinate position and the final heading angle are summarized to obtain a final prediction result of the heading angle corresponding to each vehicle in the test data set.
6. A vehicle heading angle detection device, characterized in that: The vehicle heading angle detection device comprises: A first acquisition unit is used to acquire a labeled training data set and a test data set; A construction unit, used to construct an original heading angle detection model; wherein the original heading angle detection model includes a ResNet backbone network, an FPN feature fusion network, a first detection head, a second detection head and a third detection head; A training unit, used to train the original heading angle detection model using the training data set and a pre-configured loss function to obtain a trained vehicle heading angle detection model; A testing unit, used to predict the heading angle of the test data set by using the vehicle heading angle detection model to obtain a test set prediction result; A calculation unit, used for calculating the mean absolute error of the heading angle according to the test set prediction result and the test data set; A configuration unit, configured to configure the vehicle heading angle detection model to a target vehicle when the heading angle mean absolute error is less than a preset error threshold; A second acquisition unit is used to acquire a real-time vehicle image; The detection unit is used to perform vehicle heading angle detection through the vehicle heading angle detection model and the real-time vehicle image to obtain a real-time heading angle detection result.
7. The vehicle heading angle detection device according to claim 6, characterized in that: The first acquiring unit includes: The acquisition subunit is used to collect original sample images containing various vehicle scenes; The labeling subunit is used to label the original sample image with a vehicle 2D frame, a vehicle, and a heading angle to obtain a labeled sample image; The construction subunit is used to construct a training data set and a test data set based on the labeled sample images.
8. The vehicle heading angle detection device according to claim 6, characterized in that: The test unit comprises: An extraction subunit, configured to perform feature extraction on the test data set through the ResNet backbone network to obtain image features corresponding to each picture in the test data set; A fusion subunit, used to perform feature fusion on the image features through the FPN feature fusion network to obtain image fusion features corresponding to each picture in the test data set; A first detection subunit is used to perform vehicle 2D frame detection according to the first detection head and the image fusion feature, and obtain a vehicle 2D frame feature corresponding to each image in the test data set; A second detection subunit is used to perform heading angle interval detection according to the second detection head and the image fusion feature to obtain a vehicle heading angle interval corresponding to each image in the test data set; A third detection subunit is used to perform heading angle detection according to the third detection head and the image fusion feature to obtain a vehicle heading angle feature corresponding to each image in the test data set; The summarizing subunit is used to summarize the vehicle 2D frame features, the vehicle heading angle interval and the vehicle heading angle features to obtain a test set prediction result.
9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the vehicle heading angle detection method according to any one of claims 1 to 5.
10. A readable storage medium, characterized in that: The readable storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the vehicle heading angle detection method according to any one of claims 1 to 5 is executed.