Vehicle distance and course angle detection method and device, storage medium and equipment

Abstract

The invention provides a vehicle distance and course angle detection method and device, a storage medium and equipment, and the method comprises the steps: obtaining tire grounding points and vehicle 3D target detection frames of all vehicles in a target image, screening out the tire grounding points and the vehicle 3D target detection frames within a set detection distance, carrying out the clustering of the screened tire grounding points, and obtaining a target image; the method comprises the following steps: firstly, clustering a plurality of vehicle 3D target detection frames, then matching each clustered tire grounding point cluster with the screened vehicle 3D target detection frames, determining a membership relationship between each tire grounding point cluster and each vehicle, and finally projecting a corresponding 2D tire grounding point in each vehicle 3D target detection frame back to a 3D coordinate system to obtain a coordinate value of the corresponding 3D grounding point. And the vehicle distance and the vehicle course angle are calculated. Therefore, the tire grounding point is used as the detection point, and the vehicle 3D target detection frame is fused for vehicle detection, so that the accuracy of vehicle distance and course angle detection is effectively improved, and the vehicle driving safety and the riding experience feeling are improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle detection technology, and in particular to a method, device, storage medium and equipment for detecting vehicle distance and heading angle. Background Art

[0002] Environmental perception is the basis and prerequisite for autonomous driving of intelligent vehicles, which involves technologies such as road boundary detection, obstacle detection, vehicle detection, and pedestrian detection. Among them, vehicle detection technology generally uses sensors such as cameras and lidar to sense whether there are other vehicles in the environment around the vehicle and obtain their location information, providing input information for later obstacle avoidance and path planning. Vehicle distance and heading angle detection is an important basic task in vehicle detection. The accuracy of vehicle distance and heading angle directly affects the safety and riding experience of intelligent driving.

[0003] The vehicle distance and heading angle detection methods in related technologies mainly use neural network models to detect vehicles appearing in 2D images or 3D point clouds. The detection results are given in the form of 3D detection frames, which usually contain information such as the center point coordinates, 8 corner point coordinates and heading angles of the 3D frame. However, when there are untrained special-shaped vehicles in the image or point cloud, incomplete vehicle photography, low material reflectivity, etc., it is easy to cause inaccurate estimation of vehicle distance and heading angle, which in turn leads to no braking or unreasonable braking behavior, affecting the vehicle's driving safety and riding experience. Summary of the invention

[0004] The purpose of the present application is to provide a vehicle distance and heading angle detection method, device, storage medium and equipment, aiming to solve the problem of inaccurate vehicle distance and heading angle estimation in the vehicle distance and heading angle detection methods in the related art, which easily leads to no braking or unreasonable braking behavior, thereby affecting the vehicle driving safety and riding experience.

[0005] In a first aspect, the present application provides a vehicle distance and heading angle detection method, comprising: obtaining tire contact points and vehicle 3D target detection frames of all vehicles in a target image, and screening out tire contact points and vehicle 3D target detection frames within a set detection distance; the set detection distance is set in a three-dimensional coordinate system with the vehicle as the starting point; clustering the screened tire contact points, and matching each clustered tire contact point cluster with the screened vehicle 3D target detection frame; traversing the screened vehicle 3D target detection frame, projecting the matched tire contact points in the frame to the three-dimensional coordinate system, obtaining the coordinate values ​​of the corresponding 3D contact points, and calculating the vehicle distance and vehicle heading angle according to the coordinate values ​​of the 3D contact points.

[0006] In the above implementation process, the tire contact points and vehicle 3D target detection frames of all vehicles in the target image are obtained, and the tire contact points and vehicle 3D target detection frames within the set detection distance are screened out. After that, the screened tire contact points are clustered to form several tire contact point clusters, and then each tire contact point cluster is matched with the screened vehicle 3D target detection frame to determine the affiliation of each tire contact point cluster with each vehicle. Finally, the corresponding 2D tire contact points in each vehicle 3D target detection frame are projected back to the 3D coordinate system to obtain the coordinate value of the corresponding 3D contact point, and the vehicle distance and vehicle heading angle are calculated according to the coordinate value of the 3D contact point. In this way, using the tire contact points as detection points and integrating the vehicle 3D target detection frame to perform vehicle detection can effectively solve the phenomenon of missed detection or inaccurate detection caused by problems such as untrained special-shaped vehicles, incomplete vehicle shooting, and low reflectivity of vehicle materials in vehicle detection tasks, improve the accuracy of vehicle distance and heading angle detection, avoid no braking or unreasonable braking behavior, and effectively improve vehicle driving safety and riding experience.

[0007] Furthermore, in some examples, the obtaining of tire contact points and vehicle 3D target detection frames of all vehicles in the target image includes: predicting the tire contact points of all vehicles in the target image through a pre-trained neural network model, and obtaining the vehicle 3D target detection frame in the target image through a pre-trained 3D target detection model; the tire contact points include 2D point coordinate values ​​and attribute values ​​indicating that the tire contact points belong to one of the front, rear or side of the vehicle.

[0008] In the above implementation process, the tire contact point and the vehicle 3D target detection frame in the image are respectively obtained using a pre-trained neural network model and a 3D target detection model, which can improve the detection accuracy.

[0009] Furthermore, in some examples, the screening out of the tire contact point and the vehicle 3D target detection frame within the set detection distance includes: using a dynamic ramp model to project the set detection distance onto a 2D image to obtain pixel row coordinates, and obtaining the image to-be-detected area according to the pixel row coordinates; the dynamic ramp model dynamically obtains a 3D to 2D projection matrix according to the current road slope; and screening out the tire contact point and the vehicle 3D target detection frame within the image to-be-detected area.

[0010] In the above implementation process, using a dynamic ramp model to project the set detection distance onto a 2D image can better adapt to the ups and downs of real roads, thereby improving the accuracy of the projection.

[0011] Furthermore, in some examples, clustering the screened tire contact points includes: performing transverse clustering on the screened tire contact points to obtain transverse clusters of tire contact points; and performing longitudinal clustering on the transverse clusters of tire contact points.

[0012] In the above implementation process, the tire contact points in the image to be detected area are clustered horizontally and vertically in turn, so as to obtain the tire contact point clusters at the front, rear and side of the vehicle, which lays a good foundation for the subsequent determination of the affiliation of each tire contact point cluster with each vehicle.

[0013] Further, in some examples, the matching of each clustered tire contact point cluster with the filtered vehicle 3D object detection frame includes: traversing the filtered vehicle 3D object detection frame, if the proportion of any tire contact point cluster among all the clustered tire contact point clusters falling into the current vehicle 3D object detection frame is greater than a matching threshold corresponding to the number of points in the tire contact point cluster, storing the proportion in a matching list of the current vehicle 3D object detection frame; and determining the tire contact point cluster corresponding to the maximum value in the matching list as the tire contact point cluster matching the vehicle 3D object detection frame.

[0014] In the above implementation process, a specific method for determining the affiliation between each tire contact point cluster and each vehicle is provided.

[0015] Furthermore, in some examples, the coordinate value of the 3D ground point is calculated based on the following formula:

[0016]

[0017] Where θ is the angle between the camera optical axis and the slope surface; f x 、f y 、c x 、c y is the camera internal parameter; y, z are the coordinates of the starting reference point of the ramp plane in the camera coordinate system; u, v are the pixel column coordinates and row coordinates of the tire contact point in the image; x cam ,y cam 、z cam is the 3D coordinate of the tire contact point in the camera coordinate system; X, Y, Z are the coordinate values ​​of the 3D contact point; R cam_ego is the rotation matrix from the camera coordinate system to the vehicle coordinate system; T cam_ego is the translation vector from the camera coordinate system to the vehicle coordinate system.

[0018] In the above implementation process, a specific method for calculating the coordinate value of the 3D grounding point is provided. In this process, a dynamic ramp model is used to project the 2D grounding point in the image into a 3D coordinate system, which can better cope with the actual road undulations.

[0019] Further, in some examples, the 3D grounding points include a 3D grounding point of a front tire, a 3D grounding point of a rear tire, and a 3D grounding point of a side tire; the calculating of the vehicle distance and the vehicle heading angle according to the coordinate values ​​of the 3D grounding points includes: obtaining the distance from the vehicle to the front of the target vehicle according to the coordinate value of the point closest to the vehicle among the 3D grounding points of the front tire of the target vehicle, and the distance from the tire grounding point to the front of the vehicle, or obtaining the distance from the vehicle to the rear of the target vehicle according to the coordinate value of the point closest to the vehicle among the 3D grounding points of the rear tire of the target vehicle, and the distance from the tire grounding point to the rear of the vehicle; performing straight line fitting on the 3D grounding points of the side tires of the target vehicle, and calculating the vehicle heading angle according to the fitting results.

[0020] In the above implementation process, a specific method for calculating the vehicle distance and the vehicle heading angle according to the coordinate value of the 3D grounding point is provided, which can accurately calculate the vehicle distance and the vehicle heading angle, thereby reducing the probability of no braking or unreasonable braking behavior.

[0021] In a second aspect, the present application provides a vehicle distance and heading angle detection device, including: a screening module, used to obtain the tire contact points and vehicle 3D target detection frames of all vehicles in the target image, and screen out the tire contact points and vehicle 3D target detection frames within a set detection distance; the set detection distance is set in a three-dimensional coordinate system with the vehicle as the starting point; a matching module, used to cluster the screened tire contact points, and match each clustered tire contact point cluster with the screened vehicle 3D target detection frame; a projection module, used to traverse the screened vehicle 3D target detection frame, project the matched tire contact points in the frame to the three-dimensional coordinate system, obtain the coordinate value of the corresponding 3D contact point, and calculate the vehicle distance and vehicle heading angle according to the coordinate value of the 3D contact point.

[0022] In a third aspect, the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in any one of the first aspects when executing the computer program.

[0023] In a fourth aspect, the present application provides a computer-readable storage medium, on which instructions are stored. When the instructions are executed on a computer, the computer executes the method as described in any one of the first aspects.

[0024] In a fifth aspect, the present application provides a computer program product, which, when running on a computer, enables the computer to execute the method as described in any one of the first aspects.

[0025] Other features and advantages disclosed in the present application will be described in the following description, or some features and advantages can be inferred or determined without doubt from the description, or can be learned by implementing the above-mentioned technology disclosed in the present application.

[0026] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0028] Figure 1 A flow chart of a vehicle distance and heading angle detection method provided in an embodiment of the present application;

[0029] Figure 2 A schematic diagram of the working process of a vehicle distance and heading angle detection solution provided in an embodiment of the present application;

[0030] Figure 3 A block diagram of a vehicle distance and heading angle detection device provided in an embodiment of the present application;

[0031] Figure 4 A structural block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0033] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0034] As described in the background technology, the vehicle distance and heading angle detection methods in the related art have the problem of inaccurate estimation of vehicle distance and heading angle, which easily leads to no braking or unreasonable braking behavior, thereby affecting the vehicle driving safety and riding experience. Based on this, the embodiment of the present application provides a new vehicle distance and heading angle detection solution to solve the above problems.

[0035] Next, the embodiments of the present application are introduced:

[0036] like Figure 1 As shown, Figure 1 The present invention provides a flow chart of a method for detecting vehicle distance and heading angle, which can be applied to a relevant controller in a driving assistance system on a vehicle.

[0037] The method comprises:

[0038] Step 101, obtaining tire contact points and vehicle 3D target detection frames of all vehicles in the target image, and screening out tire contact points and vehicle 3D target detection frames within a set detection distance; the set detection distance is set in a three-dimensional coordinate system with the vehicle as the starting point;

[0039] The target image mentioned in this step can be collected by a vehicle-mounted camera installed in front of the vehicle. When the current vehicle is driving, the vehicle-mounted camera is used to capture the environmental image in front of the vehicle in real time, that is, the target image. The vehicle in the target image can be other vehicles within the field of view of the vehicle-mounted camera of the current vehicle. In this embodiment, the tire contact point is used as the detection point, and the visual 3D target frame detection result is integrated to calculate the vehicle distance and heading angle information, thereby improving the accuracy of vehicle distance and heading angle detection.

[0040] In some embodiments, the step of obtaining the tire contact points and the vehicle 3D target detection frame of all vehicles in the target image mentioned in this step may include: predicting the tire contact points of all vehicles in the target image through a pre-trained neural network model, and obtaining the vehicle 3D target detection frame in the target image through a pre-trained 3D target detection model; the tire contact points include 2D point coordinate values ​​and attribute values ​​indicating that the tire contact points belong to one of the front, rear or side of the vehicle. In other words, the tire contact points of the front, rear and side of the vehicle are predicted using a pre-trained neural network model, and each predicted point includes a 2D coordinate value and an attribute value belonging to one of the front, rear or side of the vehicle. At the same time, the pre-trained 3D target detection model is used to detect the vehicles appearing in the target image to obtain the vehicle 3D target detection frame. The neural network model can be a refined target detection model, such as the YOLO model, the SSD (Single Shot MultiBox Detector, single-step multi-box target detection) model, etc., or a semantic segmentation model, such as the DeepLab model, the UNet model, etc.; and the 3D target detection model can be a vision-based 3D target detection model, such as the Mono3D model, the Deep3DBox model, etc. The specific training process of these two models can refer to the introduction of the corresponding algorithms in the relevant technology, and this application will not repeat them here.

[0041] After acquiring all tire contact points and vehicle 3D target detection frames in the target image, these tire contact points and vehicle 3D target detection frames are screened according to the set detection distance. The set detection distance is set in a three-dimensional coordinate system with the projection of the front bumper of the vehicle on the road as the starting point. The three-dimensional coordinate system here can be a BEV (Bird's-Eye-View) coordinate system. Through this screening operation, other vehicles that are far away from the current vehicle and have little impact on driving safety can be excluded, reducing the possibility of false detection, while ensuring that all other vehicles that have an important impact on driving safety are detected, reducing the risk of missed detection.

[0042] In some embodiments, the step of filtering out the tire contact point and the vehicle 3D target detection frame within the set detection distance mentioned in this step may include: using a dynamic ramp model, projecting the set detection distance onto a 2D image to obtain pixel row coordinates, and obtaining the image to be detected area according to the pixel row coordinates; the dynamic ramp model dynamically obtains a 3D to 2D projection matrix according to the current road slope; and filtering out the tire contact point and the vehicle 3D target detection frame within the image to be detected area. That is, the set detection distance in the BEV coordinate system is projected onto the 2D image coordinate system through the dynamic ramp model to obtain the image to be detected area. The dynamic ramp model is a mathematical model that models the road surface in front of the vehicle into multiple continuous planes with different slope angles. Its input is the camera internal and external parameters and the slope angle, and its output is the 3D coordinate value of the tire contact point in the BEV coordinate system. The dynamic ramp model dynamically obtains a 3D to 2D projection matrix according to the current road slope, so that it can better adapt to the ups and downs of the actual road and improve the accuracy of the projection. Therefore, the obtained image to be detected area is more accurate. Among them, the pixel row coordinates can be calculated based on the following formula:

[0043]

[0044] Among them, R thres is the pixel row coordinate projected onto the 2D image by setting the detection distance; C is the pixel column coordinate; f x 、f y 、c x 、c y is the camera internal parameter; x ego is the x-axis coordinate of the center point of the image in the three-dimensional coordinate system, and y thres is the set detection distance in the three-dimensional coordinate system, z ego is the z-axis coordinate of the center point of the image in the three-dimensional coordinate system; R ego_cam is the rotation matrix from the vehicle coordinate system to the camera coordinate system; T ego_cam is the translation vector from the vehicle coordinate system to the camera coordinate system. In implementation, the camera internal parameter f x 、fy 、c x 、c y And the rotation matrix R from the vehicle coordinate system to the camera coordinate system ego_cam and the translation vector T ego_cam It can be obtained through the calibration process. After substituting these values ​​into the above formula, the pixel row coordinates projected onto the 2D image at the set detection distance can be quickly and accurately calculated.

[0045] Step 102: clustering the screened tire contact points, and matching each clustered tire contact point cluster with the screened vehicle 3D target detection frame;

[0046] In the present embodiment, the screened tire contact points are clustered to form a plurality of tire contact point clusters, and then each tire contact point cluster is matched with the screened vehicle 3D target detection frame to determine the affiliation of each tire contact point cluster with each vehicle.

[0047] In some embodiments, clustering the selected tire contact points mentioned in this step may include: performing horizontal clustering on the selected tire contact points to obtain horizontal clusters of tire contact points; and performing vertical clustering on the horizontal clusters of tire contact points. In other words, the tire contact points in the image to be detected area are clustered horizontally and vertically in sequence, thereby obtaining tire contact point clusters at the front, rear, and side of the vehicle, which lays a good foundation for subsequently determining the affiliation between each tire contact point cluster and each vehicle.

[0048] Furthermore, in some embodiments, the matching of each clustered tire contact point cluster with the screened vehicle 3D target detection frame mentioned in this step may include: traversing the screened vehicle 3D target detection frame, and if the proportion of any tire contact point cluster among all the clustered tire contact point clusters that falls into the current vehicle 3D target detection frame is greater than the matching threshold corresponding to the number of points in the tire contact point cluster, storing the proportion in the matching list of the current vehicle 3D target detection frame; and determining the tire contact point cluster corresponding to the maximum value in the matching list as the tire contact point cluster that matches the vehicle 3D target detection frame. In other words, different matching thresholds are set for tire contact point clusters with different numbers of points, and when matching, the screened vehicle 3D target detection frame is traversed to calculate the proportion P of the j-th tire contact point cluster that falls into the i-th 3D target detection frame. ij =N ji / N j , where N ji N represents the number of points of the jth tire contact point cluster falling into the i-th 3D object detection box, j represents the total number of points in the jth tire contact point cluster. If P ijIf the number of points in the jth tire contact point cluster is greater than the matching threshold, then P ij Store the matching list P of the i-th 3D target detection box i Finally, the matching list P i The tire contact point cluster corresponding to the maximum value in is the tire contact point cluster that matches the i-th 3D object detection frame, that is, the tire contact point cluster belongs to the i-th vehicle. This is helpful to improve the accuracy of vehicle distance and heading angle detection.

[0049] Step 103, traverse the filtered vehicle 3D target detection frame, project the matched tire grounding point in the frame to the three-dimensional coordinate system, obtain the coordinate value of the corresponding 3D grounding point, and calculate the vehicle distance and vehicle heading angle according to the coordinate value of the 3D grounding point.

[0050] This step means: after completing the matching of the tire contact point cluster and the vehicle 3D target detection frame, the corresponding 2D tire contact point in each vehicle 3D target detection frame is projected back to the 3D coordinate system to obtain the corresponding 3D contact point. The coordinates of the 3D contact point at this time can reflect the position and posture of the vehicle. Therefore, according to the coordinate values ​​of the 3D contact point, the distance and heading angle of the vehicle corresponding to the currently traversed vehicle 3D target detection frame can be calculated respectively.

[0051] In some embodiments, the coordinate value of the 3D ground point mentioned in this step can be calculated based on the following formula:

[0052]

[0053] Where θ is the angle between the camera optical axis and the slope surface; f x 、f y 、c x 、c y is the camera internal parameter; y, z are the coordinates of the starting reference point of the ramp plane in the camera coordinate system; u, v are the pixel column coordinates and row coordinates of the tire contact point in the image; x cam ,y cam 、z cam is the 3D coordinate of the tire contact point in the camera coordinate system; X, Y, Z are the coordinate values ​​of the 3D contact point; R cam_ego is the rotation matrix from the camera coordinate system to the vehicle coordinate system; T cam_ego is the translation vector from the camera coordinate system to the vehicle coordinate system. In the above formula, x cam ,y cam 、z cam It is calculated using a dynamic ramp model, so it can better cope with the actual road conditions and improve the accuracy of the projection.

[0054] Furthermore, in some embodiments, the 3D grounding points mentioned in this step include the 3D grounding points of the front tires, the 3D grounding points of the rear tires, and the 3D grounding points of the side tires; accordingly, the calculation of the vehicle distance and the vehicle heading angle according to the coordinate values ​​of the 3D grounding points mentioned in this step may include: obtaining the distance from the vehicle to the front of the target vehicle according to the coordinate value of the point closest to the vehicle among the 3D grounding points of the front tires of the target vehicle, and the distance from the tire grounding point to the front of the vehicle, or obtaining the distance from the vehicle to the rear of the target vehicle according to the coordinate value of the point closest to the vehicle among the 3D grounding points of the rear tires of the target vehicle, and the distance from the tire grounding point to the rear of the vehicle; performing straight line fitting on the 3D grounding points of the side tires of the target vehicle, and calculating the vehicle heading angle according to the fitting results. That is to say, when calculating the vehicle distance, the distance from the vehicle to the front of the i-th vehicle can be obtained by using the point closest to the vehicle among the 3D grounding points of the front tire in the 3D target detection frame of the i-th vehicle, minus the distance from the tire grounding point to the vehicle front, and the distance from the vehicle to the rear of the i-th vehicle can be obtained by using the point closest to the vehicle among the 3D grounding points of the rear tire in the 3D target detection frame of the i-th vehicle, minus the distance from the tire grounding point to the vehicle rear; when calculating the vehicle heading angle, the vehicle heading angle of the i-th vehicle can be calculated by linear fitting the 3D grounding points of the side tires in the 3D target detection frame of the i-th vehicle using the least squares method, RANSAC (Random Sample Consensus) algorithm, etc. In this way, the vehicle distance and vehicle heading angle are accurately calculated, reducing the probability of not braking or unreasonable braking behavior.

[0055] In the embodiment of the present application, the tire contact points and vehicle 3D target detection frames of all vehicles in the target image are obtained, and the tire contact points and vehicle 3D target detection frames within the set detection distance are screened out. After that, the screened tire contact points are clustered to form a number of tire contact point clusters, and then each tire contact point cluster is matched with the screened vehicle 3D target detection frame to determine the affiliation of each tire contact point cluster with each vehicle. Finally, the corresponding 2D tire contact points in each vehicle 3D target detection frame are projected back to the 3D coordinate system to obtain the coordinate value of the corresponding 3D contact point, and the vehicle distance and vehicle heading angle are calculated according to the coordinate value of the 3D contact point. In this way, using the tire contact points as detection points and integrating the vehicle 3D target detection frame to perform vehicle detection can effectively solve the phenomenon of missed detection or inaccurate detection caused by problems such as untrained special-shaped vehicles, incomplete vehicle shooting, and low reflectivity of vehicle materials in the vehicle detection task, improve the accuracy of vehicle distance and heading angle detection, avoid no braking or unreasonable braking behavior, and effectively improve vehicle driving safety and riding experience.

[0056] In order to explain the solution of this application in more detail, a specific embodiment is introduced below:

[0057] This embodiment provides a vehicle distance and heading angle detection solution. The workflow of this embodiment is as follows Figure 2 As shown, including:

[0058] S201, predicting the vehicle tire contact point using a neural network model;

[0059] Specifically, a pre-trained neural network model is used to predict the front or rear tire contact points (only one of the two can appear at the same time) and the side tire contact points of all vehicles in the image, wherein each predicted point includes a 2D point coordinate value and an attribute value belonging to one of the front, rear or side of the vehicle; the image is an image of the road in front of the vehicle, and there is at least one other vehicle in the image;

[0060] S202, screening tire contact points and 3D target detection frames within a set detection distance;

[0061] Specifically, the detection distance set with the vehicle as the starting point is projected onto the 2D image to obtain the image area to be detected; in implementation, in the 3D BEV coordinate system, the detection distance y is set with the projection of the front bumper of the vehicle on the road as the starting point. thres Considering the ups and downs of real roads, if the IPM (Inverse Perspective Mapping) plane assumption algorithm with no ups and downs is used for projection, the pixel row coordinates will be inaccurate on the ups and downs of the road. Therefore, the dynamic ramp model is used to detect the distance y thres Project it onto the image and get the pixel row coordinates R thres , the dynamic ramp model will dynamically obtain the 3D to 2D projection matrix according to the current road slope, thereby improving the accuracy of the projection; this projection process can be expressed as:

[0062]

[0063] Among them, f x 、f y 、c x 、c y is the camera internal parameter; R ego_cam is the rotation matrix from the vehicle coordinate system to the camera coordinate system; T ego_cam is the translation vector from the vehicle coordinate system to the camera coordinate system;

[0064] Afterwards, based on the obtained area to be detected, the image [R thres ,height-1] The vehicle 3D target detection frame and tire contact point within the row coordinate range; where height is the height of the image;

[0065] S203, clustering the tire contact points at the front, rear and side of the vehicle respectively;

[0066] Specifically, firstly, the three types of tire contact points are clustered horizontally to obtain the tire contact point lateral clusters; in implementation, the lateral distance interval threshold of the set point is U thres , loop through all points, compare the distances with the tire contact point clusters that have been horizontally clustered point by point, if the column coordinates of the point fall within the left and right range of the tire contact point horizontal cluster, then add the point to the tire contact point cluster, otherwise add the point to the new tire contact point cluster. This process can be expressed as:

[0067]

[0068] A j =min(Ucluster j .x)-U thres

[0069] B j =max(Ucluster j .x)+U thres

[0070] Among them, point i .x represents the column coordinates of the i-th point; Ucluster j .x represents the list of column coordinates of all points in the jth tire contact point cluster; Ucluster n+1 Indicates the newly created tire contact point cluster;

[0071] Then, the lateral clusters of tire contact points are clustered longitudinally. When implementing this, the longitudinal distance interval threshold of the lateral clusters is set to V thres , loop through all tire contact point transverse clusters, and compare the distances of each cluster with the tire contact point longitudinal clusters that have been longitudinally clustered. If a transverse cluster falls within the interval threshold set by the current tire contact point longitudinal cluster, the transverse cluster is added to the current tire contact point longitudinal cluster, and the total number of tire contact point clusters is reduced by 1 accordingly. This process can be expressed as:

[0072]

[0073] C j =max(Vcluster j .y)

[0074] Among them, Ucluster i .y represents a list of row coordinates of the i-th horizontal cluster; Vcluster j .y represents a list of row coordinates of all points in the jth tire contact point cluster;

[0075] S204, matching the clustered tire contact point clusters of each category with the vehicle 3D target detection frame;

[0076] Specifically, as shown in the following Table 1, different matching thresholds are set for tire contact point clusters with different numbers of points. Table 1 is a query table of matching thresholds corresponding to tire contact point clusters with different numbers of points, and its content is:

[0077] Table 1. Lookup table of matching thresholds corresponding to tire contact point clusters with different numbers of points

[0078]

[0079] The above matching threshold can be adjusted according to actual conditions;

[0080] During matching, loop through the selected vehicle 3D target detection frames and calculate the proportion P of the jth tire contact point cluster falling into the i-th 3D target detection frame. ij =N ji / N j , where N ji N represents the number of points of the jth tire contact point cluster falling into the i-th 3D object detection box, j represents the total number of points in the jth tire contact point cluster. If P ij If it is greater than the corresponding matching threshold in Table 1, then P ij Store the matching list P of the i-th 3D target detection box i Finally, the matching list P i The tire contact point cluster corresponding to the maximum value in is the tire contact point cluster that matches the i-th 3D object detection frame, that is, the tire contact point cluster belongs to the i-th vehicle;

[0081] S205, traversing the selected vehicle 3D target detection frame, projecting the matched 2D grounding point in the frame to the BEV coordinate system, and obtaining the corresponding 3D grounding point;

[0082] Specifically, in order to cope with the actual road undulations, the 2D grounding point in the image is projected into the 3D BEV coordinate system using a dynamic ramp model. This process can be expressed as:

[0083]

[0084] Where θ is the angle between the camera optical axis and the slope surface; f x 、f y 、c x 、c y is the camera internal parameter; y, z are the coordinates of the starting reference point of the ramp plane in the camera coordinate system; u, v are the pixel column coordinates and row coordinates of the tire contact point in the image; x cam ,ycam 、z cam is the 3D coordinate of the tire contact point in the camera coordinate system; X, Y, Z are the 3D coordinate values ​​of the tire contact point in the BEV coordinate system; R cam_ego is the rotation matrix from the camera coordinate system to the vehicle coordinate system; T cam_ego is the translation vector from the camera coordinate system to the vehicle coordinate system;

[0085] S206, calculating the vehicle distance and vehicle heading angle;

[0086] Specifically, when calculating the vehicle distance, the distance from the vehicle to the front of the i-th vehicle can be obtained by subtracting the distance from the tire contact point to the front of the vehicle from the point closest to the vehicle among the 3D contact points of the front tire in the 3D target detection frame of the i-th vehicle; the distance from the vehicle to the rear of the i-th vehicle can be obtained by subtracting the distance from the tire contact point to the rear of the vehicle from the point closest to the vehicle among the 3D contact points of the rear tire in the 3D target detection frame of the i-th vehicle;

[0087] When calculating the vehicle heading angle, the vehicle heading angle of the i-th vehicle is calculated based on the result of linear fitting of the 3D ground contact point of the side tire within the 3D target detection frame of the i-th vehicle.

[0088] This embodiment scheme uses a pre-trained neural network model to detect the contact points of the front, rear and side tires of the vehicle, and integrates the visual 3D target frame detection results to calculate more accurate vehicle distance and heading angle information, thereby effectively solving the problem of missed detection or inaccurate detection caused by problems such as untrained special-shaped vehicles, incomplete vehicle photography, and low reflectivity of vehicle materials in visual or point cloud vehicle detection tasks.

[0089] Corresponding to the embodiments of the aforementioned method, the present application also provides embodiments of a vehicle distance and heading angle detection device and a terminal using the same:

[0090] like Figure 3 As shown, Figure 3 : is a block diagram of a vehicle distance and heading angle detection device provided in an embodiment of the present application, the device comprising:

[0091] The screening module 31 is used to obtain the tire contact points and vehicle 3D target detection frames of all vehicles in the target image, and screen out the tire contact points and vehicle 3D target detection frames within a set detection distance; the set detection distance is set in a three-dimensional coordinate system with the vehicle as the starting point;

[0092] A matching module 32 is used to cluster the screened tire contact points and match each clustered tire contact point cluster with the screened vehicle 3D target detection frame;

[0093] The projection module 33 is used to traverse the filtered vehicle 3D target detection frame, project the matched tire grounding point in the frame to the three-dimensional coordinate system, obtain the coordinate value of the corresponding 3D grounding point, and calculate the vehicle distance and vehicle heading angle according to the coordinate value of the 3D grounding point.

[0094] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, which will not be repeated here.

[0095] This application also provides an electronic device, see Figure 4 , Figure 4 A block diagram of an electronic device provided in an embodiment of the present application. The electronic device may include a processor 410, a communication interface 420, a memory 430, and at least one communication bus 440. The communication bus 440 is used to realize direct connection and communication between these components. The communication interface 420 of the electronic device in the embodiment of the present application is used to communicate signaling or data with other node devices. The processor 410 may be an integrated circuit chip with signal processing capabilities.

[0096] The processor 410 mentioned above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor 410 can also be any conventional processor, etc.

[0097] The memory 430 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electric erasable programmable read-only memory (EEPROM), etc. The memory 430 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 410, the electronic device can execute the above-mentioned Figure 1 The method embodiment involves various steps.

[0098] Optionally, the electronic device may further include a storage controller and an input / output unit.

[0099] The memory 430, storage controller, processor 410, peripheral interface, input and output unit components are directly or indirectly electrically connected to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses 440. The processor 410 is used to execute executable modules stored in the memory 430, such as software function modules or computer programs included in the electronic device.

[0100] The input and output unit is used to provide users with the task creation and to create a start optional time period or preset execution time for the task to realize the interaction between the user and the server. The input and output unit can be, but is not limited to, a mouse and a keyboard.

[0101] Understandably, Figure 4 The structure shown is for illustration only, and the electronic device may also include Figure 4 More or fewer components as shown, or with Figure 4 Different configurations are shown. Figure 4 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0102] An embodiment of the present application further provides a storage medium having instructions stored thereon. When the instructions are run on a computer, the computer program is executed by a processor to implement the method described in the method embodiment. To avoid repetition, the method will not be described here.

[0103] The present application also provides a computer program product, which, when executed on a computer, enables the computer to execute the method described in the method embodiment.

[0104] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0105] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0106] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0107] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0108] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0109] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A vehicle distance and heading angle detection method, characterized in that: include: Obtain the tire contact points and vehicle 3D target detection frames of all vehicles in the target image, and filter out the tire contact points and vehicle 3D target detection frames within a set detection distance; the set detection distance is set in a three-dimensional coordinate system with the vehicle as the starting point; Clustering the screened tire contact points, and matching each clustered tire contact point cluster with the screened vehicle 3D target detection frame; The filtered vehicle 3D target detection frame is traversed, and the matched tire contact points in the frame are projected into the three-dimensional coordinate system to obtain the coordinate values ​​of the corresponding 3D contact points, and the vehicle distance and vehicle heading angle are calculated according to the coordinate values ​​of the 3D contact points.

2. The method according to claim 1, characterized in that The step of obtaining tire contact points and vehicle 3D target detection frames of all vehicles in the target image includes: The tire contact points of all vehicles in the target image are predicted by a pre-trained neural network model, and the vehicle 3D target detection frame in the target image is obtained by a pre-trained 3D target detection model; the tire contact points include 2D point coordinate values ​​and attribute values ​​indicating that the tire contact points belong to one of the front, rear or side of the vehicle.

3. The method according to claim 1, characterized in that: The step of screening out the tire contact point and the vehicle 3D target detection frame within the set detection distance includes: Using a dynamic ramp model, the set detection distance is projected onto a 2D image to obtain pixel row coordinates, and the image to-be-detected area is obtained according to the pixel row coordinates; the dynamic ramp model dynamically obtains a 3D-to-2D projection matrix according to the current road slope; The tire contact point and the vehicle 3D target detection frame in the image to be detected are screened out.

4. The method according to claim 3, characterized in that The clustering of the screened tire contact points includes: Performing horizontal clustering on the screened tire contact points to obtain horizontal clusters of tire contact points; The tire contact point lateral clusters are clustered longitudinally.

5. The method according to claim 1, characterized in that The step of matching the clustered tire contact point clusters with the screened vehicle 3D target detection frame includes: Traversing the filtered vehicle 3D object detection frame, if the proportion of any tire contact point cluster among all clustered tire contact point clusters falling into the current vehicle 3D object detection frame is greater than the matching threshold corresponding to the number of points in the tire contact point cluster, storing the proportion in the matching list of the current vehicle 3D object detection frame; The tire contact point cluster corresponding to the maximum value in the matching list is determined as the tire contact point cluster matching the vehicle 3D object detection frame.

6. The method according to claim 3, characterized in that The coordinates of the 3D ground point are calculated based on the following formula: Where θ is the angle between the camera optical axis and the slope surface; f x 、f y 、c x 、c y is the camera internal parameter; y, z are the coordinates of the starting reference point of the ramp plane in the camera coordinate system; u, v are the pixel column coordinates and row coordinates of the tire contact point in the image; x cam ,y cam 、z cam is the 3D coordinate of the tire contact point in the camera coordinate system; X, Y, Z are the coordinate values ​​of the 3D contact point; R cam_ego is the rotation matrix from the camera coordinate system to the vehicle coordinate system; T cam_ego is the translation vector from the camera coordinate system to the vehicle coordinate system.

7. The method according to claim 6, characterized in that The 3D grounding points include the front tire 3D grounding point, the rear tire 3D grounding point and the side tire 3D grounding point; The calculating the vehicle distance and the vehicle heading angle according to the coordinate value of the 3D ground point includes: The distance from the vehicle to the front of the target vehicle is obtained according to the coordinate value of the point closest to the vehicle among the 3D ground contact points of the front tire of the target vehicle and the distance from the tire ground contact point to the front of the vehicle, or the distance from the vehicle to the rear of the target vehicle is obtained according to the coordinate value of the point closest to the vehicle among the 3D ground contact points of the rear tire of the target vehicle and the distance from the tire ground contact point to the rear of the vehicle; A straight line fitting is performed on the 3D contact point of the side tire of the target vehicle, and the vehicle heading angle is calculated according to the fitting result.

8. A vehicle distance and heading angle detection device, characterized in that: include: A screening module is used to obtain the tire contact points and vehicle 3D target detection frames of all vehicles in the target image, and screen out the tire contact points and vehicle 3D target detection frames within a set detection distance; the set detection distance is set in a three-dimensional coordinate system with the vehicle as the starting point; A matching module is used to cluster the screened tire contact points and match each clustered tire contact point cluster with the screened vehicle 3D target detection frame; The projection module is used to traverse the filtered vehicle 3D target detection frame, project the matched tire grounding point in the frame to the three-dimensional coordinate system, obtain the coordinate value of the corresponding 3D grounding point, and calculate the vehicle distance and vehicle heading angle according to the coordinate value of the 3D grounding point.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images