Vehicle detection method, device, equipment and medium

Abstract

The application discloses a vehicle detection method, device, equipment and medium, and belongs to the technical field of automatic driving. In the application, first, lane lines in a photographed road image are detected, vehicles in the road image and ground contact key points of the vehicles are detected, and the ground contact key points of the vehicles in the same lane are corrected to obtain corrected ground contact key points of the vehicles; then, vehicle driving information of the vehicles is determined according to the corrected ground contact key points of the vehicles. By selecting the key points of the wheel contact points and correcting the key points of the vehicles in the same lane, target detection of the vehicles is realized through a two-dimensional image in the automatic driving process, and then accurate vehicle driving information of the vehicles is obtained.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a vehicle detection method, a vehicle detection device, a vehicle detection equipment, and a computer-readable storage medium. Background Technology

[0002] Currently, in the process of autonomous driving, vehicle target detection is rarely achieved through two-dimensional images captured by conventional cameras. Instead, it is achieved more by using point cloud data from high-precision LiDAR sensors. However, this LiDAR solution is relatively expensive. Summary of the Invention

[0003] The main objective of this application is to provide a vehicle detection method, a vehicle detection device, a vehicle detection equipment, and a computer-readable storage medium, which aim to achieve vehicle target detection through two-dimensional images during autonomous driving.

[0004] To achieve the above objectives, this application provides a vehicle inspection method, the method comprising:

[0005] Acquire captured road images, detect lane lines in the road images, and detect vehicles and their ground contact points in the road images;

[0006] Based on the lane lines, vehicles in the same lane are identified, and the grounding key points of the vehicles in the same lane are corrected to obtain the corrected grounding key points of each vehicle.

[0007] Based on the grounding key points of each vehicle after correction, the vehicle driving information of each vehicle is determined.

[0008] For example, prior to the step of detecting vehicles in the road image and the grounding key points of each vehicle, the process includes:

[0009] Acquire road images for training, and read vehicle body detection information and tire detection information for each vehicle from the road images used for training; wherein, the vehicle body detection information includes the corner coordinates and width and height data of the vehicle body detection box, and the tire detection information includes the coordinates of the tire contact point of each vehicle and whether the tire contact point of each vehicle is visible; when the tire is obscured by the vehicle body, the tire contact point of the obscured tire is marked as invisible, and when the tire is not obscured by the vehicle body, the tire contact point of the unobscured tire is marked as visible.

[0010] A vehicle detection model is trained based on the vehicle body detection information and tire detection information of each vehicle.

[0011] The step of detecting vehicles and their grounding key points in the road image includes:

[0012] Based on the vehicle detection model, vehicles in the road image and the grounding key points of each vehicle are detected.

[0013] For example, the step of detecting vehicles in the road image and the grounding key points of each vehicle further includes:

[0014] Based on the vehicle detection model, the confidence level of the grounding key points of each vehicle in the road image is detected;

[0015] The step of correcting the grounding key points of vehicles in the same lane to obtain the corrected grounding key points of each vehicle includes:

[0016] Based on the confidence level of the grounding key points of the vehicles in the same lane, the grounding key points of the vehicles in the same lane are corrected to obtain the corrected grounding key points of each vehicle.

[0017] For example, the step of correcting the grounding key points of vehicles in the same lane based on the confidence level of the grounding key points of the vehicles in the same lane, and obtaining the corrected grounding key points of each vehicle, includes:

[0018] Determine the confidence level of the contact key points of each vehicle on the same side of each wheel in the same lane, and determine the reference key point of each vehicle in the same lane in the current lane, wherein the reference key point is the reference key point with the highest confidence level among the contact key points of each vehicle on the same side of each wheel.

[0019] Based on the reference key points of each vehicle in the same lane in the current lane, the grounding key points of the vehicles in the same lane are corrected to obtain the corrected grounding key points of each vehicle.

[0020] For example, the step of determining the reference key points of each vehicle in the same lane in the current lane includes:

[0021] The average confidence level of the contact key points of each wheel on the same side of each vehicle in the same lane is determined, wherein the average confidence level is calculated based on the number of wheels on the same side and the total confidence level of the contact key points of each wheel on the same side.

[0022] The reference key point of each vehicle in the same lane is determined as the reference key point corresponding to the highest average confidence among the vehicles in the same lane.

[0023] For example, the step of correcting the ground contact key points of the vehicles in the same lane based on the reference key points of each vehicle in the current lane to obtain the corrected ground contact key points of each vehicle includes:

[0024] Based on the coordinates of the reference key points of each vehicle in the same lane in the current lane, the reference angle corresponding to the current lane is determined.

[0025] Determine the fixed point among the ground contact key points of each wheel on the same side of each vehicle in the same lane, and the body detection frame of each vehicle; wherein, the fixed point is the reference key point with the highest confidence among the ground contact key points of each wheel on the same side of any vehicle.

[0026] Based on the reference angle and the fixed point, the intersection point with the vehicle body detection frame is determined, and the intersection point is used as the grounding key point of the vehicle corresponding to the fixed point after correction.

[0027] For example, the step of determining the vehicle driving information of each vehicle based on the corrected grounding key points of each vehicle includes:

[0028] The vehicle's speed is determined based on the absolute position of the same grounding key point after correction in adjacent frames;

[0029] The vehicle's driving direction is determined based on the relative positions of multiple grounding key points after correction within the same frame.

[0030] This application also provides a vehicle detection device, the vehicle detection device comprising:

[0031] The initial module is used to acquire captured road images, detect lane lines in the road images, and detect vehicles in the road images and the ground contact points of each vehicle;

[0032] The correction module is used to determine the vehicles in the same lane based on the lane lines, and correct the grounding key points of the vehicles in the same lane to obtain the corrected grounding key points of each vehicle.

[0033] The determination module is used to determine the vehicle driving information of each vehicle based on the grounding key points of each vehicle after correction.

[0034] This application also provides a vehicle inspection device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the vehicle inspection method as described above.

[0035] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the vehicle detection method described above.

[0036] This application provides a vehicle detection method, vehicle detection device, vehicle detection equipment, and computer-readable storage medium. The method involves acquiring captured road images, detecting lane lines in the road images, and detecting vehicles and their grounding key points in the road images. Based on the lane lines, vehicles within the same lane are identified, and their grounding key points are corrected to obtain corrected grounding key points for each vehicle. Based on the corrected grounding key points, the driving information of each vehicle is determined.

[0037] Currently, in autonomous driving, vehicle target detection is rarely achieved through conventional 2D images captured by cameras. Instead, it is mostly achieved by using point cloud data from high-precision LiDAR sensors. However, this LiDAR solution is relatively expensive, yet it is still used in most scenarios. The reason is that the solution of detecting vehicle targets through 2D images is not accurate enough, and the key points detected are often unusable.

[0038] Therefore, in this application, firstly, lane lines are detected in the captured road image, and vehicles and their contact key points are detected in the road image. The contact key points of vehicles within the same lane are then corrected to obtain the corrected contact key points for each vehicle. Then, based on the corrected contact key points, the vehicle's driving information is determined. Specifically, considering the obvious characteristics of the contact key points of vehicle tires, the accuracy of the detected wheel key points is guaranteed; therefore, wheel contact points are used for vehicle detection. Furthermore, by correcting the contact key points of vehicles within the same lane, errors caused by using key points from vehicles in different lanes can be further reduced, improving the usability of the key points. Moreover, by correcting the contact key points of vehicles within the same lane to obtain the corrected contact key points for each vehicle, the accuracy of the determined vehicle driving information is further improved. In summary, by selecting wheel contact key points and correcting the key points of vehicles within the same lane, target detection of vehicles using two-dimensional images is achieved during autonomous driving, thereby obtaining accurate vehicle driving information. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the structure of the operating device of the hardware operating environment involved in the embodiments of this application;

[0040] Figure 2This is a schematic flowchart of an embodiment of the vehicle detection method involved in the present application.

[0041] Figure 3 This is a schematic diagram of key points of an embodiment of the vehicle detection method involved in the present application;

[0042] Figure 4 This is a schematic diagram illustrating the correction of an embodiment of the vehicle detection method involved in this application.

[0043] Figure 5 This is a schematic diagram of the vehicle detection device involved in the embodiments of this application.

[0044] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0045] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0046] Reference Figure 1 , Figure 1 This is a schematic diagram of the operating device structure of the hardware operating environment involved in the embodiments of this application.

[0047] like Figure 1 As shown, the operating device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0048] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the operating equipment and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0049] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and computer programs.

[0050] exist Figure 1 In the illustrated operating device, the network interface 1004 is mainly used for data communication with other devices; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and memory 1005 in the operating device of this application can be disposed in the operating device, and the operating device calls the computer program stored in the memory 1005 through the processor 1001 and performs the following operations:

[0051] Acquire captured road images, detect lane lines in the road images, and detect vehicles and their ground contact points in the road images;

[0052] Based on the lane lines, vehicles in the same lane are identified, and the grounding key points of the vehicles in the same lane are corrected to obtain the corrected grounding key points of each vehicle.

[0053] Based on the grounding key points of each vehicle after correction, the vehicle driving information of each vehicle is determined.

[0054] In one embodiment, the processor 1001 may invoke a computer program stored in the memory 1005 and further perform the following operations:

[0055] Before the step of detecting vehicles in the road image and the grounding key points of each vehicle, the following steps are included:

[0056] Acquire road images for training, and read vehicle body detection information and tire detection information for each vehicle from the road images used for training; wherein, the vehicle body detection information includes the corner coordinates and width and height data of the vehicle body detection box, and the tire detection information includes the coordinates of the tire contact point of each vehicle and whether the tire contact point of each vehicle is visible; when the tire is obscured by the vehicle body, the tire contact point of the obscured tire is marked as invisible, and when the tire is not obscured by the vehicle body, the tire contact point of the unobscured tire is marked as visible.

[0057] A vehicle detection model is trained based on the vehicle body detection information and tire detection information of each vehicle.

[0058] The step of detecting vehicles and their grounding key points in the road image includes:

[0059] Based on the vehicle detection model, vehicles in the road image and the grounding key points of each vehicle are detected.

[0060] In one embodiment, the processor 1001 may invoke a computer program stored in the memory 1005 and further perform the following operations:

[0061] The step of detecting vehicles and their grounding key points in the road image further includes:

[0062] Based on the vehicle detection model, the confidence level of the grounding key points of each vehicle in the road image is detected;

[0063] The step of correcting the grounding key points of vehicles in the same lane to obtain the corrected grounding key points of each vehicle includes:

[0064] Based on the confidence level of the grounding key points of the vehicles in the same lane, the grounding key points of the vehicles in the same lane are corrected to obtain the corrected grounding key points of each vehicle.

[0065] In one embodiment, the processor 1001 may invoke a computer program stored in the memory 1005 and further perform the following operations:

[0066] The step of correcting the grounding key points of vehicles in the same lane based on the confidence level of the grounding key points of the vehicles in the same lane, and obtaining the corrected grounding key points of each vehicle, includes:

[0067] Determine the confidence level of the contact key points of each vehicle on the same side of each wheel in the same lane, and determine the reference key point of each vehicle in the same lane in the current lane, wherein the reference key point is the reference key point with the highest confidence level among the contact key points of each vehicle on the same side of each wheel.

[0068] Based on the reference key points of each vehicle in the same lane in the current lane, the grounding key points of the vehicles in the same lane are corrected to obtain the corrected grounding key points of each vehicle.

[0069] In one embodiment, the processor 1001 may invoke a computer program stored in the memory 1005 and further perform the following operations:

[0070] The step of determining the reference key points of each vehicle in the same lane in the current lane includes:

[0071] The average confidence level of the contact key points of each wheel on the same side of each vehicle in the same lane is determined, wherein the average confidence level is calculated based on the number of wheels on the same side and the total confidence level of the contact key points of each wheel on the same side.

[0072] The reference key point of each vehicle in the same lane is determined as the reference key point corresponding to the highest average confidence among the vehicles in the same lane.

[0073] In one embodiment, the processor 1001 may invoke a computer program stored in the memory 1005 and further perform the following operations:

[0074] The step of correcting the grounding key points of vehicles in the same lane based on the reference key points of each vehicle in the current lane to obtain the corrected grounding key points of each vehicle includes:

[0075] Based on the coordinates of the reference key points of each vehicle in the same lane in the current lane, the reference angle corresponding to the current lane is determined.

[0076] Determine the fixed point among the ground contact key points of each wheel on the same side of each vehicle in the same lane, and the body detection frame of each vehicle; wherein, the fixed point is the reference key point with the highest confidence among the ground contact key points of each wheel on the same side of any vehicle.

[0077] Based on the reference angle and the fixed point, the intersection point with the vehicle body detection frame is determined, and the intersection point is used as the grounding key point of the vehicle corresponding to the fixed point after correction.

[0078] In one embodiment, the processor 1001 may invoke a computer program stored in the memory 1005 and further perform the following operations:

[0079] The step of determining the vehicle driving information of each vehicle based on the grounding key points of each vehicle after correction includes:

[0080] The vehicle's speed is determined based on the absolute position of the same grounding key point after correction in adjacent frames;

[0081] The vehicle's driving direction is determined based on the relative positions of multiple grounding key points after correction within the same frame.

[0082] This application provides a vehicle detection method, referring to... Figure 2 In one embodiment of the vehicle detection method, the method includes:

[0083] Step S10: Acquire the captured road image, detect lane lines in the road image, and detect vehicles in the road image and the grounding key points of each vehicle;

[0084] The captured road images refer to images containing roads where vehicles are present. These images can be captured by cameras mounted on vehicles for autonomous driving, or by cameras such as electronic eyes located on or above the road for vehicle monitoring.

[0085] The detected lane lines in the road image include one or more lane marking lines or segments with the same or different colors, sizes, and other attributes, such as solid lines, dashed lines, yellow lines, and white lines. These lane lines are used to delineate lanes. In one embodiment, lane lines in the road image are detected by incorporating the LLDNet lane line detection algorithm. The LLDNet algorithm framework consists of three parts: a feature extraction stage, a convolutional attention module, and a decoder. In the feature extraction stage, the encoder branch extracts necessary features and generates low- to high-level feature maps from an 80×160 pixel RGB image. Then, the features generated by the encoder are passed to the convolutional attention module to obtain high-level features. The main purpose of the attention mechanism is to focus more on the road portion of the image while ignoring other objects in the image (such as the sky, trees, pedestrians, etc.), thereby accelerating the model's performance and saving processing time. Finally, the decoder reconstructs the feature maps extracted from the encoder and attention module, generating a predicted image with accurate resolution to the input image.

[0086] Vehicles detected in road images include various wheeled vehicles, and the number of wheels is not limited. In one embodiment, vehicles in road images can be detected using a YOLOX model. Furthermore, based on the detection of vehicles in the road image, the ground contact key points of the vehicle can be detected; these ground contact key points refer to the points where the vehicle wheels contact the ground. That is, in addition to detecting vehicles in road images using the YOLOX model, such as vehicle position and size, the detection of key points on each tire of the vehicle is added. This means that more vehicle information, such as vehicle speed and direction, is obtained by adding vehicle key point detection, thus satisfying higher-level autonomous driving based on key points, beyond basic autonomous driving based on vehicle body detection.

[0087] In one possible implementation, a vehicle key point detection model based on the YOLOX model is set up. This model modifies obj_output and reg_output of the three outputs of the YOLOX decoupling head to obtain the prediction results of the grounding key points.

[0088] Step S20: Based on the lane lines, determine the vehicles in the same lane and correct the grounding key points of the vehicles in the same lane to obtain the corrected grounding key points of each vehicle.

[0089] Currently, key point detection often results in inaccurate readings. For example, normally, connecting the wheel contact key points on both sides of a wheel to the same side should create two parallel line segments in the same direction as the vehicle. However, due to inaccurate detection, these two line segments may occasionally not be parallel. Therefore, this application proposes a key point correction scheme that integrates lane line detection. By identifying vehicles within the same lane using lane lines and correcting the contact key points of these vehicles, the corrected contact key points for each vehicle are obtained, thereby achieving accurate correction of the contact key points.

[0090] Step S30: Determine the vehicle driving information of each vehicle based on the grounding key points of each vehicle after correction.

[0091] After obtaining the corrected grounding key points for each vehicle, the vehicle's driving information can be determined based on these points. For example, by applying key point detection to vehicle inspection, the target vehicle's position and size can be obtained, and its direction information can be determined through the grounding key points. This allows for near-3D 2.5D vehicle inspection, effectively saving inspection costs.

[0092] For example, prior to the step of detecting vehicles in the road image and the grounding key points of each vehicle, the process includes:

[0093] Acquire road images for training, and read vehicle body detection information and tire detection information for each vehicle from the road images used for training; wherein, the vehicle body detection information includes the corner coordinates and width and height data of the vehicle body detection box, and the tire detection information includes the coordinates of the tire contact point of each vehicle and whether the tire contact point of each vehicle is visible; when the tire is obscured by the vehicle body, the tire contact point of the obscured tire is marked as invisible, and when the tire is not obscured by the vehicle body, the tire contact point of the unobscured tire is marked as visible.

[0094] A vehicle detection model is trained based on the vehicle body detection information and tire detection information of each vehicle.

[0095] The step of detecting vehicles and their grounding key points in the road image includes:

[0096] Based on the vehicle detection model, vehicles in the road image and the grounding key points of each vehicle are detected.

[0097] Before detecting vehicles and their ground contact points in road images, a vehicle detection model needs to be trained. This is done by acquiring road images for training and reading pre-annotated vehicle body and tire detection information from these images. The vehicle detection model is then trained based on this information and used to detect vehicles and their ground contact points in the road images. Essentially, the vehicle body detection information is used to detect the vehicle body, and the tire detection information is used to detect the vehicle's wheel ground contact points.

[0098] In one implementation, during the training of the original YOLOX model, the top-left and bottom-right corner points, as well as the width and height of the ground truth bounding boxes, are read from the label JSON file. In this embodiment, the YOLOX keypoint model additionally reads 12 values: the horizontal and vertical coordinates of four keypoints—the left front tire contact point, the right front tire contact point, and the right rear tire contact point—and whether these four keypoints are visible. Keypoints of wheels obscured by the vehicle body are set to invisible and do not participate in model training. If visible, the variable v = 1; otherwise, v = 0. These 16 values ​​are then passed to the variable bbox for further processing.

[0099] In the YOLOX prediction part, the model outputs three branches: cls_output for outputting the target category, obj_output (processed with sigmoid) for determining whether the target is foreground or background, and bbox_output for predicting target coordinates. When adding keypoint detection to the YOLOX model to obtain the YOLOX keypoint model in this embodiment, the category remains unchanged, so cls_output is not processed. Based on obj_output's determination of whether the target is foreground or background, variables v indicating the visibility of the four corresponding keypoints are passed in. This allows obj_output to determine the visibility of the four keypoints in addition to its function of determining whether the target is foreground or background. Based on bbox_output's prediction of target coordinates, the coordinates of the four keypoints are also included in the prediction, thus predicting the coordinates of the four keypoints along with the target bounding box coordinates.

[0100] In addition, keypoint detection was added to YOLOX, and the YOLOX model with keypoint detection was then migrated to the MMDetection framework. By migrating the model to the MMDetection framework, the training speed of the model was improved, and modifications to the model were made more convenient.

[0101] For example, the step of detecting vehicles in the road image and the grounding key points of each vehicle further includes:

[0102] Based on the vehicle detection model, the confidence level of the grounding key points of each vehicle in the road image is detected;

[0103] The step of correcting the grounding key points of vehicles in the same lane to obtain the corrected grounding key points of each vehicle includes:

[0104] Based on the confidence level of the grounding key points of the vehicles in the same lane, the grounding key points of the vehicles in the same lane are corrected to obtain the corrected grounding key points of each vehicle.

[0105] While detecting vehicles and their grounding key points in road images using a vehicle detection model, the model also determines the confidence level of each vehicle's grounding key points. This confidence level refers to the degree of certainty that the key points detected by the model are grounding key points. Next, based on the confidence level of the grounding key points of vehicles in the same lane, the grounding key points of vehicles in the same lane are corrected, thus obtaining more accurate corrected grounding key points for each vehicle.

[0106] For example, the step of correcting the grounding key points of vehicles in the same lane based on the confidence level of the grounding key points of the vehicles in the same lane, and obtaining the corrected grounding key points of each vehicle, includes:

[0107] Determine the confidence level of the contact key points of each vehicle on the same side of each wheel in the same lane, and determine the reference key point of each vehicle in the same lane in the current lane, wherein the reference key point is the reference key point with the highest confidence level among the contact key points of each vehicle on the same side of each wheel.

[0108] Based on the reference key points of each vehicle in the same lane in the current lane, the grounding key points of the vehicles in the same lane are corrected to obtain the corrected grounding key points of each vehicle.

[0109] When correcting the grounding key points of vehicles in the same lane based on their confidence levels, the confidence levels of the grounding key points of each vehicle on the same side of the same lane are determined. Each wheel on the same side is a visible wheel, not obscured by the vehicle body and detectable. Multiple wheels may exist on the same side of a vehicle. Then, the reference key point for each vehicle in the same lane is determined as the reference key point with the highest confidence among the grounding key points of each vehicle's wheels on the same side. The selection criteria for the reference key point can be the highest average confidence level, the highest confidence level of a single grounding key point, or a set of grounding key points of the vehicle with the smallest confidence variance when the difference in the highest average confidence level is within a preset difference threshold range.

[0110] In one embodiment, reference is made to Figure 3 Taking a four-wheeled vehicle as an example, among the confidence levels of the ground contact key points of each wheel on the side of the lane closest to the dashed line, the confidence level of the ground contact key points of each wheel of vehicle A on the side closest to the dashed line is the highest. Therefore, the two ground contact key points of the two wheels of vehicle A on the side closest to the dashed line are selected as the reference key points of the lane where vehicle A is located.

[0111] Then, based on the reference key point of the current lane, the grounding key points of vehicles in the same lane are corrected to obtain the corrected grounding key points of each vehicle.

[0112] In one embodiment, reference is made to Figure 3 Furthermore, based on the benchmark key points, namely the two grounding key points of the two wheels of vehicle A on the side closest to the dashed line, the key points of the other two vehicles in the lane where vehicle A is located, vehicle B and vehicle C, are corrected to obtain the corrected grounding key points of vehicle B and vehicle C.

[0113] For example, the step of determining the reference key points of each vehicle in the same lane in the current lane includes:

[0114] The average confidence level of the contact key points of each wheel on the same side of each vehicle in the same lane is determined, wherein the average confidence level is calculated based on the number of wheels on the same side and the total confidence level of the contact key points of each wheel on the same side.

[0115] The reference key point of each vehicle in the same lane is determined as the reference key point corresponding to the highest average confidence among the vehicles in the same lane.

[0116] When determining the reference key points of each vehicle in the same lane in the current lane, the average confidence of the ground contact key points of each vehicle in the same lane on the same side is calculated based on the number of wheels on each side and the total confidence of the ground contact key points of each wheel on each side. Then, the reference key point of each vehicle in the same lane in the current lane is determined as the reference key point corresponding to the highest average confidence among the vehicles in the same lane.

[0117] In one embodiment, reference is made to Figure 3 The total confidence level of the two wheels of vehicle A on the side closest to the dashed line is 'a', and its average confidence level is 'a / 2'. Similarly, the average confidence levels of the two wheels of vehicle B and vehicle C on the side closest to the dashed line are 'b / 2' and 'c / 2', respectively. If a > b > c, then the reference key points of the lanes where vehicle A, vehicle B, and vehicle C are located are determined as the two wheels of vehicle A on the side closest to the dashed line.

[0118] For example, the step of correcting the ground contact key points of the vehicles in the same lane based on the reference key points of each vehicle in the current lane to obtain the corrected ground contact key points of each vehicle includes:

[0119] Based on the coordinates of the reference key points of each vehicle in the same lane in the current lane, the reference angle corresponding to the current lane is determined.

[0120] Determine the fixed point among the ground contact key points of each wheel on the same side of each vehicle in the same lane, and the body detection frame of each vehicle; wherein, the fixed point is the reference key point with the highest confidence among the ground contact key points of each wheel on the same side of any vehicle.

[0121] Based on the reference angle and the fixed point, the intersection point with the vehicle body detection frame is determined, and the intersection point is used as the grounding key point of the vehicle corresponding to the fixed point after correction.

[0122] When correcting the ground contact points of vehicles in the same lane based on the reference key points of the current lane, firstly, the reference angle corresponding to the current lane is determined according to the coordinates of the reference key points; then, the fixed points of the ground contact points of each wheel on the same side of each vehicle and the vehicle body detection frame are determined; finally, the intersection point with the vehicle body detection frame is determined based on the reference angle and the fixed point, and the intersection point is used as the ground contact point of the vehicle corresponding to the fixed point after correction.

[0123] In one embodiment, reference is made to Figure 4After determining the reference key points of the lanes where vehicles A, B, and C are located as the contact key points of the two wheels of vehicle A on the side closest to the dashed line, firstly, based on the coordinates of the contact key points of the two wheels of vehicle A on the side closest to the dashed line, the reference angle α corresponding to the current lane is determined; then, the fixed points of the contact key points of each wheel on the same side of each vehicle and the vehicle body detection frame of each vehicle are determined. For example, if the confidence levels of the contact key points of the two wheels of vehicle B on the side closest to the dashed line are b1 and b2 respectively, and b1 > b2, then the fixed point of vehicle B is the contact key point corresponding to confidence level b1; finally, based on the reference angle α and the fixed point b1, the intersection point V with the vehicle body detection frame of vehicle B is determined, and the intersection point V is taken as the corrected contact key point of vehicle B corresponding to the fixed point b1. The correction of vehicle C is the same as that of vehicle B, and will not be described in detail here.

[0124] For example, the step of determining the vehicle driving information of each vehicle based on the corrected grounding key points of each vehicle includes:

[0125] The vehicle's speed is determined based on the absolute position of the same grounding key point after correction in adjacent frames;

[0126] The vehicle's driving direction is determined based on the relative positions of multiple grounding key points after correction within the same frame.

[0127] After correcting the grounding key points of vehicles in the same lane and obtaining the corrected grounding key points of each vehicle, the vehicle driving information of each vehicle can be determined based on the corrected grounding key points of each vehicle. The vehicle driving information includes the vehicle's driving speed and driving direction.

[0128] Furthermore, the vehicle's speed is determined based on the absolute positions of the same grounding key points after correction in adjacent frames. In other words, the vehicle's speed is calculated based on the absolute positions of the same grounding key points after correction in adjacent frames. Additionally, the vehicle's direction of travel is determined based on the relative positions of multiple grounding key points after correction in the same frame. That is, the direction of travel can be determined by the magnitude of the relative positions of multiple grounding key points after correction in the same frame, such as the magnitude of the horizontal and vertical coordinates.

[0129] In one application scenario of the vehicle detection method of this application, vehicles in each lane are divided according to the coordinates of key points and the lane detection results, and then correction is performed within the same lane. First, the coordinates of the two tires on the side of each target vehicle, i.e., the two key points, are obtained. Then, the confidence scores of the two key points of all targets under the lane are added together and averaged to obtain the two key points of the vehicle with the highest average score. The two key points of the vehicle with the highest average score are considered to have the best detection effect. Then, using these two key points as the reference, the two key points are connected and the angle between the line segment connecting the two key points and the y-axis of the current frame image is calculated (the slope can also be calculated from the angle, and either the angle or the slope is sufficient). Then, for the two key points of the detection results of other vehicles on this lane, the key point with the higher confidence score is taken as the fixed point of the other vehicle (i.e., the point that does not need to be corrected). According to the angle or slope calculated above, a ray is obtained from the fixed point. Then, this ray intersects with the vehicle bounding box (body detection box) obtained by the YOLOX model that has added key point detection. This intersection point is the corrected key point.

[0130] Reference Figure 5 Furthermore, embodiments of this application also provide a vehicle detection device, the vehicle detection device comprising:

[0131] The initial module M1 is used to acquire captured road images, detect lane lines in the road images, and detect vehicles in the road images and the grounding key points of each vehicle;

[0132] The correction module M2 is used to determine the vehicles in the same lane based on the lane lines, and correct the grounding key points of the vehicles in the same lane to obtain the corrected grounding key points of each vehicle.

[0133] The determination module M3 is used to determine the vehicle driving information of each vehicle based on the grounding key points of each vehicle after correction.

[0134] For example, the initial module is further configured to:

[0135] Before the step of detecting vehicles in the road image and the grounding key points of each vehicle,

[0136] Acquire road images for training, and read vehicle body detection information and tire detection information for each vehicle from the road images used for training; wherein, the vehicle body detection information includes the corner coordinates and width and height data of the vehicle body detection box, and the tire detection information includes the coordinates of the tire contact point of each vehicle and whether the tire contact point of each vehicle is visible; when the tire is obscured by the vehicle body, the tire contact point of the obscured tire is marked as invisible, and when the tire is not obscured by the vehicle body, the tire contact point of the unobscured tire is marked as visible.

[0137] A vehicle detection model is trained based on the vehicle body detection information and tire detection information of each vehicle.

[0138] The initial module is also used for:

[0139] Based on the vehicle detection model, vehicles in the road image and the grounding key points of each vehicle are detected.

[0140] For example, the initial module is further configured to:

[0141] Based on the vehicle detection model, the confidence level of the grounding key points of each vehicle in the road image is detected;

[0142] The step of correcting the grounding key points of vehicles in the same lane to obtain the corrected grounding key points of each vehicle includes:

[0143] Based on the confidence level of the grounding key points of the vehicles in the same lane, the grounding key points of the vehicles in the same lane are corrected to obtain the corrected grounding key points of each vehicle.

[0144] For example, the initial module is further configured to:

[0145] Determine the confidence level of the contact key points of each vehicle on the same side of each wheel in the same lane, and determine the reference key point of each vehicle in the same lane in the current lane, wherein the reference key point is the reference key point with the highest confidence level among the contact key points of each vehicle on the same side of each wheel.

[0146] Based on the reference key points of each vehicle in the same lane in the current lane, the grounding key points of the vehicles in the same lane are corrected to obtain the corrected grounding key points of each vehicle.

[0147] For example, the initial module is further configured to:

[0148] The average confidence level of the contact key points of each wheel on the same side of each vehicle in the same lane is determined, wherein the average confidence level is calculated based on the number of wheels on the same side and the total confidence level of the contact key points of each wheel on the same side.

[0149] The reference key point of each vehicle in the same lane is determined as the reference key point corresponding to the highest average confidence among the vehicles in the same lane.

[0150] For example, the initial module is further configured to:

[0151] Based on the coordinates of the reference key points of each vehicle in the same lane in the current lane, the reference angle corresponding to the current lane is determined.

[0152] Determine the fixed point among the ground contact key points of each wheel on the same side of each vehicle in the same lane, and the body detection frame of each vehicle; wherein, the fixed point is the reference key point with the highest confidence among the ground contact key points of each wheel on the same side of any vehicle.

[0153] Based on the reference angle and the fixed point, the intersection point with the vehicle body detection frame is determined, and the intersection point is used as the grounding key point of the vehicle corresponding to the fixed point after correction.

[0154] For example, the determining module is further configured to:

[0155] The vehicle's speed is determined based on the absolute position of the same grounding key point after correction in adjacent frames;

[0156] The vehicle's driving direction is determined based on the relative positions of multiple grounding key points after correction within the same frame.

[0157] The vehicle detection device provided in this application employs the vehicle detection method described in the above embodiments, aiming to achieve target detection of vehicles using two-dimensional images during autonomous driving. Compared with conventional technologies, the beneficial effects of the vehicle detection device provided in this application are the same as those of the vehicle detection method described in the above embodiments, and other technical features in the vehicle detection device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0158] Furthermore, this application embodiment also provides a vehicle detection device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the vehicle detection method described above.

[0159] Furthermore, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the vehicle detection method described above.

[0160] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0161] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to conventional technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0162] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A vehicle inspection method, characterized in that, The method includes: Acquire captured road images, detect lane lines in the road images, and detect vehicles and their ground contact points in the road images; Based on the lane lines, vehicles in the same lane are identified, and the confidence levels of the contact key points of each wheel on the same side for each vehicle in the same lane are determined. The average confidence level of the contact key points of each wheel on the same side for each vehicle in the same lane is also determined, wherein the average confidence level is calculated based on the number of wheels on the same side and the total confidence level of the contact key points of each wheel on the same side. A reference key point for each vehicle in the same lane in the current lane is determined as the contact key point corresponding to the highest average confidence level among all vehicles in the same lane. Based on the reference key points of each vehicle in the same lane in the current lane, the contact key points of the vehicles in the same lane are corrected to obtain the corrected contact key points for each vehicle. Based on the grounding key points of each vehicle after correction, the vehicle driving information of each vehicle is determined.

2. The vehicle detection method as described in claim 1, characterized in that, Before the step of detecting vehicles in the road image and the grounding key points of each vehicle, the following steps are included: Acquire road images for training, and read vehicle body detection information and tire detection information for each vehicle from the road images used for training; wherein, the vehicle body detection information includes the corner coordinates and width and height data of the vehicle body detection box, and the tire detection information includes the coordinates of the tire contact point of each vehicle and whether the tire contact point of each vehicle is visible; when the tire is obscured by the vehicle body, the tire contact point of the obscured tire is marked as invisible, and when the tire is not obscured by the vehicle body, the tire contact point of the unobscured tire is marked as visible. A vehicle detection model is trained based on the vehicle body detection information and tire detection information of each vehicle. The step of detecting vehicles and their grounding key points in the road image includes: Based on the vehicle detection model, vehicles in the road image and the grounding key points of each vehicle are detected.

3. The vehicle detection method as described in claim 2, characterized in that, The step of detecting vehicles and their grounding key points in the road image further includes: Based on the vehicle detection model, the confidence level of the grounding key points of each vehicle in the road image is detected.

4. The vehicle detection method as described in claim 1, characterized in that, The step of correcting the grounding key points of vehicles in the same lane based on the reference key points of each vehicle in the current lane to obtain the corrected grounding key points of each vehicle includes: Based on the coordinates of the reference key points of each vehicle in the same lane in the current lane, the reference angle corresponding to the current lane is determined. Determine the fixed point among the ground contact key points of each vehicle on the same side of each wheel in the same lane, and the body detection frame of each vehicle; wherein, the fixed point is the ground contact key point with the highest confidence among the ground contact key points of each vehicle on the same side of each wheel. Based on the reference angle and the fixed point, the intersection point with the vehicle body detection frame is determined, and the intersection point is used as the grounding key point of the vehicle corresponding to the fixed point after correction.

5. The vehicle detection method as described in claim 1, characterized in that, The step of determining the vehicle driving information of each vehicle based on the grounding key points of each vehicle after correction includes: The vehicle's speed is determined based on the absolute position of the same grounding key point after correction in adjacent frames; The vehicle's driving direction is determined based on the relative positions of multiple grounding key points after correction within the same frame.

6. A vehicle detection device, characterized in that, The vehicle detection device includes: The initial module is used to acquire captured road images, detect lane lines in the road images, and detect vehicles in the road images and the ground contact points of each vehicle; The correction module is used to determine vehicles in the same lane based on the lane lines, and to determine the confidence level of the contact key points of each wheel on the same side of each vehicle in the same lane, and to determine the average confidence level of the contact key points of each wheel on the same side of each vehicle in the same lane, wherein the average confidence level is calculated based on the number of wheels on the same side and the total confidence level of the contact key points of each wheel on the same side; to determine the reference key point of each vehicle in the same lane in the current lane as the contact key point corresponding to the highest average confidence level among the vehicles in the same lane; and to correct the contact key points of the vehicles in the same lane based on the reference key points of each vehicle in the current lane, thereby obtaining the corrected contact key points of each vehicle. The determination module is used to determine the vehicle driving information of each vehicle based on the grounding key points of each vehicle after correction.

7. A vehicle inspection device, characterized in that, The vehicle detection device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the vehicle detection method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the vehicle detection method as described in any one of claims 1 to 5.

Images