Vehicle pose estimation method, chip, electronic device and computer readable medium

Abstract

The application discloses a vehicle pose estimation method, a chip, an electronic device and a computer readable medium, comprising the following steps: S1, vehicle detection is performed on each frame of image based on a target detection model; S2, vehicle tracking is performed based on an improved DeepSORT algorithm; S3, three-dimensional scene model prior information is combined with the bounding box of the vehicle in the image coordinate system obtained by the vehicle tracking part, and a transformation matrix M representing the vehicle pose is output. 3×4 and the pre-defined vehicle model size closest to the target size. The application realizes pose estimation of the vehicle in the traffic monitoring video stream, can approximately estimate the position and orientation of the vehicle in the 3D model space according to the vehicle detection and tracking results in the 2D image, and provides key technical support for realizing the function of simultaneously viewing the vehicle flow of multiple cameras from any perspective in the digital twin city platform.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision and computer graphics, and particularly relates to a method for vehicle pose estimation in a digital twin city platform. BACKGROUND

[0002] The digital twin city platform aims to achieve more comprehensive and intelligent city monitoring and management by combining three-dimensional models of static objects such as buildings and roads with dynamic sensing information from various sensors in the Internet of Things system. At present, most private vehicles do not have uniform Internet of Things access conditions, and the presentation of vehicle dynamic information in the digital twin city platform is mainly direct pulling of real-time traffic monitoring video streams. The observation angle of each road is limited to the angle of the camera covering the road, and when monitoring multiple road vehicle traffic conditions, it cannot provide an intuitive and unified observation angle.

[0003] In patent application CN106651953A, a vehicle pose estimation method based on traffic signs is disclosed. The vehicle's approximate position is obtained through a GPS device, then the traffic signs are detected through a camera on the vehicle, and then the vehicle's position and attitude in the traffic sign coordinate system are located inversely through the recognized traffic signs combined with the approximate position.

[0004] In patent application CN108759833A, an intelligent vehicle positioning method based on priori map is disclosed. The intelligent vehicle's pose information and observation information are obtained through the intelligent vehicle's odometer and vehicle-mounted radar respectively. The intelligent vehicle's pose information and observation information are processed using SLAM algorithm to obtain the global map of the entire operating area, the local map of the local operating area, and the intelligent vehicle's pose information in the SLAM coordinate system. The intelligent vehicle's pose information in the GPS geographic coordinate system is obtained and converted to the SLAM coordinate system to obtain the intelligent vehicle's GPS pose information in the SLAM coordinate system. The intelligent vehicle's pose information in the SLAM coordinate system and the intelligent vehicle's GPS pose information in the SLAM coordinate system are fused using Kalman filtering technology to obtain the intelligent vehicle's preliminary pose estimation. The global map and the local map are matched, and the matching result is used to optimize the preliminary pose estimation to obtain the intelligent vehicle's final pose. This invention can improve the precision and accuracy of intelligent vehicle positioning results

[0005] In the patent application CN111400423A, a smart city CIM three-dimensional vehicle pose modeling system based on multi-view geometry is disclosed. The system includes: a vehicle key point acquisition unit acquires a vehicle key point heat map; a vehicle pose acquisition unit calculates the vehicle pose based on the vehicle key point heat map; a vehicle type information acquisition unit is used to identify the vehicle type; and a visualization unit visualizes the obtained vehicle pose and vehicle type based on the city information model. The invention can improve the vehicle pose detection accuracy in smart city traffic management.

[0006] In the paper "Monocular vehicle pose estimation based on three-dimensional model" published by Xu Lingzhi et al. of Shanghai Jiaotong University in Optics and Precision Engineering, 2021, No. 6, a method of combining monocular camera and vehicle three-dimensional model for vehicle pose estimation is proposed. First, the vehicle target is normalized in multiple scales, and then the coordinates of the vehicle key points are regressed in the form of vector field to improve the pose estimation accuracy under occlusion or truncation. In this process, a distance weighted vector field loss function and a key point error minimization voting method are used to improve the accuracy of the pose estimation algorithm. This method is mainly suitable for the field of unmanned driving.

[0007] Since the digital twin city platform has completed the modeling of the entire static scene in advance, the skilled person in the art is committed to developing a method for estimating the vehicle pose from the video stream, which can directly display the corresponding vehicle model in the scene model according to the estimated pose, thereby achieving the function of viewing multiple camera vehicle flows at the same time from any perspective. SUMMARY

[0008] The present invention is committed to fully utilizing the three-dimensional prior information provided by the digital twin city platform in a simple and effective way, combining related technologies in the fields of computer vision and computer graphics to estimate the vehicle pose in the traffic monitoring video stream in real time, and providing key technical support for realizing the function of viewing multiple camera vehicle flows at the same time from any perspective in the digital twin city platform. The characteristics of the present invention are: 1) not limited by the camera installation scheme; 2) only using the existing three-dimensional model data and video data in the digital twin city platform, without introducing additional sensor data; 3) meeting the real-time estimation requirement.

[0009] To achieve the above purpose, the present invention provides a vehicle pose estimation method for a digital twin city platform, comprising the following steps:

[0010] S1: performing vehicle detection on each frame of image based on a target detection model, and predicting the 2D bounding box of the vehicle target in the image coordinate system;

[0011] S2: Vehicle tracking is performed based on the improved DeepSORT algorithm, which performs feature matching between the vehicle detection results of the current frame and the detection results of past frames from the same camera.

[0012] S3: Combine the prior information of the 3D scene model with the 2D bounding box of the vehicle information obtained from vehicle tracking in the image coordinate system to output a transformation matrix M∈R representing the vehicle pose. 3×4 And the predefined vehicle model size that is closest to the target size; where: M represents the transformation relationship between the 3D scene model coordinate system and the vehicle model coordinate system, R 3×4 Refers to the real number space.

[0013] Preferably, the improved DeepSORT algorithm described in step S2 includes the following improvements:

[0014] S2-1: Remove the independent ReID feature extraction network part in DeepSORT and replace it with the existing deep features in the vehicle detection model;

[0015] S2-2: Use location information to constrain the range of feature matching.

[0016] Preferably, S2-1 includes the following steps:

[0017] S2-1-1: When inputting a 3×H×W image into the YOLOv5m6 model prediction process, save the four shapes generated by the neck part of the YOLOv5m6 model as follows: Feature image F i , i∈{0,1,2,3};

[0018] S2-1-2: For each predicted center point coordinate (cx, cy) of the 2D bounding box in the input image coordinate system, calculate the corresponding coordinates (cx, cy) of the center point in each feature image coordinate system according to the ratio of the size of the input image and the feature image. i cy i And at that coordinate position, extract the feature vector V along the first dimension of the feature image. i ;

[0019] S2-1-3: The four feature vectors V... i The concatenation of these features sequentially forms the feature corresponding to the feature matching.

[0020] Preferably, the feature matching is limited to 2D bounding boxes that overlap and whose aspect ratios differ by no more than twice.

[0021] Preferably, the combination in step S3 includes the following steps:

[0022] S3-1: Estimating the 2D pose of the vehicle:

[0023] The 2D pose of the vehicle is represented by position coordinates and motion direction vector. Only when the motion distance of the center point of the 2D bounding box exceeds the threshold T, the center point (cx, cy) of the 2D bounding box at the current moment is recorded as a new position coordinate. The direction from the previous position coordinate to the current position coordinate is normalized and recorded as the motion direction vector (vx, vy) at the current moment. (vx, vy) is the target's own coordinate with (cx, cy) as the origin and the x-axis and y-axis directions being the same as the image system coordinates.

[0024] S3-2: Export prior information of the 3D scene model, including: exporting predefined vehicle model dimensions, camera calibration parameters, and ground information in the static scene model;

[0025] Specifically, exporting the predefined vehicle model dimensions refers to selecting several vehicle models with significant size differences in Unreal Engine as models used in the predefined visualization process, and exporting the length, width, and height of the corresponding 3D bounding boxes, denoted as S = {S0, S1, ..., S...}. i}, i∈[0,N],S i ={l i w i h i}, where N is the number of predefined models;

[0026] Exporting the camera calibration parameters means exporting the camera calibration parameters into a general intrinsic and extrinsic parameter matrix format;

[0027] Exporting the ground information means exporting the ground information as a tensor G of shape H×W×4, where H and W are the height and width of the camera output image, and 4 is the corresponding number of channels. The specific description of G is as follows: First, place a set of segmented planes that fit the ground model in Unreal Engine, and assign a number to each of the placed segmented planes. Then, traverse each coordinate point (i, j) in the camera imaging plane in Unreal Engine, i∈[0, W-1], j∈[0, H-1]. Calculate the coordinates of the intersection point of the ray pointing from the camera center to the coordinate point and the segmented plane, as well as the number of the intersecting segmented plane. Finally, combine the coordinates and the number into a vector with 4 elements and store it in the corresponding position of G along the channel direction.

[0028] S3-3: Pre-calculated ground plane equation:

[0029] After obtaining the tensor G, the segmented plane numbers are traversed in sequence, all coordinate points belonging to the same number are taken out, and the coefficients A, B, C and D of the plane equation Ax+By+Cz+D=0 corresponding to the plane are solved by using the least square method. In the subsequent calculation process, the plane number corresponding to the target is looked up according to the table;

[0030] S3-4: Estimate the 3D pose of the vehicle:

[0031] For a given 2D pose, each predefined vehicle model size is traversed in sequence, and the corresponding 3D pose M under the size is calculated. Then the 3D bounding box under the size and the pose is back projected to the 2D image coordinate system and compared with the actual 2D bounding box. The 3D pose that best fits the 2D bounding box after back projection is selected as the final estimated value.

[0032] Preferably, the threshold T is 1 / 4 of the diagonal length of the bounding box.

[0033] Preferably, the specific process of step S3-4 is as follows:

[0034] S3-4-1: For a given 2D pose (cx, cy) and (vx, vy), the segmented plane number corresponding to the ray from the camera center to (cx, cy) is obtained according to the tensor G, and the plane P pre-calculated in S3-3 is obtained according to the number.

[0035] S3-4-2: Traverse each predefined vehicle model size S i , calculate the plane P' i , i∈[0,N]; i S3-4-3: Traverse each plane P' i , calculate the intersection point (cx i , cy i , cz i ) of the ray from the camera center to (cx, cy) and the plane P' i , calculate the intersection point (vx i , vy i , vz i ) of the ray from the camera center to (cx+vx, cy+vy) and the plane P' i , calculate the translation of the origin of the three-dimensional scene model coordinate system to (cx i , cy i , cz i ), and rotate the coordinate system to the same Z axis as the normal vector of P' i and the X axis from (cx i , cy i , cz i ).i ) corresponding to the pose (vx i , vy i , vz i ) is M i ;

[0037] S3-4-4: Calculate the 8 2D coordinate points corresponding to the 8 vertex projections of the 3D bounding box of size S i at pose M i , and find the minimum horizontal bounding box B i ;

[0038] S3-4-5: Calculate the IoU between all B i and the 2D bounding box output by the vehicle tracking part, select S i and M i as the size and 3D pose of the current vehicle.

[0039] The application also provides a chip, characterized by comprising a processor for calling and running a computer program from a memory, so that a device installed with the chip executes any of the methods.

[0040] The application also provides an electronic device comprising a processor and a memory for storing executable instructions of the processor, the processor executes any of the methods when running.

[0041] The application also provides a computer readable medium having computer program instructions stored thereon, the computer program instructions are executed by a processor to implement any of the methods.

[0042] The application realizes a method for estimating the pose of a vehicle in a traffic monitoring video stream, which approximately estimates the position and orientation of the vehicle in the 3D model space according to the results of vehicle detection and tracking in the 2D image, and can provide key technical support for the function of simultaneously viewing multiple camera traffic conditions at any viewing angle in the digital twin city platform.

[0043] The concept, specific steps and technical effects of the application will be further described below to fully understand the purpose, features and effects of the application. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 is a pose estimation effect example diagram, for the convenience of comparing with the original content, the estimated 3D bounding box is projected to 2D for visualization. DETAILED DESCRIPTION

[0045] The technical content of the present application is described more clearly and conveniently with reference to the accompanying drawings of the specification, and the scope of protection of the present application is not limited to the embodiments mentioned in the text.

[0046] To achieve the purpose of the present application, the method comprises three stages:

[0047] Stage 1: Vehicle detection based on YOLOv5

[0048] The purpose of the vehicle detection stage is to independently process each frame of image, predict the 2D bounding box of the vehicle target in the image coordinate system in the image, which is usually represented by four elements (x1, y1, x2, y2), where (x1, y1) is the coordinate of the upper left corner of the rectangular box, and (x2, y2) is the coordinate of the lower right corner of the rectangular box. The implementation of this stage is to directly retrain the target detection model widely used in the field of computer vision at present (YOLOv5 is used in this embodiment) in a data set containing various vehicle types, various perspectives and various scenes. The data set used in this patent is a road monitoring scene data set labeled internally by the company, and the model used is YOLOv5m6.

[0049] Stage 2: Vehicle tracking based on improved DeepSORT

[0050] The purpose of the vehicle tracking stage is to perform feature matching between the vehicle detection results of the current frame and the detection results of the past frames of the same camera, and divide the corresponding results of the same vehicle at different times into the same trajectory set.

[0051] Matching means judging which detection results correspond to the same vehicle. After matching, the natural result is to divide the detection results of the same vehicle at different times into the same set. Since each set records the position of a vehicle at different times, it is called a trajectory set. The "last position coordinate" and "current position coordinate" mentioned in the subsequent estimation of the 2D pose of the vehicle are all data obtained from the trajectory set.

[0052] The implementation of this stage is to improve the algorithm DeepSORT widely used in the task of single-camera multi-target tracking for the specific task of vehicle tracking, so as to improve the operation speed of the vehicle tracking stage and reduce the hardware resource occupation in the vehicle tracking process.

[0053] The improvement part is as follows:

[0054] 1) Remove the independent feature extraction network

[0055] In the original DeepSORT feature matching process, first use an independent ReID (Re-Identification) network to extract features from each image block corresponding to the bounding box, and then use the features for matching. This method has certain defects in execution efficiency. On the one hand, using an additional network requires additional computing resources. On the other hand, the number of times the ReID network is executed depends on the number of targets in the image. The time-consuming of feature extraction is not constant, so it is not suitable for vehicle-dense traffic monitoring scenarios in digital twin city systems. To solve the above problems, the independent ReID feature extraction network in DeepSORT is removed in this patent, and the existing deep features in the vehicle detection model are used for improvement, so that the feature extraction process can act on the entire image range and the running time is fixed. The specific process is as follows:

[0056] i. In the process of inputting an image with a shape of 3xHxW into the YOLOv5m6 model for prediction, save the four feature images F generated by the model neck part, which have shapes of i , respectively, i∈{0,1,2,3}.

[0057] ii. For the center point coordinates (cx, cy) of each predicted bounding box in the input image coordinate system, calculate the corresponding coordinates (cx i , cy i ) of the center point in each feature image coordinate system according to the size ratio of the input image and the feature image, and take out the feature vector V i along the first dimension of the feature image at the coordinate position.

[0058] iii. After concatenating the four feature vectors V i in turn, use them as features in the feature matching process.

[0059] 2) Restrict the feature matching range

[0060] In theory, the features extracted by the ReID network tend to reflect the differences between different individuals of the same class, while the features of the YOLOv5 target detection network tend to reflect the differences between different classes. Changing the feature from the separate ReID network to the Neck layer feature of YOLOv5 will cause the accuracy of feature matching to decrease. Therefore, this patent further uses position information to restrict the range of feature matching, reducing the possibility of error. Specifically, feature matching is only limited to bounding boxes that overlap with each other and have a height-to-width ratio difference of no more than 2 times.

[0061] Stage 3: Vehicle pose estimation based on model prior information

[0062] The purpose of the vehicle pose estimation stage is to combine the 3D prior information provided by the digital twin city platform with the bounding box of the vehicle information obtained by the vehicle tracking part in the image coordinate system, and output a transformation matrix M∈R representing the vehicle pose. 3×4 And the predefined size closest to the target size, where M represents the transformation relationship from the city 3D scene model coordinate system to the vehicle model coordinate system, and the origin of the vehicle model's own coordinate system is located at the center of the vehicle model's 3D bounding box; R 3×4 This refers to a space of real numbers. M belonging to this space means that M is a tensor of shape 3x4, and each element of this tensor takes the value of a real number. The specific steps are as follows:

[0063] 1) Estimate the 2D pose of the vehicle

[0064] In this patent, the 2D pose of the vehicle is represented as position coordinates and a motion direction vector. To avoid errors caused by occlusion or inaccurate detection and tracking, the current bounding box center point (cx, cy) is recorded as a new position coordinate only when the motion distance of the bounding box center point exceeds a threshold T. The direction from the previous position coordinate to the current position coordinate is normalized and recorded as the current motion direction vector (vx, vy). (vx, vy) is the target's own coordinate with (cx, cy) as the origin, and the x-axis and y-axis directions are the same as the image system coordinates. In this patent, T is dynamically set to the length of the bounding box diagonal.

[0065] 2) Export model prior information

[0066] The patent uses a digital twin platform based on Unreal Engine, and the prior information used in the model is the predefined vehicle model size, camera calibration parameters, and ground information in the static scene model.

[0067] i. Predefine the dimensions of the vehicle models. In Unreal Engine, select several vehicle models with significantly different dimensions as predefined models for the visualization process. Export the length, width, and height of the corresponding 3D bounding boxes, denoted as S = {S0, S1, ..., S...}. i}, i∈[0,N],S i ={l i w i h i}, where N is the number of predefined models.

[0068] ii. Camera calibration parameters. Export the camera calibration parameters as general intrinsic and extrinsic parameter matrices.

[0069] iii. Ground information. Ground information is derived as a tensor G with shape HxWx4, where H and W are the height and width of the camera output image, and 4 is the number of corresponding channels. The specific description of G is as follows: first, place a set of segmented planes in UnrealEngine that fit the ground model as much as possible, and assign a number to each placed plane. Then, traverse each coordinate point (i, j) in the camera imaging plane in UnrealEngine, i∈[0, W-1], j∈[0, H-1], respectively calculate the coordinates of the intersection point of the ray pointing from the camera center to the coordinate point and the above plane, and the number of the intersection plane. Finally, combine the coordinates and numbers into a vector containing 4 elements along the channel direction and store it in the corresponding position of G.

[0070] 3) Pre-compute ground plane equation

[0071] After obtaining the tensor G, traverse the segmented plane numbers in turn, take out all the coordinate points belonging to the same number, and solve the coefficients A, B, C, and D of the plane equation Ax+By+Cz+D=0 corresponding to the plane using the least squares method. In the subsequent calculation process, only the target corresponding segmented plane number needs to be looked up, which avoids repeated calculations.

[0072] 4) Estimate the 3D pose of the vehicle

[0073] To further calculate the 3D pose, an approximation assumption is introduced, which assumes that (cx, cy) corresponds to the center position of a vehicle model 3D bounding box, i.e., (cx, cy) corresponds to the projection of the center position of a vehicle model 3D bounding box in the image coordinate system, (cx, cy) and (cx+vx, cy+vy) are simultaneously in a plane parallel to the plane and higher than the plane , where h is the height of the vehicle model, (cx+vx, cy+vy) has no specific physical meaning, and is a new coordinate point calculated from cx, vx, cy, and vy. For a given 2D pose, traverse each predefined size in turn, and calculate the corresponding 3D pose M under that size. Then, project the 3D bounding box under that size and pose back to the 2D image coordinate system and compare it with the actual 2D bounding box. Select the pose that best fits the back-projected 2D bounding box as the final estimate. The specific process is as follows:

[0074] i. For a given 2D pose (cx, cy) and (vx, vy), obtain the ground plane number corresponding to the ray pointing from the camera center to (cx, cy) according to the tensor G, and further obtain the corresponding pre-computed plane P according to the plane number.

[0075] ii. Traverse each predefined vehicle model size S i, calculate the plane P' parallel to P and above the plane P i , i∈[0, N].

[0076] iii. For each plane P' i , calculate the intersection point (cx i , cy i , cz i ) of the ray pointing to (cx, cy) from the camera center and the plane P' i , calculate the intersection point (vx i , vy i , vz i ) of the ray pointing to (cx+vx, cy+vy) from the camera center and the plane P' i . Calculate the pose transformation matrix M i corresponding to the translation of the origin of the scene model coordinate system to (cx i , cy i , cz i ) and the rotation of the coordinate system to the same as the normal vector of P' i and the X axis points from (cx i , cy i , cz i ) to (vx i , vy i , vz i ).

[0077] iv. Calculate the 8 2D coordinate points corresponding to the projection of the 8 vertices of the 3D bounding box with size S i at the pose M i to the image coordinate system, and find the minimum horizontal bounding box B i

[0078] v. Calculate the IoU (Intersection Over Union) between all B i and the 2D bounding box output by the vehicle tracking part, select S i and M i as the size and 3D pose of the current vehicle.

[0079] In addition, the present application also provides an electronic device, comprising: at least one processor; a memory coupled with the at least one processor, the memory storing executable instructions, wherein the executable instructions, when executed by the at least one processor, cause the implementation of the above-mentioned method of the present application.

[0080] ​For example, the memory can include random access memory, flash memory, read-only memory, programmable read-only memory, non-volatile memory, or registers, etc. The processor can be a central processing unit (CPU) or a graphic processing unit (GPU), the memory can store executable instructions, and the processor can execute the executable instructions stored in the memory to implement various processes described herein.

[0081] It can be understood that the memory in the embodiments can be volatile memory or non-volatile memory, or can include both volatile and non-volatile memory. The memory described herein is intended to include, but not limited to, these and any other suitable types of memory.

[0082] In some embodiments, the memory stores elements such as operating systems and application programs, executable units or data structures, or subsets thereof, or extended sets thereof.

[0083] The operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application program includes various application programs for implementing various application services. The program for implementing the method of the embodiments of the present application can be included in the application program.

[0084] In the embodiments of the present application, the processor calls the program or instruction stored in the memory, specifically, the program or instruction stored in the application program, and the processor is used to execute the above method steps.

[0085] The embodiments of the present application also provide a chip for executing the above method. Specifically, the chip includes a processor for calling and running a computer program from a memory, so that a device installed with the chip is used to execute the above method.

[0086] The present application also provides a computer readable storage medium, and the computer readable storage medium stores a computer program. When the computer program is executed by the processor, the steps of the above method of the present application are implemented.

[0087] For example, the machine readable storage medium can include, but not limited to, various known and unknown types of non-volatile memory.

[0088] The embodiments of the present application also provide a computer program product, which includes computer program instructions, and the computer program instructions make the computer execute the above method.

[0089] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or in a combination of software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different ways to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0090] The functions, if realized in the form of software function units and sold or used as independent products, can be stored in a machine readable storage medium. Therefore, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a machine readable storage medium, and can include a plurality of instructions to make an electronic device execute all or part of the processes described in the embodiments of the present application. The above storage medium can include ROM, RAM, removable disks, hard disks, magnetic disks or optical disks, and various media that can store program codes.

[0091] The preferred embodiments of the present application are described in detail above. It should be understood that those of ordinary skill in the art can make many modifications and changes without creative labor based on the concept of the present application. Therefore, any technical solution obtained by logical analysis, reasoning or limited experiments based on the concept of the present application and the prior art should be within the protection scope defined by the claims.

Claims

1. A vehicle pose estimation method, characterized in that, Includes the following steps: S1: Based on the target detection model, vehicle detection is performed on each frame of the image, and the 2D bounding box of the vehicle target in the image coordinate system is predicted. S2: Vehicle tracking is performed based on the improved DeepSORT algorithm, which performs feature matching between the vehicle detection results of the current frame and the detection results of past frames from the same camera. S3: Combine the prior information of the 3D scene model with the 2D bounding box of the vehicle information obtained from vehicle tracking in the image coordinate system to output a transformation matrix M∈R representing the vehicle pose. 3×4 And the predefined vehicle model size that is closest to the target size; where: M represents the transformation relationship between the 3D scene model coordinate system and the vehicle model coordinate system, R 3×4 Refers to the real number space; The improved DeepSORT algorithm described in step S2 includes the following improvements: S2-1: Remove the independent ReID feature extraction network part in DeepSORT and replace it with the existing deep features in the vehicle detection model; S2-2: Use location information to constrain the range of feature matching; S2-1 includes the following steps: S2-1-1: When inputting a 3×H×W image into the YOLOv5m6 model prediction process, save the four shapes generated by the neck part of the YOLOv5m6 model as follows: Feature image F i, i∈{0,1,2,3}; S2-1-2: For each predicted center point coordinate (cx, cy) of the 2D bounding box in the input image coordinate system, calculate the corresponding coordinates (cx, cy) of the center point in each feature image coordinate system according to the ratio of the input image size to the feature image size. i ,cy i And at that coordinate position, extract the feature vector V along the first dimension of the feature image. i ; S2-1-3: The four feature vectors V... i The concatenation of these features sequentially forms the feature corresponding to the feature matching.

2. The vehicle pose estimation method as described in claim 1, characterized in that, The feature matching is limited to 2D bounding boxes that overlap and whose aspect ratios differ by no more than twice.

3. The vehicle pose estimation method as described in claim 1, characterized in that, The combination described in step S3 includes the following steps: S3-1: Estimating the 2D pose of the vehicle: The 2D pose of the vehicle is represented by position coordinates and motion direction vector. Only when the motion distance of the center point of the 2D bounding box exceeds the threshold T, the center point (cx, cy) of the 2D bounding box at the current moment is recorded as a new position coordinate. The direction from the previous position coordinate to the current position coordinate is normalized and recorded as the motion direction vector (vx, vy) at the current moment. (vx, vy) is the target's own coordinate with (cx, cy) as the origin and the X-axis and Y-axis directions being the same as the image system coordinates. S3-2: Export prior information of the 3D scene model, including: exporting predefined vehicle model dimensions, camera calibration parameters, and ground information in the static scene model; Specifically, exporting the predefined vehicle model dimensions refers to selecting several vehicle models with significant size differences in Unreal Engine as models used in the predefined visualization process, and exporting the length, width, and height of the corresponding 3D bounding boxes, denoted as S = {S0, S1, ..., S...}. i }, i∈[0,N], S i ={l i ,w i ,h i }, where N is the number of predefined models; Exporting the camera calibration parameters means exporting the camera calibration parameters into a general intrinsic and extrinsic parameter matrix format; Exporting the ground information means exporting the ground information as a tensor G of shape H×W×4, where H and W are the height and width of the camera output image, and 4 is the corresponding number of channels; the specific description of G is as follows: First, place a set of segmented planes that fit the ground model in Unreal Engine, and assign a number to each of the placed segmented planes. Then, traverse each coordinate point (i,j) in the camera imaging plane in Unreal Engine, i∈[0,W-1],j∈[0,H-1], and calculate the coordinates of the intersection point of the ray pointing from the camera center to the coordinate point and the segmented plane, as well as the number of the intersecting segmented plane. Finally, combine the coordinates and the number into a vector with 4 elements and store it in the corresponding position of G along the channel direction; S3-3: Pre-calculated ground plane equation: After obtaining the tensor G, the segmented plane numbers are traversed sequentially, and all coordinate points belonging to the same number are extracted. The coefficients A, B, C, D of the plane equation Ax + By + Cz + D = 0 corresponding to the plane are solved using the least squares method. In the subsequent calculation process, the table is looked up according to the plane number corresponding to the target. S3-4: Estimating the vehicle's 3D pose: For a given 2D pose, iterate through each predefined vehicle model size and calculate the corresponding 3D pose M at that size. Then, backproject the size and the 3D bounding box at that pose into the 2D image coordinate system and compare it with the actual 2D bounding box. Select the 3D pose that best fits the 2D bounding box after backprojection as the final estimate.

4. The vehicle pose estimation method as described in claim 3, characterized in that, The threshold T is 1 / 4 of the length of the diagonal of the bounding box.

5. The vehicle pose estimation method as described in claim 3, characterized in that, The specific process for steps S3-4 is as follows: S3-4-1: For a given 2D pose (cx,cy) and (vx,vy), obtain the segmented plane number corresponding to the ray pointing from the camera center to (cx,cy) according to the tensor G, and obtain the plane P that has been pre-calculated in S3-3 according to the number. S3-4-2: Iterate through each predefined vehicle model size S i Calculate the plane parallel to P and above that plane. plane P i ′ ,i∈[0,N]; S3-4-3: Traverse each plane P i ′ Calculate the interaction between the ray pointing from the camera center to (cx, cy) and the plane P. i ′ The intersection point (cx) i ,cy i cz i ); Calculate the ray pointing from the camera center to (cx+vx, cy+vy) and the plane P. i ′ intersection (vx) i ,vy i ,vz i ); The calculation translates the origin of the 3D scene model coordinate system to (cx) i ,cy i cz i Simultaneously rotate the coordinate system to align with the Z-axis and P-axis. i ′ The normal vectors are the same and the X-axis is from (cx) i ,cy i cz i ) points to (vx i ,vy i ,vz i The pose transformation matrix M corresponding to ) i ; S3-4-4: Calculate pose M i The size is S i Project the 8 vertices of the 3D bounding box onto the corresponding 8 2D coordinate points in the image coordinate system, and calculate its minimum horizontal bounding box B. i ; S3-4-5: Calculate all B i The IoU between the bounding box and the 2D bounding box output by the vehicle tracking part is selected by choosing S, which maximizes the IoU. i and M i As the current vehicle's dimensions and 3D pose.

6. A chip, characterized in that, Includes a processor for retrieving and running a computer program from memory, causing a device on which the chip is mounted to perform the method of any one of claims 1-5.

7. An electronic device, characterized in that: It includes a processor and a memory for storing executable instructions of the processor, wherein the processor, when running, performs the method of any one of claims 1-5.

8. A computer-readable medium, characterized in that: It stores computer program instructions, which, when processed and executed, implement the method described in any one of claims 1-5.

Images