Vehicle pose determination method and device, equipment and storage medium

Abstract

The invention belongs to the technical field of intelligent traffic, and discloses a vehicle pose determination method and device, equipment and a storage medium. The method comprises the steps of obtaining a detection bounding box of a current vehicle based on a collected two-dimensional image, and determining pixel coordinates of a bottom edge midpoint of the detection bounding box of the current vehicle; obtaining a projection coordinate of a pixel coordinate of a bottom edge midpoint of the detection bounding box in an aerial view space; determining a vehicle course angle according to the projection coordinates in the aerial view space; according to the vehicle course angle, the projection coordinates in the aerial view space and the prior size of the vehicle, obtaining the midpoint position of the bottom surface of the vehicle; and determining the pose of the current vehicle according to the vehicle course angle and the vehicle bottom surface midpoint position, and estimating the pose of the vehicle by using the vehicle course and constraints in an aerial view space, so that the positioning accuracy is improved, an additional key point detection module is not added, and the calibration cost and the equipment time consumption are saved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation technology, and in particular to a vehicle posture determination method, device, equipment and storage medium. Background Art

[0002] With the development of roadside sensing technology, obtaining real-time traffic information through the equipment's millimeter-wave radar signals and camera signals, and then realizing signal control and road safety warning, has become an important part of the smart transportation system. Vehicle positioning and posture estimation are the foundation of the smart transportation field; however, how to improve the accuracy of vehicle position and posture estimation faces huge challenges.

[0003] When it comes to vehicle 3D pose estimation, most methods use the midpoint of the bottom edge of the vehicle detection box as a simple location for the vehicle, or add a key point detection module.

[0004] The midpoint of the bottom edge of the vehicle detection frame is used as the position of the vehicle, and the vehicle posture affects the positioning accuracy of the midpoint of the bottom edge of the detection frame for the vehicle. For example, if a vehicle is in the same position but driving in a different direction, the height of the detection frame will change, thereby changing the positioning of the midpoint of the bottom edge. In the current cross-device perception scenario, the midpoint of the bottom edge of the detection frame does not have a unified representation and meaning. For example, for two devices set up in opposite directions, due to different viewing angles, the midpoint of the bottom surface of the detection frame also corresponds to different positions of the vehicle, and the positioning error of the same vehicle between the two devices is large. In addition, the use of the key point detection module requires the device to additionally capture the position of the key points. These methods often use key point detection methods based on neural networks, which require additional labeling costs and equipment time.

[0005] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0006] The main purpose of the present invention is to provide a vehicle posture determination method, device, equipment and storage medium, aiming to solve the technical problem that the prior art cannot accurately locate the same target in a multi-device collaborative perception scenario and requires additional labeling costs and equipment time.

[0007] To achieve the above object, the present invention provides a method for determining a vehicle posture, the method comprising the following steps:

[0008] Acquire a detection bounding box of the current vehicle based on the acquired two-dimensional image, and determine the pixel coordinates of the midpoint of the bottom edge of the detection bounding box of the current vehicle;

[0009] Obtaining the projection coordinates of the pixel coordinates of the midpoint of the bottom edge of the detection bounding box in the bird's-eye view space;

[0010] Determining a vehicle heading angle according to the projection coordinates in the bird's-eye view space;

[0011] Obtaining the midpoint position of the bottom surface of the vehicle according to the vehicle heading angle, the projection coordinates in the bird's-eye view space, and the prior dimensions of the vehicle;

[0012] The current vehicle posture is determined according to the vehicle heading angle and the midpoint position of the vehicle bottom surface.

[0013] Optionally, the acquiring the vehicle prior size, obtaining the midpoint position of the bottom surface of the vehicle according to the vehicle heading angle, the projection coordinates in the bird's-eye view space, and the vehicle prior size, includes:

[0014] Determine a projection position corresponding to the projection coordinates in the bird's-eye view space;

[0015] Determine the longitudinal distance between the projection position and the midpoint of the bottom surface of the vehicle based on the vehicle heading angle and the prior dimensions of the vehicle with the coordinate system of the bird's-eye view space as a reference;

[0016] Obtaining the projection point of the bottom edge endpoint of the detection bounding box in the bird's-eye view space;

[0017] The coordinate system of the bird's-eye view space is used as a reference to determine the lateral distance between the projection position and the midpoint of the bottom surface of the vehicle according to the projection point of the bottom edge endpoint of the detection boundary box in the bird's-eye view space and the prior size of the vehicle, and the midpoint position of the bottom surface of the vehicle is obtained according to the longitudinal distance and the lateral distance.

[0018] Optionally, determining the longitudinal distance between the projection point and the midpoint of the bottom surface of the vehicle according to the vehicle heading angle and the prior dimensions of the vehicle based on the coordinate system of the bird's-eye view space includes:

[0019] Determine a set of vertex coordinates of the vehicle according to the a priori dimensions of the vehicle, wherein the set of vertex coordinates of the vehicle is based on the vehicle's own coordinate system;

[0020] The longitudinal distance between the projection point and the midpoint of the bottom surface of the vehicle is calculated based on the vehicle heading angle and the vehicle vertex coordinate set with the coordinate system of the bird's-eye view space as a reference.

[0021] Optionally, determining the lateral distance between the projection position and the midpoint of the bottom surface of the vehicle based on the projection point of the bottom edge endpoint of the detection bounding box in the bird's-eye view space and the prior size of the vehicle with reference to the coordinate system of the bird's-eye view space includes:

[0022] Constructing a first connecting line between a projection point of a bottom edge endpoint of the detection bounding box in the bird's-eye view space and a coordinate origin of a coordinate system of the bird's-eye view space;

[0023] Constructing a second connecting line between the vehicle vertex corresponding to the vehicle a priori size and the coordinate origin of the coordinate system of the bird's-eye view space;

[0024] The angle between the first connecting line and the second connecting line is obtained, and the lateral distance between the projection position and the midpoint of the bottom surface of the vehicle is calculated according to the angle.

[0025] Optionally, the pixel coordinates of the midpoint of the bottom edge of the detection bounding box of the current vehicle are the pixel coordinates of the midpoint of the bottom edge of the detection bounding box of the current vehicle at the current moment, and determining the pixel coordinates of the midpoint of the bottom edge of the detection bounding box of the current vehicle includes:

[0026] Determining whether the detection bounding box of the current vehicle performs occlusion compensation;

[0027] If the detection bounding box of the current vehicle needs to be compensated for occlusion, determining the occlusion area and the non-occlusion area of ​​the detection rectangle of the current vehicle, and obtaining the detection bounding box of the current vehicle at the previous moment;

[0028] Determine the pixel coordinates of the midpoint of the bottom edge of the detection boundary box at the previous moment, and obtain the position change of the endpoint of the non-occluded area from the previous moment to the current moment;

[0029] The pixel coordinates of the midpoint of the bottom edge of the detection bounding box of the current vehicle are determined according to the pixel coordinates of the midpoint of the bottom edge of the detection bounding box at the previous moment and the position change of the endpoint of the non-occluded area from the previous moment to the current moment.

[0030] Optionally, determining whether occlusion compensation is performed on the detection bounding box of the current vehicle includes:

[0031] Compare the detection bounding box of the current vehicle with the detection bounding boxes of other vehicles to obtain the intersection-union ratio of the overlapping areas;

[0032] If the intersection-over-union ratio of the overlapping areas exceeds a set threshold, it is determined that the detection boundary box of the current vehicle needs to be occluded and compensated.

[0033] Optionally, the vehicle posture determination method further includes:

[0034] Obtaining the point cloud boundary of the current vehicle based on the collected radar signal;

[0035] Determine the coordinates of the midpoint of the bottom edge of the point cloud boundary of the current vehicle, use the coordinates of the midpoint of the bottom edge of the point cloud boundary as the projection coordinates in the bird's-eye view space, and return to execute the step of determining the vehicle heading angle based on the projection coordinates in the bird's-eye view space.

[0036] In addition, to achieve the above-mentioned purpose, the present invention also proposes a vehicle posture determination device, the vehicle posture determination device comprising:

[0037] A processing module, used to obtain a detection bounding box of the current vehicle based on the acquired two-dimensional image, and determine the pixel coordinates of the midpoint of the bottom edge of the detection bounding box of the current vehicle;

[0038] A conversion module, used to obtain the projection coordinates of the pixel coordinates of the bottom midpoint of the detection bounding box in the bird's-eye view space;

[0039] A calculation module, used to determine the vehicle heading angle according to the projection coordinates in the bird's-eye view space;

[0040] The calculation module is further used to obtain a priori dimensions of the vehicle, and obtain the midpoint position of the bottom surface of the vehicle according to the heading angle of the vehicle, the projection coordinates in the bird's-eye view space, and the priori dimensions of the vehicle;

[0041] The acquisition module is used to determine the current vehicle posture according to the vehicle heading angle and the midpoint position of the vehicle bottom surface.

[0042] The present invention obtains a detection bounding box of the current vehicle based on a collected two-dimensional image, and determines the pixel coordinates of the bottom midpoint of the detection bounding box of the current vehicle; obtains the projection coordinates of the pixel coordinates of the bottom midpoint of the detection bounding box in a bird's-eye view space; determines the vehicle heading angle according to the projection coordinates in the bird's-eye view space; obtains the midpoint position of the vehicle bottom surface according to the vehicle heading angle, the projection coordinates in the bird's-eye view space and the vehicle prior size; determines the posture of the current vehicle according to the vehicle heading angle and the midpoint position of the vehicle bottom surface, and estimates the posture of the vehicle by using the vehicle heading and constraints in the bird's-eye view space, thereby improving positioning accuracy without adding additional key point detection modules, saving calibration costs and equipment time. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a structural schematic diagram of a vehicle posture determination device in a hardware operating environment involved in an embodiment of the present invention;

[0044] Figure 2 A schematic diagram of a flow chart of a first embodiment of a method for determining a vehicle posture according to the present invention;

[0045] Figure 3 A schematic diagram of a detection boundary box of a vehicle in an embodiment of a method for determining a vehicle posture of the present invention;

[0046] Figure 4 A schematic diagram of projection coordinates of pixel coordinates converted to a bird's-eye view space in an embodiment of a method for determining a vehicle posture of the present invention;

[0047] Figure 5A schematic diagram of a flow chart of a second embodiment of a method for determining a vehicle posture according to the present invention;

[0048] Figure 6 A schematic diagram of vehicle position estimation in an embodiment of a method for determining vehicle posture of the present invention;

[0049] Figure 7 A schematic diagram of a flow chart of a third embodiment of a method for determining a vehicle posture according to the present invention;

[0050] Figure 8 This is a structural block diagram of the first embodiment of the vehicle posture determination device of the present invention.

[0051] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0052] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0053] Reference Figure 1 , Figure 1 A schematic diagram of the structure of a vehicle posture determination device in the hardware operating environment involved in an embodiment of the present invention.

[0054] like Figure 1 As shown, the vehicle posture determination device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) memory, or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0055] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the vehicle posture determination device, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0056] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a vehicle posture determination program.

[0057] exist Figure 1 In the vehicle posture determination device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the vehicle posture determination device of the present invention can be set in the vehicle posture determination device, and the vehicle posture determination device calls the vehicle posture determination program stored in the memory 1005 through the processor 1001, and executes the vehicle posture determination method provided by the embodiment of the present invention.

[0058] The embodiment of the present invention provides a method for determining a vehicle posture, referring to Figure 2 , Figure 2 The figure is a flow chart of a first embodiment of a method for determining a vehicle posture according to the present invention.

[0059] In this embodiment, the vehicle posture determination method includes the following steps:

[0060] Step S10: acquiring a detection bounding box of the current vehicle based on the acquired two-dimensional image, and determining the pixel coordinates of the midpoint of the bottom edge of the detection bounding box of the current vehicle.

[0061] It should be noted that the executor of this embodiment is a vehicle posture determination device, wherein the vehicle posture determination device has functions such as data processing, data communication and program running. The vehicle posture determination device can be an integrated controller, a control computer and other devices. Of course, it can also be other devices with similar functions, and this embodiment does not limit this.

[0062] It should be noted that with the development of roadside perception technology, real-time traffic information is obtained through the millimeter-wave radar signals and camera signals of the equipment, and then signal control and road safety warning are realized, which has become an important part of the intelligent transportation system. Positioning and estimating the posture of the vehicle is the basis of the intelligent transportation field; and how to improve the accuracy of vehicle position and posture estimation faces huge challenges. In the problem of vehicle three-dimensional posture estimation, most methods use the midpoint of the bottom edge of the vehicle detection frame as a simple positioning of the vehicle position, or add a key point detection module. The midpoint of the bottom edge of the vehicle detection frame is used as the vehicle position, and the vehicle posture affects the positioning accuracy of the midpoint of the bottom edge of the detection frame for the vehicle. For example, if a vehicle has the same position but different driving directions, the height of the detection frame will change, thereby changing the positioning of the bottom edge midpoint. In the current cross-device perception scenario, the bottom edge midpoint of the detection frame does not have a unified representation and meaning. For example, for two devices set up in opposite directions, due to different viewing angles, the bottom midpoint of the detection frame also corresponds to different positions of the vehicle, and the positioning error of the same vehicle between the two devices is large. In addition, the use of the key point detection module requires the device to additionally capture the position of the key point. These methods often use neural network-based key point detection methods, which require additional labeling costs and equipment time.

[0063] In order to solve the above problems in this embodiment, a detection bounding box of the current vehicle is obtained based on the collected two-dimensional image, and the pixel coordinates of the bottom midpoint of the detection bounding box of the current vehicle are determined; the projection coordinates of the pixel coordinates of the bottom midpoint of the detection bounding box in the bird's-eye view space are obtained; the vehicle heading angle is determined according to the projection coordinates in the bird's-eye view space; the position of the midpoint of the bottom surface of the vehicle is obtained according to the vehicle heading angle, the projection coordinates in the bird's-eye view space and the prior size of the vehicle; the posture of the current vehicle is determined according to the vehicle heading angle and the position of the midpoint of the bottom surface of the vehicle, and the posture of the vehicle is estimated by using the vehicle heading and the constraints in the bird's-eye view space, thereby improving the positioning accuracy without adding additional key point detection modules, saving calibration costs and equipment time. Specifically, it can be implemented as follows.

[0064] In the specific implementation, the basic architecture of the solution is first described in this embodiment. In this embodiment, a vehicle detection unit, a posture estimation unit, a position estimation unit and an occlusion processing unit are included. The vehicle detection unit mainly detects and processes the vehicle targets of the visual and millimeter-wave radar signals. The posture estimation unit is completed by the Kalman filter, and the Kalman filter is used to estimate the heading of the vehicle. In the position estimation unit, the midpoint of the bottom surface of the vehicle is used to locate the vehicle, and since the midpoint of the bottom surface is the geometric center of the vehicle, it is invariant to the vehicle's orientation and the angle of observation, and it has a unified representation and meaning in the cross-device fusion scene. Since the positioning method provided in this embodiment is based on a good image detection frame, and the occluded target detection frame often deforms and cannot surround the vehicle target, the occlusion processing unit can take effect when the detection frame overlaps, thereby improving the stability of the vehicle detection module.

[0065] In a specific implementation, in this embodiment, a two-dimensional image of the vehicle can be obtained by obtaining the visual signal, and then the two-dimensional image can be input into the neural network to obtain the detection bounding box of the current vehicle. The detection bounding box of the current vehicle can be referred to Figure 3 As shown, Figure 3 B and D shown in the figure are the detection bounding boxes of the vehicle. After determining the detection bounding box of the current vehicle, the pixel coordinates of the midpoint of the bottom edge of the detection bounding box can be further obtained, for example: Figure 3 Points A and C shown in .

[0066] Furthermore, in this embodiment, millimeter-wave radar signals can also be used as a supplement to visual signals. The use scenarios of millimeter-wave radar signals are bad weather, dense queues and other scenarios. At this time, when the reliability of the visual signal is poor or the target cannot be detected, the millimeter-wave radar signal is used to replace the visual signal to improve the stability of the detection. The millimeter-wave radar signal is a point cloud signal directly output by the sensor. The point cloud boundary of the current vehicle is obtained based on the collected radar signal. The point cloud boundary itself is in the bird's-eye view space. The coordinates of the midpoint of the bottom edge of the point cloud boundary are the coordinates in the final bird's-eye view space, which can be directly used for the calculation of the subsequent vehicle heading angle and the midpoint position of the bottom surface of the vehicle. The specific calculation process is similar to the visual signal processing process and will not be repeated here.

[0067] Step S20: Obtain the projection coordinates of the pixel coordinates of the midpoint of the bottom edge of the detection boundary box in the bird's-eye view space.

[0068] In a specific implementation, after obtaining the pixel coordinates of the bottom midpoint of the detection bounding box, a projection method from the pixel coordinate system to the bird's-eye view coordinate system is required in this embodiment. The projection method is not limited in this embodiment. The homography matrix used in the following is only an example and does not constitute a limitation on this embodiment. That is, the pixel coordinates of the bottom midpoint of the detection bounding box are projected into the bird's-eye view space, thereby obtaining the projection coordinates of the pixel coordinates in the bird's-eye view space. Specifically, in this embodiment, taking the homography matrix as an example, the projection method is as follows:

[0069]

[0070] Among them, the pixel coordinates of the bottom midpoint of the detection bounding box are expressed as (u c ,v c ), the projection coordinates in the bird's-eye view space are expressed as (x c ,y c ), the homography matrix is The coordinate transformation can be completed by the above method. The projection coordinates after transformation are as follows: Figure 4 As shown, A' and C' are the projection coordinates in the bird's-eye view space.

[0071] Step S30: determining the vehicle heading angle according to the projection coordinates in the bird's-eye view space.

[0072] It should be noted that the projection coordinates in the bird's-eye view space are used as the sensor measurement. By statistically analyzing the errors between the real physical position and visual positioning at different distances, the measurement covariance is established. Then, through the standard Kalman filter, the speed, direction and other information of the object can be estimated. The vehicle heading angle can be obtained using the standard Kalman filter.

[0073] Step S40: obtaining the midpoint position of the bottom surface of the vehicle according to the vehicle heading angle, the projection coordinates in the bird's-eye view space and the prior dimensions of the vehicle.

[0074] In a specific implementation, after obtaining the vehicle heading angle and the projection coordinates in the bird's-eye view space, the vehicle prior dimensions need to be further obtained in this embodiment. Then, the vehicle heading angle, the projection coordinates in the bird's-eye view space, and the vehicle prior dimensions can respectively calculate the lateral distance and longitudinal distance between the projection position corresponding to the projection coordinates in the bird's-eye view space and the midpoint of the vehicle bottom surface. The midpoint position of the vehicle bottom surface can be determined based on the calculated lateral distance and longitudinal distance.

[0075] Step S50: determining the current vehicle posture according to the vehicle heading angle and the midpoint position of the vehicle bottom surface.

[0076] In a specific implementation, the vehicle heading angle determines the vehicle's heading attitude, and the actual position of the vehicle can also be determined based on the midpoint position of the vehicle's bottom surface. The above two pieces of information are combined to obtain the current vehicle posture.

[0077] This embodiment obtains a detection bounding box of the current vehicle based on the collected two-dimensional image, and determines the pixel coordinates of the bottom midpoint of the detection bounding box of the current vehicle; obtains the projection coordinates of the pixel coordinates of the bottom midpoint of the detection bounding box in the bird's-eye view space; determines the vehicle heading angle according to the projection coordinates in the bird's-eye view space; obtains the midpoint position of the vehicle bottom surface according to the vehicle heading angle, the projection coordinates in the bird's-eye view space and the vehicle prior size; determines the posture of the current vehicle according to the vehicle heading angle and the midpoint position of the vehicle bottom surface, and estimates the posture of the vehicle by using the vehicle heading and constraints in the bird's-eye view space, thereby improving positioning accuracy without adding additional key point detection modules, saving calibration costs and equipment time.

[0078] refer to Figure 5 , Figure 5 The figure is a flow chart of a second embodiment of a method for determining vehicle posture according to the present invention.

[0079] Based on the above first embodiment, in the vehicle posture determination method of this embodiment, step S40 specifically includes:

[0080] Step S401: Determine the projection position corresponding to the projection coordinates in the bird's-eye view space.

[0081] In a specific implementation, when determining the position of the midpoint of the bottom surface of the vehicle, it is necessary to first determine the projection position corresponding to the projection coordinates in the bird's-eye view space. The projection position can be as follows: Figure 6 As shown, Figure 6 Point C shown is the projection position.

[0082] Step S402: Determine the longitudinal distance between the projection position and the midpoint of the bottom surface of the vehicle based on the vehicle heading angle and the prior dimensions of the vehicle, with the coordinate system of the bird's-eye view space as a reference.

[0083] In a specific implementation, the coordinate system of the bird's-eye view space described in this embodiment is Figure 6 The coordinate system XdOdYd shown in , with this coordinate system as the reference and then combined with the vehicle heading angle and the vehicle prior dimensions, the longitudinal distance between the projection position and the midpoint of the vehicle bottom surface can be calculated. Specifically, in this embodiment, the vehicle vertex coordinate set is determined according to the vehicle prior dimensions, and the vehicle vertex coordinate set is based on the vehicle's own coordinate system. For example, the vehicle's prior width is recorded as w, and the length is recorded as l. In the vehicle's own coordinate system ( Figure 6 The coordinate system X shown in v O v Yv ), the set of vehicle vertex coordinate markers is M, where the set M is represented by Then, the longitudinal distance between the projection point and the midpoint of the vehicle bottom surface is calculated based on the vehicle heading angle and the vehicle vertex coordinate set, with the coordinate system of the bird's-eye view space as the reference, for example: Among them, dy represents the longitudinal distance, x and y are selected from the set M, and θ represents the vehicle heading angle.

[0084] Step S403: Obtain the projection point of the bottom edge endpoint of the detection boundary box in the bird's-eye view space.

[0085] Step S404: Taking the coordinate system of the bird's-eye view space as a reference, determine the lateral distance between the projection position and the midpoint of the bottom surface of the vehicle according to the projection point of the bottom edge endpoint of the detection boundary box in the bird's-eye view space and the prior size of the vehicle, and obtain the midpoint position of the bottom surface of the vehicle according to the longitudinal distance and the lateral distance.

[0086] In the specific implementation, in this embodiment, it is also necessary to obtain the projection point of the bottom edge endpoint of the detection bounding box in the bird's-eye view space. Since the vehicle target is just surrounded by the detection rectangular box BBox in the image, the projection of the bottom edge endpoint of the detection box in the bird's-eye view space is Figure 6 Points A and B shown in .

[0087] Furthermore, in this embodiment, the coordinate system of the bird's-eye view space is also used as a reference, and then the lateral distance between the projection position and the midpoint of the bottom surface of the vehicle is calculated based on the projection point of the bottom edge endpoint of the detection boundary box in the bird's-eye view space and the coordinate origin of the coordinate system of the bird's-eye view space. Specifically, a first line is constructed between the projection point of the bottom edge endpoint of the detection boundary box in the bird's-eye view space and the coordinate origin of the coordinate system of the bird's-eye view space; a second line is constructed between the vehicle vertex corresponding to the vehicle prior size and the coordinate origin of the coordinate system of the bird's-eye view space; the angle between the first line and the second line is obtained, and the lateral distance between the projection position and the midpoint of the bottom surface of the vehicle is calculated based on the angle. Since the two bottom edge endpoints of the detection boundary box are used in this embodiment, the first line can be obtained for the projection point of any bottom edge endpoint in the bird's-eye view space and the coordinate origin of the coordinate system of the bird's-eye view space, and then the second line is constructed between the vehicle vertex corresponding to the vehicle prior size and the coordinate origin of the coordinate system of the bird's-eye view space, as shown in FIG. Figure 6 As shown, for the endpoint A and the vertex M on the left side of the vehicle l For example, the first connection is OdA and the second connection is OdM l , the angle between the first connecting line and the second connecting line is θ l , for endpoint B and vertex M on the right side of the vehicle r For example, the first connection is OdB and the second connection is OdM r, the angle between the first connecting line and the second connecting line is θ r Based on the angle obtained above, the lateral distance between the projection position and the midpoint of the vehicle bottom surface can be calculated, for example Where dx represents the lateral distance, θ r With θ l Represents the angle between the first line and the second line.

[0088] This embodiment determines the projection position corresponding to the projection coordinates in the bird's-eye view space; determines the longitudinal distance between the projection position and the midpoint of the bottom surface of the vehicle according to the vehicle heading angle and the prior dimensions of the vehicle based on the coordinate system of the bird's-eye view space; obtains the projection point of the bottom edge endpoint of the detection boundary box in the bird's-eye view space; determines the lateral distance between the projection position and the midpoint of the bottom surface of the vehicle according to the projection point of the bottom edge endpoint of the detection boundary box in the bird's-eye view space and the prior dimensions of the vehicle based on the coordinate system of the bird's-eye view space, and obtains the midpoint position of the bottom surface of the vehicle according to the longitudinal distance and the lateral distance. The midpoint position of the vehicle on the ground is calculated in the above manner, thereby achieving accurate positioning of the vehicle position.

[0089] refer to Figure 7 , Figure 7 The figure is a flow chart of a second embodiment of a method for determining vehicle posture according to the present invention.

[0090] Based on the above first embodiment, in the vehicle posture determination method of this embodiment, step S10 specifically includes:

[0091] Step S101: Determine whether the detection boundary box of the current vehicle is subjected to occlusion compensation.

[0092] Step S102: If the detection bounding box of the current vehicle needs to be compensated for occlusion, the occlusion area and the non-occlusion area of ​​the detection rectangle of the current vehicle are determined, and the detection bounding box of the current vehicle at the previous moment is obtained.

[0093] In a specific implementation, when obtaining the pixel coordinates of the midpoint of the bottom edge of the detection bounding box of the current vehicle, it is necessary to first determine whether the detection bounding box performs occlusion compensation. Specifically, in this embodiment, when determining the vehicle posture, the detection bounding boxes of multiple vehicles can be obtained, and then the detection bounding box of the current vehicle is compared with the detection bounding boxes of other vehicles to obtain the intersection and union ratio of the overlapping area. If the intersection and union ratio of the overlapping area exceeds the set threshold, it is determined that the detection bounding box of the current vehicle needs to be occluded. The threshold can be set according to actual needs, and this embodiment is not limited to this. If the intersection and union ratio of the overlapping area exceeds the set threshold, the pixel coordinates of the midpoint of the bottom edge can be directly obtained through the detection bounding box. Furthermore, if occlusion compensation is required, the occluded area and non-occluded area of ​​the detection rectangle of the current vehicle are further obtained, and the detection bounding box of the current vehicle at the previous moment is obtained.

[0094] Step S103: determining the pixel coordinates of the midpoint of the bottom edge of the detection boundary box at the previous moment, and obtaining the position change of the endpoint of the non-occluded area from the previous moment to the current moment.

[0095] Step S104: Determine the pixel coordinates of the midpoint of the bottom edge of the detection bounding box of the current vehicle according to the pixel coordinates of the midpoint of the bottom edge of the detection bounding box at the previous moment and the position change of the endpoint of the non-occluded area from the previous moment to the current moment.

[0096] In the specific implementation, the detection bounding box at the previous moment is used to predict the detection bounding box at the current moment. The pixel coordinates of the corresponding bottom edge midpoint can also be obtained based on the detection bounding box at the previous moment, and then the position change of the endpoint in the non-occluded area from the previous moment to the current moment is recorded. Finally, the pixel coordinates of the bottom edge midpoint of the detection bounding box of the current vehicle can be calculated based on the pixel coordinates of the bottom edge midpoint of the detection bounding box at the previous moment and the position change of the endpoint in the non-occluded area from the previous moment to the current moment, for example (u t ,v t )=(u t-1 ,v t-1 )+(△u,△t), where (u t-1 ,v t-1 ) represents the pixel coordinates of the midpoint of the bottom edge of the detection bounding box at the previous moment, and (△u,△t) represents the position change of the endpoint of the non-occluded area from the previous moment to the current moment.

[0097] In addition, for continuous occlusion of n frames, the position at time tn before the occlusion is used to continuously compensate for the real pixel coordinate position of the occluded target. For continuous occlusion, since there is a certain error between the compensated predicted position and the real position, a linear weighted combination is performed by compensating the predicted position and the observed position output by unit 201 to improve the stability of the system, that is, in the case of continuous occlusion, the pixel coordinates predicted above and the pixel coordinates obtained by observation are used for weighted calculation to obtain the pixel coordinates at the current moment. The weight value used in the calculation can be set based on actual needs, and this is not limited in this embodiment.

[0098] This embodiment determines whether the detection bounding box of the current vehicle needs to be compensated for occlusion; if the detection bounding box of the current vehicle needs to be compensated for occlusion, the occluded area and the non-occluded area of ​​the detection rectangle of the current vehicle are determined, and the detection bounding box of the current vehicle at the previous moment is obtained; the pixel coordinates of the midpoint of the bottom edge of the detection bounding box at the previous moment are determined, and the position change of the endpoint of the non-occluded area from the previous moment to the current moment is obtained; the pixel coordinates of the midpoint of the bottom edge of the detection bounding box of the current vehicle are determined according to the pixel coordinates of the midpoint of the bottom edge of the detection bounding box at the previous moment and the position change of the endpoint of the non-occluded area from the previous moment to the current moment. The above method can take effect when the detection frames overlap, improve the stability of the vehicle detection module, and ensure the accuracy of vehicle posture acquisition.

[0099] In addition, an embodiment of the present invention further proposes a storage medium, on which a vehicle posture determination program is stored. When the vehicle posture determination program is executed by a processor, the steps of the vehicle posture determination method described above are implemented.

[0100] Reference Figure 8 , Figure 8 This is a structural block diagram of the first embodiment of the vehicle posture determination device of the present invention.

[0101] like Figure 8 As shown, the vehicle posture determination device proposed in the embodiment of the present invention includes:

[0102] The processing module 10 is used to obtain a detection bounding box of the current vehicle based on the acquired two-dimensional image, and determine the pixel coordinates of the midpoint of the bottom edge of the detection bounding box of the current vehicle.

[0103] The conversion module 20 is used to obtain the projection coordinates of the pixel coordinates of the midpoint of the bottom edge of the detection boundary box in the bird's-eye view space.

[0104] The calculation module 30 is used to determine the vehicle heading angle according to the projection coordinates in the bird's-eye view space.

[0105] The calculation module 30 is further used to obtain a priori dimensions of the vehicle, and obtain the midpoint position of the bottom surface of the vehicle according to the vehicle heading angle, the projection coordinates in the bird's-eye view space, and the priori dimensions of the vehicle;

[0106] The acquisition module 40 is used to determine the current vehicle posture according to the vehicle heading angle and the midpoint position of the vehicle bottom surface.

[0107] This embodiment obtains a detection bounding box of the current vehicle based on the collected two-dimensional image, and determines the pixel coordinates of the bottom midpoint of the detection bounding box of the current vehicle; obtains the projection coordinates of the pixel coordinates of the bottom midpoint of the detection bounding box in the bird's-eye view space; determines the vehicle heading angle according to the projection coordinates in the bird's-eye view space; obtains the midpoint position of the vehicle bottom surface according to the vehicle heading angle, the projection coordinates in the bird's-eye view space and the vehicle prior size; determines the posture of the current vehicle according to the vehicle heading angle and the midpoint position of the vehicle bottom surface, and estimates the posture of the vehicle by using the vehicle heading and constraints in the bird's-eye view space, thereby improving positioning accuracy without adding additional key point detection modules, saving calibration costs and equipment time.

[0108] In one embodiment, the calculation module 30 is further used to determine the projection position corresponding to the projection coordinates in the bird's-eye view space; determine the longitudinal distance between the projection position and the midpoint of the bottom surface of the vehicle based on the vehicle heading angle and the prior size of the vehicle with the coordinate system of the bird's-eye view space as a reference; obtain the projection point of the bottom edge endpoint of the detection bounding box in the bird's-eye view space; determine the lateral distance between the projection position and the midpoint of the bottom surface of the vehicle based on the projection point of the bottom edge endpoint of the detection bounding box in the bird's-eye view space and the prior size of the vehicle with the coordinate system of the bird's-eye view space as a reference, and obtain the midpoint position of the bottom surface of the vehicle based on the longitudinal distance and the lateral distance.

[0109] In one embodiment, the calculation module 30 is further used to determine a vehicle vertex coordinate set based on the vehicle prior dimensions, wherein the vehicle vertex coordinate set is based on the vehicle's own coordinate system; and to calculate the longitudinal distance between the projection point and the midpoint of the vehicle bottom surface based on the vehicle heading angle and the vehicle vertex coordinate set based on the coordinate system of the bird's-eye view space.

[0110] In one embodiment, the calculation module 30 is further used to construct a first line between the projection point of the bottom edge endpoint of the detection bounding box in the bird's-eye view space and the coordinate origin of the coordinate system of the bird's-eye view space; construct a second line between the vehicle vertex corresponding to the vehicle prior size and the coordinate origin of the coordinate system of the bird's-eye view space; obtain the angle between the first line and the second line, and calculate the lateral distance between the projection position and the midpoint of the bottom surface of the vehicle based on the angle.

[0111] In one embodiment, the processing module 10 is also used to determine whether the detection bounding box of the current vehicle needs to be compensated for occlusion; if the detection bounding box of the current vehicle needs to be compensated for occlusion, the occluded area and the non-occluded area of ​​the detection rectangle of the current vehicle are determined, and the detection bounding box of the current vehicle at the previous moment is obtained; the pixel coordinates of the midpoint of the bottom edge of the detection bounding box at the previous moment are determined, and the position change of the endpoint of the non-occluded area from the previous moment to the current moment is obtained; the pixel coordinates of the midpoint of the bottom edge of the detection bounding box of the current vehicle are determined according to the pixel coordinates of the midpoint of the bottom edge of the detection bounding box at the previous moment and the position change of the endpoint of the non-occluded area from the previous moment to the current moment.

[0112] In one embodiment, the processing module 10 is further used to compare the detection bounding box of the current vehicle with the detection bounding boxes of other vehicles to obtain an intersection-and-union ratio of overlapping areas; if the intersection-and-union ratio of overlapping areas exceeds a set threshold, it is determined that the detection bounding box of the current vehicle needs to be occluded.

[0113] In one embodiment, the processing module 10 is also used to obtain the point cloud boundary of the current vehicle based on the collected radar signal; determine the coordinates of the midpoint of the bottom edge of the point cloud boundary of the current vehicle, use the coordinates of the midpoint of the bottom edge of the point cloud boundary as the projection coordinates in the bird's-eye view space, and return to execute the step of determining the vehicle heading angle according to the projection coordinates in the bird's-eye view space.

[0114] It should be understood that the above is only an example and does not constitute any limitation on the technical solution of the present invention. In specific applications, technicians in this field can make settings as needed, and the present invention does not limit this.

[0115] It should be understood that, although the various steps in the flowchart in the embodiment of the present application are displayed in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and it can be performed in other orders. Moreover, at least a portion of the steps in the figure may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and their execution order is not necessarily performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0116] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present invention. In practical applications, technicians in this field can select part or all of them according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.

[0117] In addition, it should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0118] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0119] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0120] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for determining vehicle posture, characterized in that: The vehicle posture determination method comprises: Acquire a detection bounding box of the current vehicle based on the acquired two-dimensional image, and determine the pixel coordinates of the midpoint of the bottom edge of the detection bounding box of the current vehicle; Obtaining the projection coordinates of the pixel coordinates of the midpoint of the bottom edge of the detection bounding box in the bird's-eye view space; Determining a vehicle heading angle according to the projection coordinates in the bird's-eye view space; Obtaining the midpoint position of the bottom surface of the vehicle according to the vehicle heading angle, the projection coordinates in the bird's-eye view space, and the prior dimensions of the vehicle; The current vehicle posture is determined according to the vehicle heading angle and the midpoint position of the vehicle bottom surface.

2. The vehicle posture determination method according to claim 1, characterized in that: The obtaining of the vehicle prior dimensions, obtaining the midpoint position of the bottom surface of the vehicle according to the vehicle heading angle, the projection coordinates in the bird's-eye view space, and the vehicle prior dimensions, includes: Determine a projection position corresponding to the projection coordinates in the bird's-eye view space; Determine the longitudinal distance between the projection position and the midpoint of the bottom surface of the vehicle based on the vehicle heading angle and the prior dimensions of the vehicle with the coordinate system of the bird's-eye view space as a reference; Obtaining the projection point of the bottom edge endpoint of the detection bounding box in the bird's-eye view space; The coordinate system of the bird's-eye view space is used as a reference to determine the lateral distance between the projection position and the midpoint of the bottom surface of the vehicle according to the projection point of the bottom edge endpoint of the detection boundary box in the bird's-eye view space and the prior size of the vehicle, and the midpoint position of the bottom surface of the vehicle is obtained according to the longitudinal distance and the lateral distance.

3. The vehicle posture determination method according to claim 2, characterized in that: The method of determining the longitudinal distance between the projection point and the midpoint of the bottom surface of the vehicle based on the vehicle heading angle and the prior dimensions of the vehicle with the coordinate system of the bird's-eye view space as a reference includes: Determine a set of vertex coordinates of the vehicle according to the a priori dimensions of the vehicle, wherein the set of vertex coordinates of the vehicle is based on the vehicle's own coordinate system; The longitudinal distance between the projection point and the midpoint of the bottom surface of the vehicle is calculated based on the vehicle heading angle and the vehicle vertex coordinate set with the coordinate system of the bird's-eye view space as a reference.

4. The vehicle posture determination method according to claim 3, characterized in that: The step of determining the lateral distance between the projection position and the midpoint of the bottom surface of the vehicle based on the projection point of the bottom edge endpoint of the detection bounding box in the bird's-eye view space and the prior size of the vehicle with the coordinate system of the bird's-eye view space as a reference comprises: Constructing a first connecting line between a projection point of a bottom edge endpoint of the detection bounding box in the bird's-eye view space and a coordinate origin of a coordinate system of the bird's-eye view space; Constructing a second connecting line between the vehicle vertex corresponding to the vehicle a priori size and the coordinate origin of the coordinate system of the bird's-eye view space; The angle between the first connecting line and the second connecting line is obtained, and the lateral distance between the projection position and the midpoint of the bottom surface of the vehicle is calculated according to the angle.

5. The vehicle posture determination method according to any one of claims 1 to 4, characterized in that: The pixel coordinates of the midpoint of the bottom edge of the detection bounding box of the current vehicle are the pixel coordinates of the midpoint of the bottom edge of the detection bounding box of the current vehicle at the current moment, and the determining the pixel coordinates of the midpoint of the bottom edge of the detection bounding box of the current vehicle includes: Determining whether the detection bounding box of the current vehicle performs occlusion compensation; If the detection bounding box of the current vehicle needs to be compensated for occlusion, determining the occlusion area and the non-occlusion area of ​​the detection rectangle of the current vehicle, and obtaining the detection bounding box of the current vehicle at the previous moment; Determine the pixel coordinates of the midpoint of the bottom edge of the detection boundary box at the previous moment, and obtain the position change of the endpoint of the non-occluded area from the previous moment to the current moment; The pixel coordinates of the midpoint of the bottom edge of the detection bounding box of the current vehicle are determined according to the pixel coordinates of the midpoint of the bottom edge of the detection bounding box at the previous moment and the position change of the endpoint of the non-occluded area from the previous moment to the current moment.

6. The method according to claim 5, characterized in that The determining whether the detection boundary box of the current vehicle performs occlusion compensation includes: Compare the detection bounding box of the current vehicle with the detection bounding boxes of other vehicles to obtain the intersection-union ratio of the overlapping areas; If the intersection-over-union ratio of the overlapping areas exceeds a set threshold, it is determined that the detection boundary box of the current vehicle needs to be occluded and compensated.

7. The vehicle posture determination method according to claim 1, characterized in that: The vehicle posture determination method also includes: Obtaining the point cloud boundary of the current vehicle based on the collected radar signal; Determine the coordinates of the midpoint of the bottom edge of the point cloud boundary of the current vehicle, use the coordinates of the midpoint of the bottom edge of the point cloud boundary as the projection coordinates in the bird's-eye view space, and return to execute the step of determining the vehicle heading angle based on the projection coordinates in the bird's-eye view space.

8. A vehicle posture determination device, characterized in that: The vehicle posture determination device comprises: A processing module, used to obtain a detection bounding box of the current vehicle based on the acquired two-dimensional image, and determine the pixel coordinates of the midpoint of the bottom edge of the detection bounding box of the current vehicle; A conversion module, used to obtain the projection coordinates of the pixel coordinates of the bottom midpoint of the detection bounding box in the bird's-eye view space; A calculation module, used to determine the vehicle heading angle according to the projection coordinates in the bird's-eye view space; The calculation module is further used to obtain a priori dimensions of the vehicle, and obtain the midpoint position of the bottom surface of the vehicle according to the heading angle of the vehicle, the projection coordinates in the bird's-eye view space, and the priori dimensions of the vehicle; The acquisition module is used to determine the current vehicle posture according to the vehicle heading angle and the midpoint position of the vehicle bottom surface.

9. A vehicle posture determination device, characterized in that: The device comprises: a memory, a processor, and a vehicle posture determination program stored in the memory and executable on the processor, wherein the vehicle posture determination program is configured to implement the steps of the vehicle posture determination method as claimed in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores a vehicle posture determination program, and when the vehicle posture determination program is executed by the processor, the steps of the vehicle posture determination method according to any one of claims 1 to 7 are implemented.

Images