A method, device, and storage medium for obtaining vehicle orientation information

Abstract

The present invention relates to the technical field of intelligent transportation, and specifically discloses a method for obtaining vehicle orientation information, including: collecting an image of a vehicle to be detected; obtaining the body bounding box, the head / tail bounding box, the wheel bounding box, the vehicle category, and the head / tail category information in the image of the vehicle to be detected; calculating a first orientation angle and its uncertainty according to the vehicle category, the head / tail category, the body bounding box, and the head / tail bounding box information; calculating a second orientation angle and its uncertainty according to the wheel bounding box information; calculating a third orientation angle and its uncertainty by weighted calculation according to the first orientation angle, the second orientation angle, and their respective uncertainties, and updating them to output the target orientation angle and its uncertainty of the vehicle to be detected. The present invention also discloses a device for obtaining vehicle orientation information. The method for obtaining vehicle orientation information provided by the present invention can more quickly and robustly determine the target vehicle orientation information.

Description

Technical Field

[0001] The present invention relates to the technical fields of intelligent transportation and intelligent driving, and more particularly, to a method, apparatus, and storage medium for obtaining vehicle orientation information. Background Art

[0002] For the fields of autonomous driving and intelligent transportation, identifying the vehicle orientation angle is very important for judging the state and future trajectory of a target vehicle. In the prior art, methods that rely on lidar and other means to obtain vehicle orientation information are costly and relatively complex. Most methods for obtaining the orientation of a target vehicle using image information rely on information such as vehicle key points and structure lines, which pose challenges in terms of real-time performance and robustness for applications in the fields of intelligent transportation and intelligent driving. Summary of the Invention

[0003] To address the deficiencies in the prior art, the present invention provides a method for obtaining vehicle orientation information, which can more quickly and robustly determine the orientation information of a target vehicle.

[0004] As a first aspect of the present invention, there is provided a method for obtaining vehicle orientation information, including the following steps:

[0005] Collect an image of the vehicle to be detected;

[0006] Obtain the body bounding box, head / tail bounding box, wheel bounding box, vehicle category, and head / tail category information in the image of the vehicle to be detected;

[0007] Calculate the first orientation angle and its uncertainty of the vehicle to be detected based on the vehicle category, head / tail category, body bounding box, and head / tail bounding box information;

[0008] Calculate the second orientation angle and its uncertainty of the vehicle to be detected based on the wheel bounding box information;

[0009] Based on the first orientation angle of the vehicle to be detected, the uncertainty of the first orientation angle, the second orientation angle, and the uncertainty of the second orientation angle, calculate the third orientation angle and its uncertainty of the vehicle to be detected by weighted calculation;

[0010] Input the third orientation angle and its uncertainty of the vehicle to be detected into a Kalman filter for updating, so as to output the target orientation angle and its uncertainty of the vehicle to be detected.

[0011] Further, in the step of calculating the first orientation angle and its uncertainty of the vehicle to be detected based on the vehicle category, head / tail category, body bounding box, and head / tail bounding box information, the following steps are further included:

[0012] Obtain the length and width information of the vehicle to be detected based on the vehicle category information and the pre-stored vehicle size information of each type;

[0013] Calculate the first orientation angle of the vehicle to be detected based on the length and width information of the vehicle to be detected, the internal and external camera parameters, the body bounding box, the front / rear bounding box, and the front / rear category information;

[0014] Output the first orientation angle of the vehicle to be detected and its uncertainty.

[0015] Furthermore, the length and width information of the vehicle to be detected includes the vehicle length L and the width W;

[0016] Define l a as the x-axis pixel distance from the point on the bottom edge of the body bounding box or the front / rear bounding box closest to the left side of the image to the image center point C;

[0017] Define l b as the x-axis pixel distance from the point on the bottom edge of the body bounding box or the front / rear bounding box closest to the right side of the image to the image center point C;

[0018] Define l c as the x-axis pixel distance from the point on the bottom edge of the rear bounding box that is closer to the center point of the body bounding box among the two bottom edge endpoints to the image center point C;

[0019] The camera internal parameter calibration information of the camera internal and external parameter information is as follows:

[0020]

[0021] Among them, f x , f y is the camera pixel focal length, c x , c y is the optical center coordinate;

[0022] Define (x outer , y outer , w outer , h outer ) as the body bounding box information, (x inner , y inner , w inner , h inner ) as the front / rear bounding box information;

[0023] It is calculated that:

[0024] l a = x outer - c x

[0025] l b = x outer+w outer -c x

[0026] l c = x inner +w inner -c x

[0027] Define left_diff as the absolute difference in the x-axis pixel distance between the vehicle body bounding box and the front / rear bumper bounding box at the left bottom endpoint of the image, and right_diff as the absolute difference in the x-axis pixel distance between the vehicle body bounding box and the front / rear bumper bounding box at the right bottom endpoint of the image;

[0028] If the calculated left_diff is greater than right_diff, then l c is calculated by the following formula:

[0029] l c = x inner -c x

[0030] According to the above definition and the calculated l a ,l b ,l c and the camera internal parameter f x ,calculate the angle θ a ,θ b ,θ c as follows:

[0031]

[0032]

[0033]

[0034] Define the angle as the included angle formed by the perspective line formed by the left bottom point of the vehicle to be detected and the optical center and the rear bumper line or the vehicle body line; Define the angle as the included angle formed by the perspective line formed by the right bottom point of the vehicle to be detected and the optical center and the vehicle body line or the rear bumper line;

[0035] Derived from the sine law:

[0036]

[0037] Also, according to the quadrilateral interior angle theorem, the angle and the angle and the angle θ a and θ b satisfy the following formula two:

[0038]

[0039] Substituting the second formula into the first formula, we can obtain:

[0040]

[0041] If left_diff is greater than right_diff, then after swapping W and L in formula three, that is, swapping the values of the preset length and width, calculate the angle according to formula three;

[0042] After obtaining the angle the first orientation angle θ of the vehicle to be detected heading1 can be calculated by the following formula four:

[0043]

[0044] If the left_diff is greater than the right_diff, the first orientation angle θ of the vehicle to be detected calculated heading1 is corrected by the following formula five:

[0045]

[0046] If the head / tail bounding box category of the vehicle to be detected is the head, the first orientation angle θ of the vehicle to be detected calculated heading1 is corrected by the following formula six:

[0047] θ heading1 = θ heading1 + π Formula six

[0048] where the uncertainty of the first orientation angle of the vehicle to be detected is represented by the Gaussian noise σ heading1 2 and is obtained by fitting and tuning the parameters with the true value, and is determined by the following formula seven:

[0049] σ heading1 2 = (arctan(a * w inner )) 2 Formula seven

[0050] where a is a parameter that can be adjusted according to the true value result.

[0051] Furthermore, in calculating the second orientation angle of the vehicle to be detected and its uncertainty according to the wheel bounding box information, the following steps are further included:

[0052] Step 601: Obtain the detection result of the wheel bounding box information;

[0053] Step 602: Sort the wheel bounding boxes according to the width or height of the wheel bounding boxes;

[0054] Step 603: Select the two wheel bounding boxes that are ranked the highest and the lowest;

[0055] Step 604: Calculate the coordinates of the midpoints of the bottom edges of the two wheel bounding boxes that are ranked the highest and the lowest;

[0056] Step 606: Combine the internal and external camera parameter information, and project the coordinates of the midpoints of the bottom edges of the two wheel bounding boxes that are ranked the highest and the lowest back to the camera coordinate system;

[0057] Step 608: According to the information of the two wheel bounding boxes projected back to the camera coordinate system and the head / tail category information, calculate the second orientation angle and its uncertainty of the vehicle to be detected;

[0058] Step 609: Output the second orientation angle and its uncertainty of the vehicle to be detected.

[0059] Further, let (u, v) be the pixel coordinates to be calculated, and (x c , y c , z c ) be the coordinates of this pixel projected back to the camera coordinate system; then according to the camera model and the internal and external camera parameter information we can get:

[0060]

[0061] Get Get

[0062] Respectively substitute the coordinates of the midpoints of the bottom edges of the two wheel bounding boxes (x wheel1 , y wheel1 ) and (x wheel2 , y wheel2 ) as the u and v values, and assume that the heights of the two wheel bounding boxes are the same, then we get and From the arctangent function, we can get the angle:

[0063]

[0064] Then the second orientation angle θ of the vehicle to be detected heading2 :

[0065] θ heading2 = φ wheel

[0066] When the category of the head / tail bounding box is the head, the second orientation angle θ of the vehicle to be detectedheading2 Corrected by the following formula:

[0067] θ heading2 = π + φ wheel

[0068] wherein, the uncertainty of the second orientation angle of the vehicle to be detected is described by Gaussian noise σ heading2 2 σ heading2 2 is determined by the following formula:

[0069] σ heading2 2 = (arctan(b * w inner + c * abs(x inner - c x ))) 2

[0070] wherein, b and c are adjustable parameters.<s

[0071] Furthermore, calculating the third orientation angle and its uncertainty of the vehicle to be detected by weighted calculation according to the first orientation angle, the uncertainty of the first orientation angle, the second orientation angle and the uncertainty of the second orientation angle of the vehicle to be detected further includes the following steps:

[0072] Step ⑦〇④: Performing weighted summation based on the first orientation angle and its uncertainty information, the second orientation angle and its uncertainty information of the vehicle to be detected and the weighting coefficients stored in the storage device to obtain the third orientation angle and its uncertainty of the vehicle to be detected;

[0073] Step ⑦〇⑤: Outputting the third orientation angle and its uncertainty of the vehicle to be detected;

[0074] wherein, the weighting coefficients are composed of an orientation angle weighting coefficient α and an uncertainty weighting coefficient β, and the initial value of the orientation angle weighting coefficient α and the initial value of the uncertainty weighting coefficient β of the orientation angle are calculated according to the following formula:

[0075]

[0076] wherein, the third orientation angle θ heading3 and its uncertainty σ heading3 2 obtained by weighted summation are obtained by the following formula:

[0077] θ heading3 = αθ heading1 [[ID=6,2]]+(1 - α)θ heading2

[0078] σ heading32 = βσ heading1 2 +(1 - β)σ heading2 2 。

[0079] Further, inputting the third orientation angle and its uncertainty of the vehicle to be detected into the Kalman filter for updating to output the target orientation angle and its uncertainty of the vehicle to be detected further includes:

[0080] The state space of the Kalman filter is:

[0081]

[0082] Transfer function:

[0083]

[0084] where the posterior vehicle orientation angle θ of the vehicle to be detected is output through the Kalman filter update module kf and σ characterizing θ kf uncertainty kf 2 ;

[0085] Updating the weighting coefficients for the posterior vehicle orientation angle θ kf and σ characterizing θ kf uncertainty kf 2 The update formula for the orientation angle weighting coefficient α:

[0086]

[0087] The update formula for the orientation angle uncertainty weighting coefficient β:

[0088]

[0089] where the updated weighting coefficients are stored in the storage device.

[0090] Further, it further includes:

[0091] Correcting the target orientation angle and its uncertainty of the vehicle to be detected according to the external camera parameters; where the corrected yaw angle φ yaw is calculated by the following formula:

[0092] θ final = θ kf - φ yaw

[0093] σ final 2 = σ kf2

[0094] Output the corrected target orientation angle and its uncertainty of the vehicle to be detected.

[0095] As a second aspect of the present invention, there is provided a device for obtaining vehicle orientation information, including:

[0096] An acquisition module for acquiring an image of the vehicle to be detected;

[0097] An acquisition module for obtaining the body bounding box, the front / rear bounding box, the wheel bounding box, the vehicle category, and the front / rear category information in the image of the vehicle to be detected;

[0098] A first calculation module for calculating a first orientation angle and its uncertainty of the vehicle to be detected according to the vehicle category, the front / rear category, the body bounding box, and the front / rear bounding box information;

[0099] A second calculation module for calculating a second orientation angle and its uncertainty of the vehicle to be detected according to the wheel bounding box information;

[0100] A third calculation module for calculating a third orientation angle and its uncertainty of the vehicle to be detected by weighted calculation according to the first orientation angle of the vehicle to be detected, the uncertainty of the first orientation angle, the second orientation angle, and the uncertainty of the second orientation angle;

[0101] A Kalman filter for updating the third orientation angle and its uncertainty of the vehicle to be detected to output the target orientation angle and its uncertainty of the vehicle to be detected.

[0102] As a third aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions for causing the computer to execute the steps of the method as described above.

[0103] The method for obtaining vehicle orientation information provided by the present invention has the following advantages: it can determine the target vehicle orientation information more quickly and robustly, and can well solve the situation of accidental loss of single information. Brief Description of the Drawings

[0104] The drawings are used to provide a further understanding of the present invention and constitute a part of the specification, and are used to explain the present invention together with the following specific embodiments, but do not constitute a limitation to the present invention.

[0105] Figure 1 It is a flowchart of the method for obtaining vehicle orientation information provided by an embodiment of the present invention.

[0106] Figure 2Flowchart of the specific implementation of the method for obtaining vehicle orientation information provided by the embodiments of the present invention.

[0107] Figure 3 Schematic diagram of the body surround frame, front / rear surround frame, and wheel surround frame provided by the embodiments of the present invention.

[0108] Figure 4 Schematic diagram of the process for calculating the first orientation angle of the vehicle to be detected based on the body surround frame and the front / rear surround frame provided by the embodiments of the present invention.

[0109] Figure 5 Schematic diagram of the angular relationship for calculating the first orientation angle of the vehicle to be detected based on the body surround frame and the front / rear surround frame provided by the embodiments of the present invention.

[0110] Figure 6 Schematic diagram of the process for calculating the second orientation angle of the vehicle to be detected based on the wheel surround frame provided by the embodiments of the present invention.

[0111] Figure 7 Schematic diagram of the process for calculating the third orientation angle by combining the first orientation angle and the second orientation angle provided by the embodiments of the present invention.

[0112] Figure 8 Schematic diagram of the working process of the Kalman filter provided by the embodiments of the present invention.

[0113] Figure 9 Schematic diagram of the working process of the external orientation correction module provided by the embodiments of the present invention. Specific implementation

[0114] To further elaborate on the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following describes in detail the specific implementation, structure, features, and effects of the method, device, and storage medium for obtaining vehicle orientation information proposed according to the present invention in combination with the accompanying drawings and preferred embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the protection scope of the present invention.

[0115] In this embodiment, a method for obtaining vehicle orientation information is provided. As Figure 1 shown, the method for obtaining vehicle orientation information includes:

[0116] Collect an image of the vehicle to be detected;

[0117] It should be noted that the image of the vehicle to be detected is collected by devices such as cameras or cameras;

[0118] Obtaining the vehicle body bounding box, the front / rear bounding boxes, the wheel bounding boxes, the vehicle category, and the front / rear category information in the image of the vehicle to be detected;

[0119] It should be noted that the detection model stored in the memory performs inference on the vehicle image to be detected to obtain the vehicle body bounding box, front / rear bounding box, wheel bounding box, vehicle category, and front / rear category information;

[0120] It should be noted that if Figure 2 As shown, when the vehicle body bounding box and the front / rear bounding box are not obtained in the image of the vehicle to be detected, a first orientation angle calculation function deactivation flag is output.

[0121] Calculate a first heading angle of the vehicle to be detected and its uncertainty based on the vehicle category, the front / rear category, the vehicle body bounding box, and the front / rear bounding box information;

[0122] Calculating a second orientation angle of the vehicle to be detected and its uncertainty based on the wheel bounding box information;

[0123] Calculate a third orientation angle of the vehicle to be detected and its uncertainty by weighted calculation based on the first orientation angle of the vehicle to be detected, the uncertainty of the first orientation angle, the second orientation angle, and the uncertainty of the second orientation angle;

[0124] The third orientation angle of the vehicle to be detected and its uncertainty are input into the Kalman filter for updating to output the target orientation angle of the vehicle to be detected and its uncertainty.

[0125] Preferably, if Figure 2-4 As shown, the step of calculating the first orientation angle of the vehicle to be detected and its uncertainty based on the vehicle category, the front / rear category, the vehicle body bounding box, and the front / rear bounding box information further includes the following steps:

[0126] Obtaining the length and width information (405) of the vehicle to be detected based on the vehicle category information (401) and pre-stored size information (404) of each type of vehicle;

[0127] Calculating (406) a first orientation angle of the vehicle to be detected based on the length and width information (405) of the vehicle to be detected, the internal and external parameter information of the camera (403), the vehicle body bounding box, the front / rear bounding box, and the front / rear category information (402);

[0128] The first orientation angle of the vehicle to be detected and its uncertainty are output (407).

[0129] Preferably, if Figure 5As shown, the length and width information (405) of the vehicle to be detected includes the vehicle length L and width W;

[0130] Define l a as the x-axis pixel distance from the point on the bottom edge of the vehicle body bounding box or the front / rear vehicle body bounding box closest to the left side of the image to the image center point C;

[0131] Define l b as the x-axis pixel distance from the point on the bottom edge of the vehicle body bounding box or the front / rear vehicle body bounding box closest to the right side of the image to the image center point C;

[0132] Define l c as the x-axis pixel distance from the endpoint of the two bottom edges of the rear vehicle body bounding box that is closer to the center point of the vehicle body bounding box to the image center point C;

[0133] The camera internal parameter calibration information in the camera internal and external parameter information (403) is as follows:

[0134]

[0135] Among them, f x , f y is the camera pixel focal length, c x , c y is the optical center coordinate;

[0136] Define (x outer , y outer , w outer , h outer ) as the vehicle body bounding box information, and (x inner , y inner , w inner , h inner ) as the front / rear vehicle body bounding box information;

[0137] It is calculated that:

[0138] l a = x outer - c x

[0139] l b = x outer + w outer - c x

[0140] l c = x inner + w inner - c x

[0141] Define left_diff as the absolute difference in the x-axis pixel distance between the vehicle body bounding box and the front / rear bumper bounding box at the left bottom endpoint of the image, and right_diff as the absolute difference in the x-axis pixel distance between the vehicle body bounding box and the front / rear bumper bounding box at the right bottom endpoint of the image.

[0142] Preferably, if the calculated left_diff is greater than right_diff, then l c is calculated by the following formula:

[0143] l c = x inner - c x

[0144] According to the above definition and the calculated l a l b l c and the camera internal parameter f x the angle θ a θ b θ c is calculated as follows:

[0145]

[0146]

[0147]

[0148] Define the angle as the included angle formed by the perspective line formed by the left bottom point of the vehicle to be detected and the optical center and the rear bumper line or the vehicle body line; define the angle as the included angle formed by the perspective line formed by the right bottom point of the vehicle to be detected and the optical center and the vehicle body line or the rear bumper line;

[0149] It is derived from the sine law:

[0150]

[0151] Also, according to the theorem of the sum of interior angles of a quadrilateral, the angle and the angle and the angle θ a and θ b satisfy the following formula two:

[0152]

[0153] Substituting formula two into formula one, we can get:

[0154]

[0155] If left_diff is greater than right_diff, then after swapping W and L in Equation 3, that is, swapping the values of the preset length and width, calculate the angle according to Equation 3. Calculation;

[0156] After obtaining the angle The first orientation angle θ of the vehicle to be detected heading1 Can be calculated by the following Equation 4:

[0157]

[0158] Preferably, if the left_diff is greater than the right_diff, the first orientation angle θ of the vehicle to be detected obtained by calculation heading1 Is corrected by the following Equation 5:

[0159]

[0160] Preferably, if the head / tail surround frame category of the vehicle to be detected is the head, the first orientation angle θ of the vehicle to be detected obtained by calculation heading1 Is corrected by the following Equation 6:

[0161] θ heading1 = θ heading1 + π Equation 6

[0162] Preferably, the uncertainty of the first orientation angle of the vehicle to be detected is represented by Gaussian noise σ heading1 2 And is obtained by fitting and adjusting parameters with the true value, and is determined by the following Equation 7:

[0163] σ heading1 2 =(arctan(a * w inner )) 2 Equation 7

[0164] Where a is a parameter that can be adjusted according to the true value result.

[0165] Preferably, as Figure 6 Shown, in calculating the second orientation angle and its uncertainty of the vehicle to be detected according to the wheel surround frame information, the following steps are further included:

[0166] Step 601: Obtain the detection result of the wheel surround frame information;

[0167] Step 602: Sort the wheel surround frames according to the width or height of the wheel surround frame;

[0168] It should be noted that according to the phenomenon that the wheel surround frame is wider near and narrower far, sort by the width of the wheel surround frame;

[0169] Step 603: Select the two wheel bounding boxes with the earliest and latest sorting positions from the wheel sorting list of the vehicle to be detected;

[0170] Step 604: Calculate the coordinates (x wheel1 , y wheel1 ) and (x wheel2 , y wheel2 ) of the midpoints of the bottom sides of the two wheel bounding boxes with the earliest and latest sorting positions;

[0171] Step 606: Combine the internal and external camera parameter information 605 to project the coordinates of the midpoints of the bottom sides of the two wheel bounding boxes with the earliest and latest sorting positions back to the camera coordinate system;

[0172] Step 608: Calculate the second orientation angle and its uncertainty of the vehicle to be detected according to the information of the two wheel bounding boxes projected back to the camera coordinate system and the head / tail category information;

[0173] Step 609: Output the second orientation angle and its uncertainty of the vehicle to be detected.

[0174] Preferably, let (u, v) be the pixel coordinates to be calculated, and (x c , y c , z c ) be the coordinates of this pixel projected back to the camera coordinate system; then according to the camera model and the internal and external camera parameter information we can get:

[0175]

[0176] Get Get

[0177] Substitute the coordinates (x wheel1 , y wheel1 ) and (x wheel2 , y wheel2 ) of the midpoints of the bottom sides of the two wheel bounding boxes into u and v values respectively, and assume that the heights of the two wheel bounding boxes are the same, then we get and The angle can be obtained by the arctangent function:

[0178]

[0179] Then the second orientation angle θ heading2 of the vehicle to be detected is:

[0180] θ heading2 = φ wheel

[0181] Optionally, when the category of the front / rear bumper frame is the front of the vehicle, the second orientation angle θ of the vehicle to be detected heading2 is corrected by the following formula:

[0182] θ heading2 = π + φ wheel

[0183] wherein, the uncertainty of the second orientation angle of the vehicle to be detected is described by Gaussian noise σ heading2 2 σ heading2 2 is determined by the following formula:

[0184] σ heading2 2 = (arctan(b * w inner + c * abs(x inner - c x ))) 2

[0185] wherein, b and c are adjustable parameters.

[0186] Preferably, as Figure 7 shown, calculating the third orientation angle and its uncertainty of the vehicle to be detected by weighted calculation according to the first orientation angle, the uncertainty of the first orientation angle, the second orientation angle and the uncertainty of the second orientation angle of the vehicle to be detected further includes the following steps:

[0187] Step 704: Perform weighted summation based on the first orientation angle and its uncertainty information 701, the second orientation angle and its uncertainty information 702 of the vehicle to be detected and the weighting coefficient 703 stored on the storage device to obtain the third orientation angle and its uncertainty of the vehicle to be detected;

[0188] Step 705: Output the third orientation angle and its uncertainty of the vehicle to be detected;

[0189] wherein, the weighting coefficient 703 is composed of an orientation angle weighting coefficient α and an uncertainty weighting coefficient β, and the initial values of the orientation angle weighting coefficient α and the uncertainty weighting coefficient β of the orientation angle are calculated according to the following formula:

[0190]

[0191] wherein, the weighting coefficients α and β will be updated in the update module of the subsequent Kalman filter module and stored on the storage device;

[0192] wherein, the third orientation angle θ heading3 of the vehicle to be detected obtained by weighted summation and its uncertainty σ heading32 It is obtained from the following formula:

[0193] θ heading3 =αθ heading1 +(1 - α)θ heading2

[0194] σ heading3 2 =βσ heading1 2 +(1 - β)σ heading2 2 .

[0195] Preferably, as Figure 8 shown, the third orientation angle of the vehicle to be detected and its uncertainty 801 are input into the Kalman filter prediction module 804, and the state space of the Kalman filter prediction module 804 is:

[0196]

[0197] Transfer function:

[0198]

[0199] Among them, the posterior vehicle orientation angle θ of the vehicle to be detected is output after passing through the Kalman filter update module 805 kf and σ representing the uncertainty of θ kf 807; if there is no measurement value, the output is the predicted value of the Kalman filter. kf 2 In addition, the posterior vehicle orientation angle θ

[0200] and σ representing the uncertainty of θ kf are used to update the weighting coefficients 806. The update formula for the orientation angle weighting coefficient α: kf σ kf 2 The update formula for the orientation angle uncertainty weighting coefficient β:

[0201]

[0202] Among them, the updated weighting coefficients are stored in the storage device.

[0203]

[0204] Preferably, as

[0205] shown, the working process of the external parameter correction module for the target vehicle includes: Figure 9

[0206] ​Since the orientation of the target vehicle is defined as the orientation relationship between the target vehicle and the vehicle or other device on which the camera is installed, therefore, the output of the orientation information needs to be corrected by the yaw angle in the extrinsic camera information 901. The yaw angle φ of the camera relative to the vehicle on which it is installed can be obtained from the extrinsic camera information 901 yaw .

[0207] The target orientation angle and its uncertainty 902 of the vehicle to be detected are corrected 903 according to the extrinsic camera information 901; among them, the corrected yaw angle φ yaw is calculated by the following formula:

[0208] θ final = θ kf - φ yaw

[0209] σ final 2 = σ kf 2

[0210] The corrected target orientation angle and its uncertainty 904 of the vehicle to be detected are output.

[0211] As another embodiment of the present invention, a device for obtaining vehicle orientation information is provided, including:

[0212] An acquisition module for acquiring an image of the vehicle to be detected;

[0213] An acquisition module for obtaining the body bounding box, the head / tail bounding box, the wheel bounding box, the vehicle category, and the head / tail category information in the image of the vehicle to be detected;

[0214] A first calculation module for calculating a first orientation angle and its uncertainty of the vehicle to be detected according to the vehicle category, the head / tail category, the body bounding box, and the head / tail bounding box information;

[0215] A second calculation module for calculating a second orientation angle and its uncertainty of the vehicle to be detected according to the wheel bounding box information;

[0216] A third calculation module for calculating a third orientation angle and its uncertainty of the vehicle to be detected by weighted calculation according to the first orientation angle of the vehicle to be detected, the uncertainty of the first orientation angle, the second orientation angle, and the uncertainty of the second orientation angle;

[0217] A Kalman filter for updating the third orientation angle and its uncertainty of the vehicle to be detected to output the target orientation angle and its uncertainty of the vehicle to be detected.

[0218] An embodiment of the present invention also provides a non-transitory computer storage medium, which stores computer-executable instructions that can execute the method for obtaining vehicle orientation information in any of the above method embodiments.

[0219] Among them, the storage medium can be a magnetic disk, an optical disc, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above types of memories.

[0220] The above are only the preferred embodiments of the present invention, and do not impose any form of limitation on the present invention. Although the present invention has been disclosed above with preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some changes or modifications to equivalent embodiments by using the disclosed technical content within the scope of the technical solution of the present invention. However, as long as it does not depart from the content of the technical solution of the present invention, any simple modification, equivalent change, and modification made to the above embodiments based on the technical essence of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A method for obtaining vehicle orientation information, characterized in that, It includes the following steps: Collect the images of the vehicle to be detected; Obtain the body bounding box, the front / rear bounding box, the wheel bounding box, the vehicle category, and the front / rear category information in the images of the vehicle to be detected; Calculate the first orientation angle and its uncertainty of the vehicle to be detected based on the vehicle category, the front / rear category, the body bounding box, and the front / rear bounding box information; Calculate the second orientation angle and its uncertainty of the vehicle to be detected based on the wheel bounding box information; Weightedly calculate the third orientation angle and its uncertainty of the vehicle to be detected based on the first orientation angle, the uncertainty of the first orientation angle, the second orientation angle, and the uncertainty of the second orientation angle of the vehicle to be detected; Input the third orientation angle and its uncertainty of the vehicle to be detected into the Kalman filter for updating to output the target orientation angle and its uncertainty of the vehicle to be detected; Among them, inputting the third orientation angle and its uncertainty of the vehicle to be detected into the Kalman filter for updating to output the target orientation angle and its uncertainty of the vehicle to be detected further includes: The state space of the Kalman filter is: Transfer function: Among them, the posterior vehicle orientation angle θ of the vehicle to be detected is output by the Kalman filter update module kf and the σ kf characterizing the uncertainty of θ kf 2 ; The posterior vehicle orientation angle θ kf and the σ kf representing the uncertainty of θ kf 2 are used to update the weighting coefficient. The update formula for the orientation angle weighting coefficient α is: Among them, θ heading1 is the first orientation angle, and θ heading2 is the second orientation angle; Update formula for the weighting coefficient β of the orientation angle uncertainty: Among them, σ heading1 2 is the first Gaussian noise, and σ heading2 2 is the second Gaussian noise; Among them, the updated weighting coefficient is stored in the storage device.

2. The method for obtaining vehicle orientation information according to claim 1, wherein In calculating the first orientation angle and its uncertainty of the vehicle to be detected based on the vehicle category, the front / rear category, the body bounding box, and the front / rear bounding box information, it further includes the following steps: Obtain the length and width information (405) of the vehicle to be detected according to the vehicle category information (401) and the pre-stored vehicle size information (404) of each type; Calculate the first orientation angle of the vehicle to be detected (406) according to the length and width information (405) of the vehicle to be detected, the camera internal and external parameter information (403), the body bounding box, the front / rear bounding box, and the front / rear category information (402); Output the first orientation angle and its uncertainty of the vehicle to be detected (407).

3. The method for obtaining vehicle orientation information according to claim 2, wherein The length and width information (405) of the vehicle to be detected includes the vehicle length L and the width W; Define l a as the x-axis pixel distance from the point on the bottom edge of the vehicle body enclosure or the front / rear enclosure closest to the left side of the image to the image center point C; Define l b as the x-axis pixel distance from the point on the bottom edge of the body surround frame or the front / rear surround frame that is closest to the right side of the image to the image center point C; Define l c as the x-axis pixel distance from the center point C of the image to the one of the two bottom endpoints of the rear bumper frame that is closer to the center point of the vehicle body bumper frame; The camera internal parameter calibration information in the camera internal and external parameter information (403) is as follows: Among them, f x , f y is the focal length of the camera pixel, c x , c y is the optical center coordinate; Define (x outer , y outer , w outer , h outer ) as the body surround frame information, and (x inner， y inner , w inner , h inner ) as the front / rear surround frame information; Calculate to obtain: l a = x outer - c x l b = x outer + w outer - c x l c = x inner + w inner - c x Define left_diff as the absolute difference of the x-axis pixel distances of the body bounding box and the front / rear bounding box at the left bottom endpoint of the image, and right_diff as the absolute difference of the x-axis pixel distances of the body bounding box and the front / rear bounding box at the right bottom endpoint of the image; If the calculated left_diff is greater than right_diff, then l c is calculated by the following formula: l c = x inner - c x l obtained according to the above definition and calculation a , l b , l c and the internal camera parameter f x , the angle θ a , θ b , θ c is calculated as follows: Define the angle as the included angle formed by the perspective line formed by the left bottom point of the vehicle to be detected and the optical center and the rear end line or the vehicle body line; Define the angle as the included angle formed by the perspective line formed by the right bottom point of the vehicle to be detected and the optical center and the vehicle body line or the rear end line; Derive from the sine law: Also, according to the theorem of the interior angles of a quadrilateral, the said angle and the angle and the said angle θ a and θ b satisfy the following formula (2): Substitute the second formula into the first formula, then we can obtain: If left_diff is greater than right_diff, after swapping W and L in Equation 3, that is, swapping the values of the preset length and width, calculate the angle according to Equation 3. Calculation; Obtain the angle After that, the first orientation angle θ of the vehicle to be detected heading1 Can be calculated by the following formula (4): If the left_diff is greater than the right_diff, the first orientation angle θ of the vehicle to be detected obtained by calculation heading1 is corrected by the following formula five: If the category of the front / rear bumper frame of the vehicle to be detected is the front, the first orientation angle θ of the vehicle to be detected obtained by calculation heading1 is corrected by the following formula (6): θ heading1 = θ heading1 + π Equation 6 Among them, the uncertainty of the first orientation angle of the vehicle to be detected is represented by Gaussian noise σ heading1 2 which is obtained by fitting and adjusting parameters with the true value and is determined as follows in Equation VII: σ heading1 2 = (arctan(a*w inner )) 2 Equation Seven Among them, a is a parameter that can be calibrated according to the true value result.

4. The method for obtaining vehicle orientation information according to claim 1, wherein In calculating the second orientation angle and its uncertainty of the vehicle to be detected based on the wheel bounding box information, it further includes the following steps: Step 601: Obtain the detection result of the wheel bounding box information; Step 602: Sort the wheel bounding boxes according to the width or height of the wheel bounding box; Step 603: Select the two wheel bounding boxes that are the most forward and the most backward in the sorting; Step 604: Calculate the midpoint coordinates of the bottom edges of the two wheel bounding boxes with the highest and lowest rankings; Step 606: Combine the internal and external camera parameter information to project the midpoint coordinates of the bottom edges of the two wheel bounding boxes with the highest and lowest rankings back to the camera coordinate system; Step 608: Calculate the second orientation angle and its uncertainty of the vehicle to be detected based on the information of the two wheel bounding boxes projected back to the camera coordinate system and the head / tail category information; Step 609: Output the second orientation angle and its uncertainty of the vehicle to be detected.

5. The method for obtaining vehicle orientation information according to claim 4, characterized in that, Let (u, v) be the pixel coordinates to be calculated, and (x c , y c , z c ) be the coordinates of the pixel projected back into the camera coordinate system. Then, according to the camera model and the internal and external camera parameters information we can obtain: Obtain Obtain The midpoint coordinates (x wheel1 , y wheel1 ) and (x wheel2 , y wheel2 ) of the bottom edges of the two wheel enclosing frames are respectively substituted as u and v values, and it is set that the heights of the two wheel enclosing frames are the same, thus obtaining and The angle can be obtained from the arctangent function: Then the second orientation angle θ of the vehicle to be detected heading2 : θ heading2 = φ wheel When the category of the front / rear bumper frame is the front, the second orientation angle θ of the vehicle to be detected heading2 is corrected by the following formula: θ heading2 = π + φ wheel Among them, the uncertainty of the second orientation angle of the vehicle to be detected is represented by Gaussian noise σ heading2 2 described, and σ heading2 2 is determined by the following formula: σ heading2 2 = (arctan(b*w inner + c*abs(x inner - c x ))) 2 Wherein, b and c are adjustable parameters.

6. The method for obtaining vehicle orientation information according to claim 1, wherein The step of calculating the third orientation angle and its uncertainty of the vehicle to be detected by weighted calculation according to the first orientation angle, the uncertainty of the first orientation angle, the second orientation angle, and the uncertainty of the second orientation angle of the vehicle to be detected further includes the following steps: Step 704: Perform weighted summation based on the first orientation angle and its uncertainty information (701), the second orientation angle and its uncertainty information (702) of the vehicle to be detected, and the weighting coefficients (703) stored in the storage device to obtain the third orientation angle and its uncertainty of the vehicle to be detected; Step 705: Output the third orientation angle and its uncertainty of the vehicle to be detected; Wherein, the weighting coefficient (703) consists of an orientation angle weighting coefficient α and an uncertainty weighting coefficient β, and the initial value of the orientation angle weighting coefficient α and the initial value of the uncertainty weighting coefficient β of the orientation angle are calculated according to the following formula: Among them, the third orientation angle θ of the vehicle to be detected obtained by weighted summation h e ading3 and its uncertainty σ heading3 2 are obtained by the following formula: θ heading3 = αθ heading1 + (1 - α)θ heading2 σ heading3 2 = βσ heading1 2 + (1 - β)σ heading2 2 。 7. The method for obtaining vehicle orientation information according to claim 1, wherein Further included: Correct the target orientation angle and its uncertainty of the vehicle to be detected according to the external camera parameters; among them, the corrected yaw angle φ yaw is calculated according to the following formula: θ final = θ kf - φ yaw σ final 2 = σ kf 2 Output the target orientation angle and its uncertainty of the vehicle to be detected after correction.

8. An apparatus for obtaining vehicle orientation information, which is used to implement the method for obtaining vehicle orientation information according to any one of claims 1-7, characterized in that, The device for obtaining vehicle orientation information includes: An acquisition module for acquiring an image of the vehicle to be detected; An acquisition module for obtaining the body bounding box, head / tail bounding box, wheel bounding box, vehicle category, and head / tail category information in the image of the vehicle to be detected; A first calculation module for calculating the first orientation angle and its uncertainty of the vehicle to be detected based on the vehicle category, head / tail category, body bounding box, and head / tail bounding box information; A second calculation module for calculating the second orientation angle and its uncertainty of the vehicle to be detected based on the wheel bounding box information; A third calculation module for calculating the third orientation angle and its uncertainty of the vehicle to be detected by weighted calculation according to the first orientation angle, the uncertainty of the first orientation angle, the second orientation angle, and the uncertainty of the second orientation angle of the vehicle to be detected; A Kalman filter for updating the third orientation angle and its uncertainty of the vehicle to be detected to output the target orientation angle and its uncertainty of the vehicle to be detected.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions, and the computer instructions are used to cause the computer to execute the steps of the method according to any one of claims 1-7.

Images