Vehicle body relative posture calculation method, device, vehicle, equipment and storage medium

Abstract

The present invention discloses a method, device, vehicle, equipment and storage medium for calculating the relative posture of a vehicle body, wherein the method comprises: obtaining a current frame image and a previous frame image collected by an on-board sensing device; determining the vanishing point of the current frame image and the vanishing point of the previous frame image; constructing a first region of interest based on the vanishing point of the current frame image, and constructing a second region of interest based on the vanishing point of the previous frame image; extracting a first feature point in the first region of interest, and extracting a second feature point in the second region of interest; matching the first feature point with the second feature point to obtain k matching point pairs; judging whether the number k of matching point pairs is greater than a first threshold; if so, calculating the relative posture of the vehicle body between the current frame image and the previous frame image based on the matching point pairs. The present invention can quickly calculate the relative posture of the vehicle body between consecutive image frames.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a method, device, vehicle, equipment and storage medium for calculating the relative posture of a vehicle body. Background Art

[0002] With the continuous development of sensor technology and related scientific and technological technologies, it provides a strong guarantee for the realization of autonomous driving. However, the development of autonomous driving is inseparable from an important goal, that is, how to efficiently ensure vehicle safety, which means that the autonomous driving vehicle cannot collide with anything during driving, and the vehicle needs to measure the surrounding obstacles in real time.

[0003] Figure 1 This is the working principle diagram of the monocular vision system detecting vehicles provided by the existing technology. Please refer to Figure 1 (a) When pitch does not exist, by constructing a triangular relationship The absolute pitch angle of online (offline) calibration is α; please refer to Figure 1 (b) When pitch exists, the relative attitude pitch angle β is added for compensation. Relative refers to the pitch angle from time t-1 to time t.

[0004] In order for a monocular camera to accurately measure distance using similar triangulation, in addition to the absolute external parameter attitude angle, a dynamic attitude angle is also required to compensate for the angle change. Existing methods for compensating the dynamic attitude angle at connected moments include a method of calculating the attitude using a vehicle-mounted inertial measurement unit (IMU) and a method of continuously iterating the entire image to find the optimal model and then using the optimal model to restore the posture.

[0005] Figure 2 This is the working principle diagram of the IMU provided by the existing technology, please refer to Figure 2 Generally, an IMU is equipped with a three-axis gyroscope and three-directional accelerometers to measure the acceleration and angular velocity in three-dimensional space, and use this to calculate the relative attitude angle of the vehicle. However, the gyroscope in the IMU has a zero-point drift phenomenon, and the cumulative integration of the gyroscope will produce a relatively large cumulative error. As the vehicle runs for a long time, the distance error calculated by the IMU will also increase.

[0006] The method of iterating the entire image to find the optimal model generally requires thousands of RANSAC iterations for the entire image to solve the homography matrix and the basic matrix respectively to select the optimal model. For most vehicle-mounted embedded devices, the data processing capability is weak. Such multiple iterations will consume a lot of time and it is impossible to quickly solve the posture to compensate for the measured distance.

[0007] Therefore, it is necessary to provide a method that can quickly calculate the relative posture of the vehicle body between consecutive image frames to compensate for the connected moments. Summary of the invention

[0008] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the first aspect of the present invention proposes a method for calculating the relative posture of a vehicle body, comprising:

[0009] Obtain the current frame image and the previous frame image collected by the vehicle-mounted perception device;

[0010] Determining a vanishing point of the current frame image and a vanishing point of the previous frame image;

[0011] Constructing a first region of interest based on the vanishing point of the current frame image, and constructing a second region of interest based on the vanishing point of the previous frame image;

[0012] Extracting a first feature point in the first region of interest, and extracting a second feature point in the second region of interest;

[0013] Matching the first feature point with the second feature point to obtain k matching point pairs, where k is an integer;

[0014] Determine whether the number k of the matching point pairs is greater than a first threshold;

[0015] If so, the relative posture of the vehicle body between the current frame image and the previous frame image is calculated based on the matching point pairs.

[0016] Further, the calculating the relative posture of the vehicle body between the current frame image and the previous frame image based on the matching point pair includes:

[0017] Calculating a first homography matrix between the current frame image and the previous frame image according to the k matching point pairs;

[0018] Calculating projection errors of k matching point pairs and a mean value of the projection errors based on the first homography matrix;

[0019] Determining whether the mean of the projection errors is greater than a second threshold;

[0020] If not, the relative posture of the vehicle body between the current frame image and the previous frame image is calculated based on the projection error and a pinhole camera model.

[0021] Furthermore, before calculating the relative posture of the vehicle body between the current frame image and the previous frame image based on the projection error and the pinhole camera model, it also includes:

[0022] Screening out m smallest projection errors from the k projection errors, and determining the matching point pairs corresponding to the m smallest projection errors according to the screening results; wherein m is less than k and m is not less than 4;

[0023] Calculating a second homography matrix between the current frame image and the previous frame image based on the m matching point pairs;

[0024] Projecting the second feature point onto the current frame image by using the second homography matrix to obtain second homogeneous coordinates;

[0025] The first feature point coordinates and the second homogeneous coordinates are differentially calculated to obtain projection errors of the m matching point pairs.

[0026] Further, the calculating the projection errors of the k matching point pairs and the mean of the projection errors based on the first homography matrix includes:

[0027] Projecting the second feature point onto the current frame image through the first homography matrix to obtain a first homogeneous coordinate;

[0028] Performing differential calculation on the first feature point coordinates and the first homogeneous coordinates to obtain a projection error of the matching point pair;

[0029] The mean value of the projection error is calculated according to the projection error of the matching point pair and the number k of the matching point pairs.

[0030] Further, the calculating the relative posture of the vehicle body between the current frame image and the previous frame image based on the projection error and the pinhole camera model includes:

[0031] Calculating a mean value of the projection error and a minimum value of the projection error;

[0032] Determining whether an absolute value of a difference between a mean value of the projection error and a minimum value of the projection error is less than a third threshold;

[0033] If yes, substitute the minimum error value into the pinhole camera model to calculate the relative posture of the vehicle body between the current frame image and the previous frame image;

[0034] If not, the error mean is substituted into the pinhole camera model to calculate the relative posture of the vehicle body between the current frame image and the previous frame image.

[0035] Furthermore, after determining the vanishing point of the current frame image and the vanishing point of the previous frame image, the method further includes:

[0036] Obtaining the confidence of the vanishing point of the current frame image;

[0037] Determining whether the confidence of the vanishing point is greater than a preset confidence threshold;

[0038] If yes, turn to the step of constructing a first region of interest with the vanishing point of the current frame image as the center, and constructing a second region of interest with the vanishing point of the previous frame image as the center;

[0039] If not, go to the step of determining the vanishing point of the current frame image and the vanishing point of the previous frame image.

[0040] A second aspect of the present invention provides a vehicle body relative posture calculation device, comprising:

[0041] An image acquisition module is used to acquire the current frame image and the previous frame image collected by the vehicle-mounted sensing device;

[0042] A vanishing point determination module, used to determine the vanishing point of the current frame image and the vanishing point of the previous frame image;

[0043] An area of ​​interest construction module, configured to construct a first area of ​​interest based on a vanishing point of the current frame image, and to construct a second area of ​​interest based on a vanishing point of the previous frame image;

[0044] A feature point extraction module, used to extract first feature points in the first region of interest and second feature points in the second region of interest;

[0045] A feature point matching module, used to match the first feature point with the second feature point to obtain k matching point pairs; wherein k is an integer;

[0046] A first judging module, used to judge whether the number k of the matching point pairs is greater than a first threshold;

[0047] The vehicle body relative posture calculation module is used to calculate the vehicle body relative posture between the current frame image and the previous frame image based on the matching point pairs when the number k of the matching point pairs is greater than a first threshold.

[0048] A third aspect of the present invention provides a vehicle, wherein the vehicle has the vehicle body relative posture calculation device as described in the second aspect of the present invention.

[0049] A fourth aspect of the present invention proposes an electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method for calculating the relative posture of the vehicle body as described in the first aspect of the present invention.

[0050] The fifth aspect of the present invention proposes a computer-readable storage medium, which stores at least one instruction, at least one program, a code set or an instruction set. The at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by a processor to implement the method for calculating the relative posture of the vehicle body as described in the first aspect of the present invention.

[0051] The implementation of the present invention has the following beneficial effects:

[0052] (1) The embodiment of the present invention only needs to take feature points within the plane of the virtual infinite rectangular frame, and does not need to take feature points of the entire image for calculation. Compared with the solution of taking feature points of the entire image for calculation, the number of feature points is reduced, thereby reducing the amount of calculation and processing time, and can quickly calculate the relative posture of the vehicle body between consecutive image frames.

[0053] (2) The embodiment of the present invention only needs to calculate the homography matrix twice at most, avoiding multiple RANSAC iterations to obtain the homography matrix (or basic matrix). Compared with the solution of multiple RANSAC iterations to obtain the homography matrix, the number of iterations is reduced, thereby reducing the amount of calculation and processing time for calculating the ideal model, and can quickly calculate the relative posture of the vehicle body between consecutive image frames.

[0054] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 It is a working principle diagram of a monocular vision system detecting a vehicle provided by the prior art;

[0057] Figure 2 It is a working principle diagram of the IMU provided by the prior art;

[0058] Figure 3 is a flow chart of a method for calculating relative posture of a vehicle body provided by an embodiment of the present invention;

[0059] Figure 4 is a schematic diagram of the positional relationship of matching point pairs provided by an embodiment of the present invention;

[0060] Figure 5 is a schematic diagram of feature point matching results provided by an embodiment of the present invention;

[0061] Figure 6 is a flow chart of step S107 provided by an embodiment of the present invention;

[0062] Figure 7 is another flow chart of step S107 provided by an embodiment of the present invention;

[0063] Figure 8 is a schematic diagram of coordinate relationships of a pinhole camera provided by an embodiment of the present invention;

[0064] Fig. 9 is a comparison diagram of relative attitude angle calculation results provided by an embodiment of the present invention;

[0065] Fig.10 It is a structural block diagram of a vehicle body relative posture calculation device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Examples of the embodiments are shown in the drawings, where the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions.

[0067] It should be noted that the terminals involved in the embodiments of the present invention may include but are not limited to mobile phones, personal digital assistants (PDAs), wireless handheld devices, tablet computers, personal computers (PCs), MP3 players, MP4 players, wearable devices (for example, smart glasses, smart watches, smart bracelets, etc.), etc.

[0068] In addition, the term "and / or" in this article is only a description of the association relationship between the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0069] Example

[0070] Figure 3It is a flow chart of a method for calculating the relative posture of a vehicle body provided by an embodiment of the present invention. This specification provides method operation steps as described in the embodiment or flow chart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or server product is executed, it can be executed in sequence or in parallel according to the method shown in the embodiment or the accompanying drawings (for example, in a parallel processor or multi-threaded processing environment). Figure 3 The method for calculating the relative posture of the vehicle body shown is applied to the vehicle side and can also be used on the server side. Figure 3 As shown, the method may include the following steps:

[0071] S101: Acquire a current frame image and a previous frame image collected by an on-board sensing device;

[0072] Specifically, the vehicle-mounted sensing device can be a monocular vision camera or other sensing devices with monocular vision-based image acquisition functions; the vehicle-mounted sensing device is arranged at a position on the vehicle where it can obtain images of the scene in front of the vehicle or the scene behind the vehicle, including but not limited to the inside / outside of the front windshield that does not affect the driver's line of sight, above the hood, above the luggage compartment, etc.

[0073] Specifically, the current frame image and the previous frame image are continuous frame images acquired by the vehicle-mounted sensing device. The current frame image is the image acquired by the vehicle-mounted sensing device at the current moment, and the previous frame image is the image acquired by the vehicle-mounted sensing device at the moment before the current moment.

[0074] Specifically, the scene in front of the vehicle may include lane lines of the current lane, lane lines of adjacent lanes, and road ancillary facilities and equipment regularly arranged along the extension direction of the road. Road ancillary facilities and equipment may be roadside guardrails, curbstones, street lights, etc. Other road ancillary facilities and equipment with highly consistent shapes may also be used in this article according to actual conditions.

[0075] S102: Determine the vanishing point of the current frame image and the vanishing point of the previous frame image;

[0076] Vanishing point is a concept in focal perspective, which refers to the straight lines that are not parallel to the picture. They will converge into the distance until they disappear. This focal point is the vanishing point.

[0077] The image captured by the camera is segmented. The lane lines and road ancillary facilities in the image segmentation result can be used to extract or construct some auxiliary lines. These auxiliary lines are parallel to each other and are not parallel to the picture in the image captured by the camera. These auxiliary lines converge in the direction of the vehicle until they disappear. The concentration point of these auxiliary lines is the vanishing point. Multiple auxiliary lines intersect at the circular point marked by the straight arrow near the middle lane. This circular point is the vanishing point of the image captured by the camera.

[0078] In one embodiment, after determining the vanishing point of the current frame image and the vanishing point of the previous frame image, the method further includes:

[0079] Get the confidence of the vanishing point of the current frame image;

[0080] Determine whether the confidence of the vanishing point is greater than a preset confidence threshold;

[0081] If yes, turn to the step of constructing a first region of interest with the vanishing point of the current frame image as the center, and constructing a second region of interest with the vanishing point of the previous frame image as the center;

[0082] If not, the process proceeds to the step of determining the vanishing point of the current frame image and the vanishing point of the previous frame image.

[0083] S103: constructing a first region of interest based on the vanishing point of the current frame image, and constructing a second region of interest based on the vanishing point of the previous frame image; wherein the first region of interest at least partially overlaps with the second region of interest;

[0084] Specifically, the first region of interest is the infinite plane of the current frame image, and the second region of interest is the infinite plane of the previous frame image.

[0085] When constructing a region of interest based on the vanishing point of an image, the relative position information between the vanishing point and the region of interest is also required, including:

[0086] Acquire first relative position information, wherein the first relative position information is relative position information between a vanishing point of the current frame image and a first region of interest;

[0087] Constructing a first region of interest based on the vanishing point of the current frame image and the first relative position information;

[0088] Acquire second relative position information, wherein the second relative position information is relative position information between the vanishing point of the current frame image and the second region of interest;

[0089] The second region of interest is constructed based on the vanishing point of the previous frame image and the second relative position information.

[0090] Specifically, the relative position information between the vanishing point and the region of interest includes but is not limited to the relative position of the vanishing point to a specific point in the region of interest. The specific point in the region of interest may be a center point, a corner point of the region of interest, a specific point of a border of the region of interest, etc. This embodiment is not limited to this.

[0091] Specifically, the first relative position information is the same as or different from the second relative position information; the shapes of the first region of interest and the second region of interest are the same as or different; and the sizes of the first region of interest and the second region of interest are the same as or different.

[0092] For example, the first region of interest is a box with the vanishing point as the lower left corner point, and the second region of interest is a circle with the vanishing point as the center;

[0093] For another example, the first region of interest is an ellipse centered at the vanishing point, and the second region of interest is an irregular polygon with the vanishing point as the midpoint of a specified side;

[0094] For another example, the first region of interest and the second region of interest are rectangles of the same size, the center point of the first region of interest is on the left side of the vanishing point and is at a first distance from the vanishing point, and the center point of the second region of interest is on the right side of the vanishing point and is at a second distance from the vanishing point;

[0095] For another example, the first region of interest is a rectangle centered at the vanishing point, the second region of interest is a rectangle centered at the vanishing point, and the first region of interest and the second region of interest have the same size of rectangles.

[0096] It should be noted that the above examples are only used for illustration, and there are other situations different from the above examples in actual applications.

[0097] S104: extracting first feature points in the first region of interest, and extracting second feature points in the second region of interest;

[0098] In the prior art, feature points are usually extracted from the entire image, but in the embodiments of the present invention, only feature points in the plane of the virtual infinite rectangular frame need to be taken. There is no need to extract feature points from the entire image, nor is there any need to perform subsequent calculations based on the feature points in the entire image, thereby reducing processing time.

[0099] Specifically, the first region of interest may include a plurality of first feature points, and the second region of interest may include a plurality of second feature points. Figure 4 is a schematic diagram of the position relationship of the matching point pairs provided in an embodiment of the present invention, such as Figure 4As shown, the first feature point is a point on the virtual infinite plane of the current frame image, and the second feature point is a point on the virtual infinite plane of the previous frame image. When the first feature point matches the second feature point, the projection of the first feature point on the infinite plane should coincide with the projection of the second feature point on the infinite plane. Therefore, after the feature points are extracted, the feature points can be matched according to the positional relationship of the mutually matching feature points.

[0100] S105: Match the first feature point with the second feature point to obtain k matching point pairs, where k is an integer;

[0101] Figure 5 is a schematic diagram of a feature point matching result provided by an embodiment of the present invention, such as Figure 5 As shown in the figure, the solid circle is the vanishing point, and a virtual infinite rectangular plane is constructed with the vanishing point as the center. m matching point pairs are output in the rectangular frame, wherein the virtual infinite rectangular plane is represented by a rectangular frame, and the matching point pairs in the rectangular frame are represented by circles. Feature points from one region of interest are filled with oblique lines, and feature points from another region of interest are not filled. The matching point pairs with the smallest projection error are represented by solid circles. In order to visually distinguish the vanishing point from the matching point pair, the area of ​​the illustrated vanishing point is larger than the area of ​​the matching point pair.

[0102] Specifically, the feature point matching method may be brute force matching, K nearest neighbor matching, cross matching, etc., but this embodiment is not limited thereto.

[0103] S106: Determine whether the number k of matching point pairs is greater than a first threshold; if so, execute the next step; if not, end the process; wherein the first threshold is not less than 4;

[0104] Calculating the relative camera pose between the current frame image and the previous frame image based on matching point pairs requires solving the homography matrix between the current frame image and the previous frame image. Since the homography matrix has a total of 8 degrees of freedom and each pair of matching points gives two constraints, at least 4 pairs of matching points are required to solve the set of equations, that is, the first threshold is not less than 4.

[0105] S107: Calculating the relative posture of the vehicle body between the current frame image and the previous frame image based on the matching point pairs;

[0106] Figure 6 is a flow chart of step S107 provided in an embodiment of the present invention, and specifically Figure 6 As shown, step S107 includes:

[0107] S1071: Calculate a first homography matrix between the current frame image and the previous frame image according to k matching point pairs;

[0108] In one embodiment, step S1071 includes:

[0109] Normalize the feature point coordinates of the k matching point pairs to obtain normalized feature point coordinates;

[0110] A first homography matrix is ​​calculated based on the normalized feature point coordinates.

[0111] S1072: Calculate projection errors of k matching point pairs and the mean of the projection errors based on the first homography matrix;

[0112] In one embodiment, step S1072 includes:

[0113] Projecting the second feature point onto the current frame image through the first homography matrix to obtain the first homogeneous coordinates;

[0114] The first feature point coordinates are differentially calculated with the first homogeneous coordinates to obtain the projection error of the matching point pair;

[0115] The mean of the projection error is calculated based on the projection error of the matching point pairs and the number k of matching point pairs.

[0116] S1073: Determine whether the mean value of the projection error is greater than a second threshold; if so, end the process; if not, proceed to the next step;

[0117] S1074: Calculate the relative posture of the vehicle body between the current frame image and the previous frame image based on the projection error and the pinhole camera model.

[0118] In one embodiment, step S1074 includes:

[0119] Calculate the mean value of the projection error and the minimum value of the projection error;

[0120] Determine whether the absolute value of the difference between the mean value of the projection error and the minimum value of the projection error is less than a third threshold;

[0121] If so, substitute the minimum error value into the pinhole camera model to calculate the relative pose of the vehicle body between the current frame image and the previous frame image;

[0122] If not, the error mean is substituted into the pinhole camera model to calculate the relative posture of the vehicle body between the current frame image and the previous frame image.

[0123] Specifically, calculating the relative posture of the vehicle body between the current frame image and the previous frame image may include:

[0124] Calculate the relative camera pose between the current frame image and the previous frame image based on the matching point pairs;

[0125] The relative position of the vehicle body between the current frame image and the previous frame image is determined according to the relative position of the camera.

[0126] Figure 7 is another flow chart of step S107 provided in an embodiment of the present invention, specifically as follows Figure 7 As shown, Figure 6 Compared with step S107, the illustrated embodiment further includes:

[0127] S1075: Filter out m smallest projection errors from the k projection errors, and determine matching point pairs corresponding to the m smallest projection errors according to the filter results; wherein m is less than k and m is not less than 4;

[0128] S1076: Calculating a second homography matrix between the current frame image and the previous frame image based on the m matching point pairs;

[0129] S1077: Projecting the second feature point onto the current frame image through a second homography matrix to obtain a second homogeneous coordinate;

[0130] S1078: Perform differential calculation on the first feature point coordinates and the second homogeneous coordinates to obtain projection errors of m matching point pairs.

[0131] In one embodiment, a fixed rectangular area is set with the vanishing point as the midpoint using the vanishing point calculated by the lane line. Assume that the infinite plane is π, the normal vector of the plane is n, the depth is d, and a point on the infinite plane of the previous frame image is p. l =(u l ,v l ), a point on the infinite plane of the current frame image is p c =(u c ,v c ), they are a pair of matching points, and the point sets on the plane of the two consecutive frames are first calculated using the direct linear transformation method to calculate the homography matrix H1:

[0132] The homography matrix is ​​usually used to describe the transformation relationship between some points on the same plane between two images. Assume that a pair of matching points p l and p c , these points fall on point P on plane π, and the result of the transpose of the normal vector multiplied by the point on this surface is equal to the distance from the origin of the coordinate system to the plane. Therefore, the equation of the plane with the vanishing point as the center and infinite distance can be set as

[0133] n T P+d=0 (1)

[0134] Slightly tidied up

[0135]

[0136] Assume that the intrinsic parameters of the camera are K, R, and t is the motion between the two cameras (if you like, you can also assume it is the motion between two consecutive frames of a camera)

[0137] p l =KP, p c =K(RP+t) (3)

[0138] In particular, if homogeneous coordinates p are used c =(u c ,v c ,1) and p l =(u l ,v l ,1); adding one less scale factor is also valid, and finally we can deduce:

[0139]

[0140] Then the homography matrix H1 is equal to:

[0141]

[0142] Abbreviations available:

[0143] p c =H1*p l (6)

[0144] Since the homography matrix is ​​a 3*3 matrix, we can get:

[0145]

[0146] Arranged to write a linear equation system

[0147]

[0148]

[0149] Rearranged

[0150] -h 00 u l -h 01 v l -h 02 +h 20 u l u c +h 21 v l u c +h 22 u c =0 (9)

[0151] -h 10 u l -h 11 v l -h 12 +h 20 ul v c +h 21 v l v c +h 22 v c =0 (10)

[0152] Rewritten in matrix form,

[0153]

[0154] The equation of AX=0 is obtained. Each pair of matching points gives two constraints. Since the homography matrix has a total of 8 degrees of freedom, at least 4 pairs of matching points are required to solve the equations. Assume that there are k matching points on the plane at infinity, satisfying the following equation:

[0155]

[0156] For the linear equation in the above formula, singular value decomposition (SVD) can be used to solve the homography matrix H1.

[0157] Project the point on the infinite plane of the previous frame onto the current frame through the homography matrix H1 to obtain the homogeneous coordinates p' c =(u' c , v' c , 1)

[0158] p' c =H1*p l (13)

[0159] and point p of the current frame c The difference can be obtained (also called projection error)

[0160] ΔU=|u c -u' c | (14)

[0161] ΔV=|v c -v' c | (15)

[0162] Arrange the projection error values ​​of k pairs of matching points from small to large and find the average error:

[0163]

[0164]

[0165] Take out m groups of matching points with the smallest projection error, and use formula (12) again to solve the homography matrix H2 for these m groups of matching points. Combine formulas (14), (15), (16), and (17) to get m groups of projection error values ​​and means, arrange them from small to large, and take out the matching point p with the smallest difference value. c_min =(u c_min ,v c_min ) and p l_min =(u l_min ,v l_min ), and find the minimum projection error value

[0166] ΔU min =u c_min -u l_min (15)

[0167] ΔV min =v c_min -v l_min (16)

[0168] Figure 8 is a schematic diagram of the coordinate relationship of the pinhole camera provided by an embodiment of the present invention, such as Figure 8 As shown, ΔU min Horizontal red line, ΔV min The vertical red line is the projection of the fx focal length in the x direction and the projection of the fy focal length in the y direction. According to the pinhole camera model, a triangular relationship can be constructed:

[0169]

[0170]

[0171] Specifically, the relative position of the vehicle body can be used for posture estimation at the back end, and the posture estimation result can be used as input information for autonomous driving assistance systems.

[0172] It should be noted that the present invention is not limited by the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously.

[0173] VO is the abbreviation of Visual Odometry, and its corresponding Chinese name is Visual Odometry. VP is the abbreviation of Vanish Point, and its corresponding Chinese name is Vanishing Point. Fig. 9 is a comparison diagram of the relative attitude angle calculation results provided by an embodiment of the present invention, such as Fig. 9As shown in the figure, the legend corresponding to VO is used to indicate the line type and color of the fluctuation curve of the relative pitch angle calculated based on VO, and the legend corresponding to VP is used to indicate the line type and color of the fluctuation curve of the relative pitch angle calculated based on VP. By comparing and observing, it can be seen that the relative attitude angle curve calculated based on VO and the relative attitude angle curve calculated based on VP have a high degree of overlap, and the change trends are roughly the same.

[0174] Fig.10 is a structural block diagram of a vehicle body relative posture calculation device provided by an embodiment of the present invention, specifically as follows Fig.10 As shown, the device may include:

[0175] The image acquisition module 201 is used to acquire the current frame image and the previous frame image collected by the vehicle-mounted sensing device;

[0176] A vanishing point determination module 202, used to determine the vanishing point of the current frame image and the vanishing point of the previous frame image;

[0177] An area of ​​interest construction module 203 is used to construct a first area of ​​interest based on the vanishing point of the current frame image, and to construct a second area of ​​interest based on the vanishing point of the previous frame image; wherein the first area of ​​interest and the second area of ​​interest have the same shape and size;

[0178] A feature point extraction module 204, configured to extract first feature points in the first region of interest and second feature points in the second region of interest;

[0179] The feature point matching module 205 is used to match the first feature point with the second feature point to obtain k matching point pairs, where k is an integer;

[0180] The first judgment module 206 is used to judge whether the number k of matching point pairs is greater than a first threshold; if so, execute the next step; if not, end the process; wherein the first threshold is not less than 4;

[0181] The vehicle body relative posture calculation module 207 is used to calculate the vehicle body relative posture between the current frame image and the previous frame image based on the matching point pairs.

[0182] Preferably, the device is integrated with a camera for photographing the scene in front of the vehicle.

[0183] The embodiment of the present invention further provides a vehicle having a vehicle body relative posture calculation device as described in the above device embodiment. The relative position of the vehicle body output by the vehicle body relative posture calculation device can be used for posture estimation of the back end, and the posture estimation result can be used as input information of the automatic driving assistance system. It should be noted that, without departing from the scope of the present disclosure, the vehicle of the present invention may be a truck, a sports utility vehicle, a van, a motorhome or any other type of vehicle.

[0184] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0185] An embodiment of the present invention also provides an electronic device, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, a code set or an instruction set is loaded and executed by the processor to implement a method for calculating the relative posture of a vehicle body as described in the method embodiment.

[0186] An embodiment of the present invention also provides a storage medium, which can be set in a server to store at least one instruction, at least one program, code set or instruction set related to the method for calculating the relative posture of the vehicle body in the method embodiment. The at least one instruction, the at least one program, the code set or instruction set is loaded and executed by the processor to implement the method for calculating the relative posture of the vehicle body provided in the above-mentioned method embodiment.

[0187] Optionally, in this embodiment, the storage medium may be located in at least one of the multiple network servers of the computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0188] It can be seen from the embodiments of the method, device, vehicle, equipment and storage medium for calculating a dynamic attitude angle provided by the present invention that the implementation of the present invention has the following beneficial effects:

[0189] (1) The embodiment of the present invention only needs to obtain feature points within the plane of the virtual infinite rectangular frame, and does not need to calculate feature points of the entire image, thereby reducing processing time.

[0190] (2) The embodiment of the present invention only needs to calculate the homography matrix twice at most, avoiding multiple RANSAC iterations to obtain the homography matrix (or basic matrix), thereby reducing the time for calculating the ideal model.

[0191] It should be noted that the sequence of the embodiments of the present invention described above is for description only and does not represent the advantages and disadvantages of the embodiments. The above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0192] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and server embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0193] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0194] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for calculating relative posture of a vehicle body, characterized in that: include: Obtain the current frame image and the previous frame image collected by the vehicle-mounted perception device; Determining a vanishing point of the current frame image and a vanishing point of the previous frame image; Constructing a first region of interest based on the vanishing point of the current frame image, and constructing a second region of interest based on the vanishing point of the previous frame image; Extracting a first feature point in the first region of interest, and extracting a second feature point in the second region of interest; Matching the first feature point with the second feature point to obtain k matching point pairs, where k is an integer; Determine whether the number k of the matching point pairs is greater than a first threshold; If so, calculate the first homography matrix between the current frame image and the previous frame image based on the k matching point pairs; calculate the projection errors of the k matching point pairs and the mean of the projection errors based on the first homography matrix; if the mean of the projection errors is not greater than the second threshold, screen out the m smallest projection errors from the k projection errors, and determine the matching point pairs corresponding to the m smallest projection errors based on the screening results; wherein m is less than k and m is not less than 4; calculate the second homography matrix between the current frame image and the previous frame image based on the m matching point pairs; project the second feature point onto the current frame image through the second homogeneous coordinates to obtain the second homogeneous coordinates; perform differential calculation on the first feature point coordinates and the second homogeneous coordinates to obtain the projection errors of the m matching point pairs; calculate the relative posture of the vehicle body between the current frame image and the previous frame image based on the projection errors and the pinhole camera model.

2. The method according to claim 1, characterized in that The calculating the relative posture of the vehicle body between the current frame image and the previous frame image based on the projection error and the pinhole camera model includes: Calculating a mean value of the projection error and a minimum value of the projection error; Determining whether an absolute value of a difference between a mean value of the projection error and a minimum value of the projection error is less than a third threshold; If yes, substitute the minimum error value into the pinhole camera model to calculate the relative posture of the vehicle body between the current frame image and the previous frame image; If not, the error mean is substituted into the pinhole camera model to calculate the relative posture of the vehicle body between the current frame image and the previous frame image.

3. The method according to claim 1, characterized in that: After determining the vanishing point of the current frame image and the vanishing point of the previous frame image, the method further includes: Obtaining the confidence of the vanishing point of the current frame image; Determining whether the confidence of the vanishing point is greater than a preset confidence threshold; If yes, turn to the step of constructing a first region of interest based on the vanishing point of the current frame image as the center, and constructing a second region of interest based on the vanishing point of the previous frame image as the center; If not, go to the step of determining the vanishing point of the current frame image and the vanishing point of the previous frame image.

4. A vehicle body relative posture calculation device, characterized in that: include: An image acquisition module is used to acquire the current frame image and the previous frame image collected by the vehicle-mounted sensing device; A vanishing point determination module, used to determine the vanishing point of the current frame image and the vanishing point of the previous frame image; An area of ​​interest construction module, configured to construct a first area of ​​interest based on a vanishing point of the current frame image, and to construct a second area of ​​interest based on a vanishing point of the previous frame image; A feature point extraction module, used to extract first feature points in the first region of interest and second feature points in the second region of interest; A feature point matching module, used to match the first feature point with the second feature point to obtain k matching point pairs; wherein k is an integer; A first judging module, used to judge whether the number k of the matching point pairs is greater than a first threshold; A vehicle body relative posture calculation module is used to calculate the first homography matrix between the current frame image and the previous frame image based on the k matching point pairs; calculate the projection errors of the k matching point pairs and the mean of the projection errors based on the first homography matrix; if the mean of the projection errors is not greater than a second threshold, screen out m smallest projection errors from the k projection errors, and determine the matching point pairs corresponding to the m smallest projection errors based on the screening results; wherein m is less than k and m is not less than 4; calculate the second homography matrix between the current frame image and the previous frame image based on the m matching point pairs; project the second feature point onto the current frame image through the second homogeneous coordinates to obtain the second homogeneous coordinates; perform differential calculation on the first feature point coordinates and the second homogeneous coordinates to obtain the projection errors of the m matching point pairs; calculate the vehicle body relative posture between the current frame image and the previous frame image based on the projection errors and a pinhole camera model.

5. A vehicle, characterized in that: The vehicle has the vehicle body relative posture calculation device according to claim 4.

6. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method for calculating the relative posture of the vehicle body as described in any one of claims 1 to 3.

7. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method for calculating the relative posture of the vehicle body as described in any one of claims 1-3.

Images