A lateral positioning method for an autonomous commercial vehicle

Abstract

The present invention discloses a method for lateral positioning of an autonomous commercial vehicle, which is a high-precision lateral positioning method for an autonomous commercial vehicle based on deep learning lane line detection, Tukey weighted least squares fitting, and real-time homography transformation. The method is based on highly robust deep learning lane line detection, uses the accurate lane line fitted by Tukey weighted least squares method, calculates the lateral positioning result of the vehicle on the optimal sampling plane through the homography transformation matrix between the camera and the sampling ground plane obtained by Gaussian sampling maximum likelihood solution, and finally uses the scale s to restore the lateral positioning result of the vehicle on the real ground plane, effectively reducing the lane line detection error caused by traditional methods and the positioning error caused by vehicle vibration and uneven ground, and realizing high-precision lateral positioning of the self-lane.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a lateral positioning method for an autonomous driving commercial vehicle based on homography transformation. Background Art

[0002] With the continuous development of automobile intelligence, high-precision positioning is becoming more and more important. Achieving high-precision positioning of smart cars is the premise for obtaining road traffic conditions and perceiving the road environment, which has a huge impact on smart car path planning and the decision-making of various systems. For autonomous vehicles driving on the road, one of the first and most important tasks is the positioning of the vehicle. To this end, the vehicle needs to be able to consider information from multiple sensors and fuse it with data from the road map. The lane-level positioning problem can be summarized into three main modules. The first is to infer the road the vehicle is currently traveling on. In fact, the global satellite navigation system itself is not accurate enough to infer this information, so a refined positioning step is required. The second step is to estimate the position of the vehicle in its lane, and finally to evaluate the lane the vehicle is currently traveling in. The last two parts are very necessary for safe driving. For some applications on smart cars, it is far from enough to know the road the vehicle is traveling on. These systems must be informed of the position of the main lane in the road and the specific position of the vehicle in the current driving lane to provide sufficient maneuvering instructions and keep the vehicle safe. Therefore, autonomous vehicle applications require more accurate positioning, which can be achieved by estimating the lateral and longitudinal positions of the vehicle in the ego lane.

[0003] For intelligent vehicle systems such as lane keeping, lane departure warning, automatic cruise control, and autonomous driving, the accuracy of system judgment and decision-making requires a high degree of accuracy in the lateral positioning of the vehicle. In practice, the positioning of autonomous vehicles can be achieved by locating the vehicle based on some visual features (such as lane markings or traffic signs). These visual landmarks can be detected using on-board sensors. At present, the lateral positioning of the vehicle in the current lane can be achieved by many methods, but most of them are based on pre-collected high-precision maps, GPS, or high-cost lidar, inertial navigation, etc., which cannot be achieved in some scenarios or under low-cost requirements. Low-cost solutions usually only use cameras, and use traditional methods such as edge detection + color threshold to detect lane lines through Hough transform, screen straight lines through slope, use general least squares method to fit lane lines, and finally complete lateral positioning through geometric model or transformation matrix calculation. On the one hand, traditional lane line detection methods are greatly affected by noise and lighting, and the results of general least squares fitting cannot meet the requirements of high-precision positioning, resulting in low positioning accuracy. On the other hand, when the car is driving, due to the influence of vehicle vibration or uneven ground, the positioning result calculated by the transformation matrix calibrated once is of low accuracy. Therefore, the external parameters of the camera need to be calibrated in real time, and the transformation matrix calibrated once cannot be used for calculation. Summary of the invention

[0004] In order to solve the deficiencies in the prior art, the present invention proposes a method for lateral positioning of an autonomous commercial vehicle, which is a high-precision lateral positioning method for an autonomous commercial vehicle based on deep learning lane line detection, Tukey weighted least squares fitting, and real-time homography transformation. The method is based on highly robust deep learning lane line detection, uses the precise lane line fitted by Tukey weighted least squares method, calculates the lateral positioning result of the vehicle on the optimal sampling plane through the homography transformation matrix between the camera and the sampling ground plane obtained by Gaussian sampling maximum likelihood solution, and finally uses the scale s to restore the lateral positioning result of the vehicle on the real ground plane, effectively reducing the lane line detection error caused by traditional methods and the positioning error caused by vehicle vibration and uneven ground, and achieving high-precision lateral positioning of the self-lane.

[0005] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0006] A method for lateral positioning of an autonomous driving commercial vehicle comprises the following steps:

[0007] S1, camera intrinsic parameter matrix calibration and camera installation angle calibration;

[0008] S2, using the monocular camera after completing the camera internal parameter matrix and the installation angle calibration to collect the front view image of the car when it is driving, and process the collected image;

[0009] Perform lane line detection on the image based on deep learning, and obtain and filter the point coordinates on the lane line where the current vehicle is located; perform linear fitting of Tukey weighted least squares on the filtered current lane line point coordinates to eliminate misdetected lane points;

[0010] S3. Based on the current lane line point coordinates filtered in S2, by uniformly sampling the plane normal in the Gaussian spherical cap and combining the internal camera parameters obtained by calibration, the homography matrix between the image plane and the sampling plane can be calculated; calculate the maximum likelihood according to the lane line constraint to find the optimal plane normal vector;

[0011] S4. Perform lateral positioning on the plane corresponding to the maximum likelihood and restore the real positioning result through the scale s.

[0012] Furthermore, the lane line detection method in S2 is as follows:

[0013] Take the processed image as the input of the deep learning network, use the deep learning network to detect the coordinate points of the lane line in the image, pre-classify the coordinate points according to the lane line markers in the output result of the deep learning network, and obtain the coordinate points on the lane line where the current vehicle is located.

[0014] Furthermore, the method for filtering the coordinate points on the lane line where the current vehicle is located is as follows:

[0015] For the coordinate points on the lane line where the current vehicle is located, first sort them in descending order of the distance from the coordinate points to the current vehicle, and select 50% of the coordinate points close to the current vehicle in the coordinate array for lateral positioning.

[0016] Furthermore, use the Tukey weighted least squares method to fit the filtered near-field points. The steps of Tukey weighted least squares are as follows:

[0017] Step (1). Set the weight ω of all points to 1, and perform standard least squares fitting to obtain an approximate straight line;

[0018] Step (2). Set a distance threshold τ, calculate the distance d from all points to the straight line in step (1). If d < τ, the weight of this point is If d > τ, the weight of this point is ω = 0;

[0019] Step (3). Perform weighted least squares fitting on all weighted points in step (2) to obtain a new straight line, and repeat the above steps until the most accurate straight line is obtained through multiple iterations of fitting;

[0020]

[0021] According to the fitted straight line, a suitable starting point and step size are selected to recalculate the coordinates of the near-field point, and based on this, the angle β between the center line of the current lane on the image and the midline of the image is calculated as the actual deviation angle of the vehicle during driving.

[0022] Furthermore, the method for finding the optimal plane normal vector in S3 is as follows:

[0023] After filtering out the lane line coordinate points in the image, the normalized plane normal is uniformly sampled on the Gaussian sphere cap. After obtaining the sampled normal, the sampling angle θ and The rotation matrix between the camera coordinate system and the sampling plane coordinate system is calculated, and the translation matrix is ​​calculated using the coordinates of the sampling normal vector endpoints. After completing the above steps, the homography matrix between the image plane and the sampling plane can be calculated in combination with the camera intrinsic parameters obtained by calibration, and the lane line coordinate points on the image are converted to the sampling plane. After that, the joint likelihood of the two plane constraints, namely the parallel constraint and the angle constraint, is calculated on each sampling plane. The maximum joint likelihood corresponds to the optimal plane normal vector.

[0024] Further, the method for performing lateral positioning in S4 is:

[0025] After finding the optimal plane normal vector, calculate the distance d between the two parallel lines fitted in step S3 on the optimal plane. * ;

[0026] The scale s is expressed as:

[0027]

[0028] Among them, d0 is the actual distance between the lane lines on the left and right sides of the road section;

[0029] The lane line equation obtained by fitting in the optimal plane in step S3 is used to calculate the distance between the origin of the plane coordinate system (0, 0) and the fitted lane line, and then the scale information is used to restore the distance between the camera and the actual left and right lane lines.

[0030]

[0031] Among them, a, b, and c are the parameters of the fitted lane line equation, and X and Y are the coordinate points projected from the camera center to the ground plane.

[0032] Furthermore, the checkerboard method is used to calibrate the camera's intrinsic parameter matrix.

[0033] Furthermore, the method for calibrating the camera intrinsic parameter matrix is:

[0034] The chessboard is placed flat on the ground, and the chessboard plane is taken as the origin by the detected chessboard corner point. The Zhang Zhengyou calibration method is used to measure multiple times, and the intrinsic parameter matrix K between the camera plane and the chessboard plane is calculated, which is expressed as:

[0035]

[0036] Among them, fx is the length of the camera focal length in the x-axis direction, fy is the length of the camera focal length in the y-axis direction, u0 is the horizontal pixel coordinate of the actual principal point of the image, and v0 is the vertical pixel coordinate of the actual principal point of the image.

[0037] Furthermore, the installation angle calibration method is: for a camera fixedly installed on a vehicle, a level is used to calibrate the installation angle α of the camera.

[0038] Furthermore, the image processing in S2 includes: video frame acquisition, de-distortion, cropping, image grayscale, and image format conversion.

[0039] Beneficial effects of the present invention:

[0040] 1. To ensure the robustness of lane line detection, a lane line detection algorithm based on deep learning is used. After network training, it can be applied to various scenarios. Compared with traditional detection methods, it has stronger applicability and higher detection accuracy.

[0041] 2. The optimal homography problem is transformed into a Gaussian sampling maximum likelihood problem, which effectively reduces the positioning error caused by vehicle vibration and uneven ground.

[0042] 3. Use the Tukey weighted least squares fitting method to effectively eliminate the misdetected lane coordinate points and accurately fit the detected lane lines, effectively reducing the positioning error caused by misdetection of lane line points. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 The overall flow chart of this method

[0044] Figure 2 Schematic diagram for chessboard placement

[0045] Figure 3 Schematic diagram of lane detection and screening

[0046] Figure 4 Schematic diagram of uniform sampling of plane normals

[0047] Figure 5 Schematic diagram of the relationship between the camera coordinate system and the sampling plane coordinate system

[0048] Figure 6 Schematic diagram of the conversion between image coordinates and sampling ground plane coordinates

[0049] Figure 7 Schematic diagram of the positional relationship between the camera coordinate system and the real ground plane coordinate system

[0050] Figure 8 Schematic diagram of scale recovery

[0051] Fig. 9 Schematic diagram of vehicle lateral positioning DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0053] The specific implementation of the present invention is further described below in conjunction with the accompanying drawings.

[0054] A lateral positioning method for an autonomous driving commercial vehicle proposed in the present invention comprises the following steps:

[0055] S1, camera intrinsic parameter matrix calibration and camera installation angle calibration.

[0056] More specifically, the intrinsic parameter matrix of the camera is calibrated using the chessboard method in S1. First, the placement position of the chessboard is reasonably selected. In this embodiment, the chessboard is placed flat on the ground at different positions, and the camera is used to take multiple shots.

[0057] The chessboard plane uses the detected chessboard corner points as the origin. When calibrating using the chessboard calibration method, the chessboard is partially placed on the ground as shown in the following figure. Figure 2 As shown in the figure, ①②③ represent different placement positions respectively. The camera intrinsic parameter matrix K can be calculated by Zhang Zhengyou method. Formula (1) represents the camera intrinsic parameter matrix obtained by calibration, which is expressed as:

[0058]

[0059] Among them, fx is the length of the camera focal length in the x-axis direction, fy is the length of the camera focal length in the y-axis direction, u0 is the horizontal pixel coordinate of the actual principal point of the image, and v0 is the vertical pixel coordinate of the actual principal point of the image.

[0060] The camera is then mounted on the car using a fixed device, and the mobile device level is placed on the camera to measure the camera's installation angle α.

[0061] S2, using the monocular camera after the camera internal parameter matrix and the installation angle calibration to collect the front view image of the car when it is driving, and process the collected image;

[0062] Based on deep learning, lane line detection is performed on the image to obtain the coordinates of the current lane line point, and the coordinates of the current lane line point are filtered; Tukey weighted least squares straight line fitting is performed on the filtered current lane line point coordinates to eliminate the lane points that are detected incorrectly.

[0063] The front view image collected by S2 when the car is driving and the image processing process are as follows:

[0064] After completing the camera calibration, the monocular camera is used to collect the forward-looking video of the car while it is driving. The video frames are taken and each frame is dedistorted and cropped to remove the useless sky scene. The image is grayed and converted to the image format. Taking a 1280*720p image as an example, when the camera is fixed on the car at an angle α and an installation height h, the image size is cropped to 800*288, and the image is converted to a grayscale image through Opencv. The existing function is used to convert the image format to the input format required by the deep learning network.

[0065] The lane line detection process based on deep learning in S2 is as follows:

[0066] As attached Figure 3 As shown in the figure, the lane line detection method based on deep learning can accurately detect the coordinate points of the lane lines in the image. At most, four lane lines can be detected, which are marked from left to right as ① (left-left), ② (left), ③ (right), and ④ (right-right). It can be clearly seen that the area included in lane lines ② and ③ is the lane where the current vehicle is located. In the output results of the deep learning network, the coordinate points are pre-classified according to the lane line markings, and the coordinate points on lane lines ② and ③ are selected respectively.

[0067] At the same time, since lane line points that are far away from the vehicle have little reference significance for lateral positioning of the car in the current driving state, it is necessary to further filter the points on the image that are closer to the current vehicle. Since the coordinates of the current lane line points obtained in the previous step are arranged in the array in order from far to near from the current vehicle; for the current lane line point coordinates that are arranged in order in the previous step, the last 50% of the coordinate points in the coordinate array are selected for lateral positioning, that is, the 50% of the coordinate points that are closer to the current vehicle. Based on these 50% of the coordinates, Tukey weighted least squares straight line fitting is performed to effectively eliminate the lane points that are misdetected.

[0068] More specifically, Tukey weighted least squares method is used for fitting, and the specific process is as follows:

[0069] The equation of a straight line uses the classical form: aX+bY+c=0, because the classical form of the equation can represent all types of straight lines. Here a and b have the constraint condition a^2+b^2=1. In this way, the problem of fitting the optimal lane line is transformed into the problem of solving the optimal a, b, and c.

[0070]

[0071] Among them, D is the function for which the extreme value is to be solved, N is the number of lane line coordinate points, and λ is the Lagrange multiplier.

[0072] The steps of Tukey weighted least squares are as follows:

[0073] Step (1): Set the weight ω = 1 for all points, and perform standard least squares fitting to obtain an approximate straight line.

[0074] Step (2): Set a distance threshold τ, calculate the distance d from all points to the straight line in step (1). If d < τ, the weight of this point is If d > τ, the weight of this point is ω = 0.

[0075] Step (3): Perform weighted least squares fitting on all weighted points in step (2) to obtain a new straight line. Repeat the above steps until the most accurate straight line is obtained through multiple iterations of fitting.

[0076]

[0077] For the convenience of subsequent plane constraint calculations, select appropriate starting points and step sizes according to the fitted straight line to recalculate the coordinates of the near-field points. In this embodiment, take the smallest y value among these coordinates as the starting point, the step size is 10, and the end point is the height of the image to divide new y values. Based on the fitted straight line, recalculate the new lane line coordinate points. The lane center line can be fitted from these coordinate points, and the included angle β between this center line and the center line of the image can be calculated, which is the vehicle driving deviation angle.

[0078] S3: Transform the homography solving problem into a normalized Gaussian function model. By uniformly sampling the plane normal on the Gaussian spherical cap, calculate the maximum likelihood according to the lane line constraint, and find the optimal plane normal vector; the specific process is as follows:

[0079] After screening out the lane line coordinate points in the image, perform uniform sampling of the normalized plane normal on the Gaussian spherical cap. The sampling schematic diagram is as shown in the appendix Figure 4 As shown. After obtaining the sampled normal, calculate the rotation matrix between the camera coordinate system and the sampled plane coordinate system through the sampling angle θ and Calculate the translation matrix with the endpoint coordinates of the sampled normal vector. After completing the above steps, combined with the camera internal parameters obtained by calibration, the homography matrix between the image plane and the sampled plane can be calculated, as Figure 5 shown. Thus, as shown in the appendix Figure 6As shown, the lane line coordinate points on the image can be converted to the sampling plane. Then, the joint likelihood of the two plane constraints, namely the parallel constraint and the angle constraint, on each sampling plane is calculated and sorted from small to large. The maximum joint likelihood corresponds to the optimal plane normal vector.

[0080] Assuming that the plane normal is uniformly distributed in space without constraints, after one or more constraints are given, the distribution of the plane normal is updated to a non-uniform distribution. The following definition of plane constraints is given:

[0081] d i (N) = Q i (N)-u i (4)

[0082] Among them, d i (N) is the constraint of the plane determined by the given normal N, Q i (N) represents the geometric properties calculated from the given plane normal N, u i is a priori determined value of a geometric property.

[0083] In this way, the problem of solving the homography can be transformed into a conditional probability maximization problem:

[0084] N * =argmaxp(N|c1,c2…c M ) (5)

[0085] In the formula, c is the constraint condition and M is the number of constraints.

[0086] Since the above problem cannot be solved, the Bayesian formula can be used to transform the problem into:

[0087]

[0088] In order to solve this problem, it is necessary to make all constraints contribute equally to the solution and normalize the measurement errors with different forms, units, scales and other geometric properties. For this purpose, a normalized Gaussian function model is proposed:

[0089]

[0090] Finally, the optimal plane normal vector can be solved by equation (8):

[0091]

[0092] When solving this equation, it is necessary to uniformly sample the plane normals. To this end, part of the spherical cap of the Gaussian sphere is divided into equal areas as the discrete maximum likelihood search space of the plane normals. Considering the bumps and slope characteristics that may be encountered during the driving process of the car, the 30° spherical cap on the Gaussian sphere is divided into equal areas and the plane normal vectors are uniformly sampled. The sampling process is as follows:

[0093] θ=cos -1 (1-(1-cos 30°)*a) (9)

[0094] φ=2πb (10)

[0095] Where θ is the angle between the projection of the vector on the xy plane and the x-axis, is the angle between the vector and the z-axis, and a and b are uniformly distributed random numbers in (0, 1).

[0096] The plane normal vector determined by the above sampling equation is relative to the plane normal coordinate system determined by the pre-calibrated camera installation angle β. This coordinate system is obtained by rotating the camera coordinate system 90°-β counterclockwise around the X-axis. Therefore, to obtain the sampling normal vector in the camera coordinate system, it is necessary to transform the coordinates of the above sampling normal vector into the camera coordinate system and multiply it by a rotation matrix in the opposite direction.

[0097] It can be seen that in determining θ and After that, because the radius r of the Gaussian sphere is 1, a normalized plane normal vector N can be uniquely determined. Therefore, we upsampled the uniformly distributed normalized plane normal vector N on the Gaussian sphere cap. i .

[0098] The plane in the camera coordinate space can be represented by (N i , d) is uniquely determined. Since the distance d is unknown, the distance d is normalized to 1. Thus, a plane (N) is uniquely determined in the camera space. i , 1). For each sampled normal vector, we need to find a set of orthogonal vectors as the directions of the x-axis and y-axis. To facilitate calculation, based on the inspiration of the rotation matrix, we can use the above θ and A rotation matrix R can be constructed to rotate the z-axis of the camera coordinate system to the direction of the sampling normal vector. The origin of the plane coordinate system is the endpoint of the sampling normal vector, and the translation matrix t is obtained. i , the homography matrix H of the coordinates on 1) and the image coordinates can be determined:

[0099] H=K*R_t (11)

[0100] Among them, K is the camera intrinsic parameter matrix obtained by calibration in step S1, and R_t is the matrix formed by combining the rotation matrix R and the translation matrix t.

[0101] The transformation relationship between plane and image coordinates can be given by equation (12):

[0102]

[0103] After the coordinates of the points are converted, the geometric properties need to be calculated.

[0104] The constraint of the lane line on the ground plane can be regarded as a parallel relationship. In order to avoid the problem of infinity and 0 caused by calculating the relationship between the slopes, two points are selected on the two transformed lane lines respectively, and the vector is calculated. The parallel constraint is characterized by the angle between the vectors. At the same time, the vehicle driving deviation angle β calculated in step S2 can also be used as a plane constraint, which is specifically expressed as the ratio of the distance between the point (0, Y) and the corresponding lane line to the distance from the nearest field point to the longitudinal axis, that is, the absolute value of its Y coordinate. According to formula (8), the joint likelihood of the two plane constraints in each sampling plane is calculated, and the maximum joint likelihood is found by sorting. The corresponding plane normal vector is the optimal plane normal vector, and the corresponding plane is the optimal sampling plane.

[0105] S4, perform lateral positioning on the plane corresponding to the maximum likelihood, and restore the true positioning result through the scale s. After finding the optimal plane normal vector, perform lateral positioning on the optimal sampling plane and calculate the distance d from the fitted lane line to the plane coordinate origin * .

[0106] As attached Figure 6 As shown in the figure, since the plane normal vector passes through the origin of the camera coordinate system, and the plane coordinate system is obtained by rotating and translating the camera coordinate system, as shown in the attached figure Figure 7 As shown, the mapping of the normal vector direction of the origin of the camera coordinate system on the ground plane coincides with the origin of the ground plane, so d * On a plane, it can also be considered as the distance from the lane line to the origin of the camera coordinate system.

[0107] The optimal normal vector N obtained in step S3 * is the normalized plane normal vector, which determines the plane (N * , 1) is a plane on the Gaussian sphere, so lateral positioning cannot be performed directly on this plane, but a scale s is needed to restore the distance information on the real plane. Usually, the acquisition of scale s needs to be restored by the distance between two points on the real plane and the distance between the corresponding two points on the sampling plane. For special scenes such as highways, urban roads, and logistics trunk lines, the lane spacing d0 can be used as prior information according to the national standard. The recovery process is shown in the attached figure. Figure 8On the real ground plane, you can choose to calculate the distance from the left lane line or the right lane line to the ground plane coordinate origin and use it as the lateral positioning result, as shown in the attached figure. Fig. 9 As shown, A and B are the intersection points of the camera's field of view and the left and right lane lines. The equation of the straight line in the right lane is aX+bY+c=0. For the lateral positioning problem of the vehicle in the ground plane coordinate system, no matter whether the vehicle's heading deviates, it can be solved by solving the distance from the origin to the straight line.

[0108] To verify the feasibility of the algorithm, a ZED camera was used to calibrate its left eye intrinsic parameter matrix using Zhang Zhengyou's chessboard calibration method. The side length of the chessboard used was 95 mm. The camera was then fixed on the test vehicle, and its installation angle was measured to be 5° using the level function of the mobile device. The test vehicle was then driven to collect a video image of a test road on the campus of Jiangsu University. The image size was 1280*720. The image was processed according to the above implementation steps, and then based on the existing training data, it was input into the deep learning network for processing, and the lane line coordinate points of the lane where the test vehicle was located were extracted. The Tukey weighted least squares method was used to fit the vehicle's near-field lane line straight line and recalculate the coordinate points. The lane departure angle was then calculated by fitting the lane centerline. The lane departure angle in this test was approximately 6°. Based on the installation angle of the camera, the plane normals of the Gaussian spherical cap with an axis of 85° as the base axis, a maximum opening angle of 30° and a radius R of 1 in the camera coordinate system are uniformly sampled. The sampling times are set to 10,000 times. Based on the sampling angle and sampling normal, the coordinate system of each sampling plane is given, and the joint likelihood of the above plane constraints in each sampling plane is calculated. The joint maximum likelihood is obtained after sorting to find the optimal sampling normal. The lateral deviation between the position of the optimal sampling normal obtained in this experiment in the camera coordinate system and the position of the ground plane normal at the actual calibration location is less than 1°, and the longitudinal deviation is about 4°. The normal plane is used as the actual plane where the lane is located to calculate the lateral positioning result of the vehicle, proving that the algorithm is indeed feasible.

[0109] In summary, this application detects the current lane information through a more robust and adaptable deep learning lane line detection method, which is less affected by illumination and noise. It uses the Tukey weighted least squares method to fit a more accurate lane line. Based on Gaussian spherical crown uniform sampling, the problem of finding the real-time optimal homography matrix is ​​transformed into a maximum likelihood problem. The homography matrix of the optimal sampling plane and the image plane is calculated through accurately calibrated camera intrinsic parameters, the image coordinates are transformed, and the lateral distance is calculated. Finally, the true lateral positioning result is restored through the scale s, thereby achieving high-precision lateral positioning of the autonomous driving commercial vehicle in the current lane.

[0110] The above embodiments are only used to illustrate the design ideas and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, any equivalent changes or modifications made based on the principles and design ideas disclosed by the present invention are within the protection scope of the present invention.

Claims

1. A method for lateral positioning of an autonomous commercial vehicle, characterized in that: The steps include: S1, camera intrinsic parameter matrix calibration and camera installation angle calibration; S2, using the monocular camera after completing the camera internal parameter matrix and the installation angle calibration to collect the front view image of the car when it is driving, and process the collected image; Perform lane line detection on the image based on deep learning, and obtain and filter the point coordinates on the lane line where the current vehicle is located; Perform Tukey weighted least squares straight line fitting on the selected current lane line point coordinates to eliminate the misdetected lane line points; S3, based on the coordinates of the current lane line point selected in S2, the plane normal is uniformly sampled on the Gaussian spherical cap, and the homography matrix between the image plane and the sampling plane can be calculated by combining the camera intrinsic parameters obtained by calibration; the maximum likelihood is calculated according to the lane line constraint to find the optimal plane normal vector; S4: Perform lateral positioning on the plane corresponding to the maximum likelihood, and restore the true positioning result through scale s.

2. The method for lateral positioning of an autonomous driving commercial vehicle according to claim 1, characterized in that: The lane line detection method in S2 is as follows: the processed image is used as the input of the deep learning network, the deep learning network is used to detect the coordinate points of the lane lines in the image, the coordinate points are pre-classified according to the lane line markings in the output results of the deep learning network, and the coordinate points on the lane line where the current vehicle is located are obtained.

3. The method for lateral positioning of an autonomous driving commercial vehicle according to claim 2, characterized in that: The method for filtering the coordinate points on the lane line where the current vehicle is located is as follows: For the coordinate points on the lane line where the current vehicle is located, they are first sorted from far to near according to the distance from the coordinate points to the current vehicle, and 50% of the coordinate points close to the current vehicle in the coordinate array are selected for lateral positioning.

4. The method for lateral positioning of an autonomous driving commercial vehicle according to claim 3, characterized in that: The selected near-field points are fitted using Tukey weighted least squares method. The steps of Tukey weighted least squares are as follows: Step (1), set the weight of all points ω = 1, and use the standard least squares method to fit an approximate straight line; Step (2): Set a distance threshold τ, calculate the distance d from all points to the straight line in step (1). If d < τ, the weight of this point is If d > τ, the weight of this point is ω = 0; Step (3), performing weighted least squares fitting on all weighted points in step (2) to obtain a new straight line, and repeating the above steps until the most accurate straight line is obtained through multiple iterations of fitting; According to the fitted straight line, a suitable starting point and step size are selected to recalculate the coordinates of the near-field point, and based on this, the angle β between the center line of the current lane on the image and the midline of the image is calculated as the actual deviation angle of the vehicle during driving.

5. The method for lateral positioning of an autonomous driving commercial vehicle according to claim 3, characterized in that: The method for finding the optimal plane normal vector in S3 is as follows: After filtering out the lane line coordinate points in the image, perform uniform sampling of normalized plane normals on the Gaussian sphere cap. After obtaining the sampled normals, calculate the rotation matrix between the camera coordinate system and the sampling plane coordinate system through the sampling angles θ and φ, and calculate the translation matrix with the coordinates of the sampling normal vector endpoints. θ is the angle between the projection of the vector on the xy plane and the x-axis, and φ is the angle between the vector and the z-axis. After completing the above steps, the homography matrix between the image plane and the sampling plane can be calculated in combination with the camera intrinsic parameters obtained by calibration, and the lane line coordinate points on the image are converted to the sampling plane. After that, calculate the joint likelihood of the two plane constraints, namely the parallel constraint and the angle constraint, on each sampling plane. The maximum joint likelihood corresponds to the optimal plane normal vector.

6. The method for lateral positioning of an autonomous driving commercial vehicle according to claim 3, characterized in that: The method for lateral positioning in S4 is: After finding the optimal plane normal vector, calculate the distance d between the two parallel lines fitted in step S3 on the optimal plane. * ; The scale s is expressed as: Among them, d0 is the actual distance between the lane lines on the left and right sides of the road section; The lane line equation obtained by fitting in the optimal plane in step S3 is used to calculate the distance between the origin of the plane coordinate system (0,0) and the fitted lane line, and then the scale information is used to restore the distance between the camera and the actual left and right lane lines: Among them, a, b, and c are the parameters of the fitted lane line equation, and X and Y are the coordinate points projected from the camera center to the ground plane.

7. A method for lateral positioning of an autonomous driving commercial vehicle according to any one of claims 1 to 6, characterized in that: The camera’s intrinsic parameter matrix is ​​calibrated using the checkerboard method.

8. The method for lateral positioning of an autonomous driving commercial vehicle according to claim 7, characterized in that: The camera intrinsic parameter matrix calibration method is: put the chessboard flat on the ground, with the chessboard plane taking the detected chessboard corner point as the origin, and calculate the intrinsic parameter matrix K between the camera plane and the chessboard plane through multiple measurements using Zhang Zhengyou calibration method, which is expressed as: Among them, fx is the length of the camera focal length in the x-axis direction, fy is the length of the camera focal length in the y-axis direction, u0 is the horizontal pixel coordinate of the actual principal point of the image, and v0 is the vertical pixel coordinate of the actual principal point of the image.

9. The method for lateral positioning of an autonomous driving commercial vehicle according to claim 7, characterized in that: The method for calibrating the installation angle is as follows: for a camera fixedly installed on a vehicle, a level is used to calibrate the installation angle α of the camera.

10. The method for lateral positioning of an autonomous driving commercial vehicle according to claim 7, characterized in that: Image processing in S2 includes: video frame acquisition, dedistortion, cropping, image grayscale, and image format conversion.

Images