External parameter calibration methods, devices, equipment, vehicles, and media for vehicle-mounted cameras

By utilizing cross-ratio invariance in road surface images to obtain vanishing points from key points of vehicles of the same model, the accuracy and generalization problems of extrinsic parameter calibration of vehicle cameras in complex scenarios are solved, enabling effective distance and speed measurement in scenarios without parallel lane lines.

CN116664691BActive Publication Date: 2026-11-10YINGCHE XINGCHUANG INTELLIGENT TECH (SHANGHAI) CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202310602702.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-05-25
Publication Date
2026-11-10
Estimated Expiration
2043-05-25

AI Technical Summary

Technical Problem

In the existing technology, the extrinsic parameter calibration method for vehicle cameras cannot effectively measure distance or speed in scenarios without parallel lane lines, and traditional manual calibration is prone to introducing errors. The existing self-calibration method has insufficient generalization performance in real-world scenarios.

Method used

By acquiring target vehicle information and key points from road surface images, and utilizing cross ratio invariance to obtain vanishing points from key points of vehicles of the same model, external parameter calibration is achieved, avoiding dependence on lane lines.

Benefits of technology

In complex scenarios such as complex lane lines, intersections, and rural roads, accurate calibration of the external parameters of the vehicle-mounted camera was achieved, improving generalization ability and the accuracy of ranging and speed measurement.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116664691B_ABST
    Figure CN116664691B_ABST
Patent Text Reader

Abstract

This invention provides a method, apparatus, device, vehicle, and medium for extrinsic parameter calibration of an onboard camera, comprising: acquiring vehicle model information and key points of the target vehicle from a road surface image; selecting key points according to preset key point selection conditions to obtain a first key point corresponding to a first vehicle and a second key point corresponding to a second vehicle; and obtaining one of the extrinsic parameters corresponding to the onboard camera—the vanishing point—using cross-ratio invariance based on the first and second key points; wherein the preset key point selection conditions are: selecting key points from multiple key points that satisfy the condition that the line connecting the key points is parallel to the bottom or top edge of the road surface image, and that, in the world coordinate system, the distance between key points in the first vehicle is equal to the distance between key points in the second vehicle. The extrinsic parameter calibration of this invention does not rely on lane line information and has better generalization ability.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method, apparatus, equipment, vehicle, and medium for calibrating the external parameters of a vehicle-mounted camera. Background Technology

[0002] In ADAS (Advanced Driving Assistant Systems), forward-looking tasks include FCW (Forward Collision Warning), LDW (Lane Departure Warning), PCW (Pedestrian Collision Warning), and TSR (Traffic Sign Recognition). Among these tasks, the most crucial is estimating the distance or speed of objects ahead or lane markings on the road. To achieve accurate distance or speed estimation, the external parameters of the onboard camera used for forward-looking tasks must first be calibrated.

[0003] Traditional external parameter calibration requires camera installers to manually calibrate the camera's height, vanishing point, and other parameters. However, errors are easily introduced during manual calibration, and as time goes on, various external factors such as bumps and external impacts can cause the actual parameters of the camera to deviate from the initial calibration, thus affecting the accuracy of ranging or speed measurement algorithms.

[0004] In existing technologies, in order to reduce the cost of manual calibration and improve the robustness of the algorithm during operation, the self-calibration of external parameters, especially the self-calibration of the vanishing point, has become a key focus in ADAS forward-looking tasks. Currently, the self-calibration of the vanishing point in external parameter calibration algorithms is generally divided into the following two types: (1) Based on the perspective principle, the intersection of two parallel lines is taken as the vanishing point, and generally the intersection of lane lines is taken as the vanishing point. (2) The vanishing line (skyline) is learned directly through artificial data, and then the point corresponding to the horizontal coordinate of the center of the image is taken as the vanishing point.

[0005] However, method (1) requires at least two clearly visible lane lines on the road surface, and the lane lines should be kept as straight as possible. Method (2) mainly relies on the generalization performance of the model. There is a large difference between the artificial dataset on which the model is based and the real front-view camera scene. When the model is applied to the real scene, the model effect is not guaranteed.

[0006] Since the above method (1) is constrained in road scenarios, especially trunk road scenarios, method (1) is often used as the standard method for camera self-calibration. However, due to the defects of the above-mentioned method (1), in scenarios without parallel lane lines, such as merging lanes, forking lanes, intersections, rural roads, etc., the extrinsic parameter calibration algorithm often fails because the corresponding parallel lane lines cannot be obtained, and distance or speed measurement cannot be performed.

[0007] In summary, there is an urgent need for a camera extrinsic parameter calibration method applicable to real-world and diverse road scenarios. Summary of the Invention

[0008] This invention provides a method, apparatus, equipment, vehicle, and medium for calibrating the external parameters of a vehicle-mounted camera, in order to solve the above-mentioned problems.

[0009] This invention provides a method for calibrating the extrinsic parameters of a vehicle-mounted camera, comprising:

[0010] Use the vehicle's onboard camera to acquire road surface images;

[0011] The vehicle type information and multiple key points of the target vehicle are obtained from the road surface image. The target vehicle refers to the vehicles surrounding the vehicle. The road surface image includes multiple target vehicles.

[0012] Based on the vehicle model information, a target vehicle pair is obtained from the plurality of target vehicles, wherein the target vehicle pair includes a first vehicle and a second vehicle;

[0013] Based on preset key point selection conditions, key points corresponding to the first vehicle and the second vehicle are selected to obtain at least two first key points corresponding to the first vehicle and at least two second key points corresponding to the second vehicle.

[0014] Based on the first key point and the second key point, the vanishing point corresponding to the vehicle camera is obtained by utilizing the cross ratio invariance.

[0015] The preset key point selection criteria are as follows: from the plurality of key points, key points are selected that satisfy the condition that the line connecting the key points is parallel to the bottom or top edge of the road surface image, and in the world coordinate system, the distance between key points in the first vehicle is equal to the distance between key points in the second vehicle.

[0016] According to the extrinsic parameter calibration method for a vehicle-mounted camera provided by the present invention, when there are two first key points and two second key points, and the first key point is a and b, and the second key point is c and d, wherein the first key point a corresponds to the second key point c, and the first key point b corresponds to the second key point d; the distance between the first vehicle and the vehicle itself is less than the distance between the second vehicle and the vehicle itself.

[0017] The step of obtaining the vanishing point corresponding to the vehicle-mounted camera based on the first key point and the second key point using cross-ratio invariance includes:

[0018] Based on the positional relationship between the second key point c and the first key point a, and the distance between the first key point a and the first key point b, select equidistant points f on the straight lines corresponding to the first key point a and the first key point b;

[0019] Obtain the intersection point g between the line corresponding to the equidistant point f and the second key point d and the line corresponding to the first key point b and the second key point c;

[0020] Obtain the shadow line parallel to the bottom or top edge of the road surface image based on the intersection point g;

[0021] The vanishing point is obtained from the vanishing line based on the size information of the road surface image.

[0022] According to the extrinsic parameter calibration method for a vehicle-mounted camera provided by the present invention, after obtaining the vehicle model information and multiple key points of the target vehicle from the road surface image, the method further includes:

[0023] When the vehicle model information of the target vehicles are all different, multiple frames of road surface images are acquired, and the same target vehicle is located in different positions in the multiple frames of road surface images.

[0024] The multiple road surface images are merged to obtain a merged image;

[0025] The vehicle model information and several key features of the target vehicle are obtained from the merged image.

[0026] According to the present invention, a method for calibrating the extrinsic parameters of a vehicle-mounted camera is provided, wherein the target vehicle pair is multiple target vehicle pairs, and the step of obtaining the vanishing point corresponding to the vehicle-mounted camera based on the first key point and the second key point using cross ratio invariance includes:

[0027] For each of the plurality of target vehicle pairs, the vanishing point corresponding to the target vehicle pair is obtained by utilizing the cross ratio invariance based on the first key point and the second key point corresponding to the target vehicle pair.

[0028] The vanishing point corresponding to the vehicle-mounted camera is obtained by averaging the vanishing points corresponding to the multiple target vehicle pairs.

[0029] According to the present invention, a method for calibrating the extrinsic parameters of an in-vehicle camera, wherein obtaining target vehicle pairs from a plurality of target vehicles based on the vehicle model information includes:

[0030] Group target vehicles with the same vehicle model information into multiple pairs of target vehicles.

[0031] The vehicle type information is at least one of the following: small car, micro car, compact car, mid-range car, high-end car, luxury car, sedan, CDV, MPV, or SUV.

[0032] According to the present invention, the external parameter calibration method for a vehicle-mounted camera, the first key point and the second key point are wheel external grounding key points.

[0033] The present invention also provides an external parameter calibration device for a vehicle-mounted camera, comprising:

[0034] The image acquisition module is used to acquire road surface images using the vehicle's onboard camera.

[0035] The target vehicle information acquisition module is used to acquire the vehicle type information and multiple key points of the target vehicle from the road surface image. The target vehicle is the vehicle around the vehicle. The road surface image includes multiple target vehicles.

[0036] A vehicle pair determination module is used to obtain a target vehicle pair from the plurality of target vehicles based on the vehicle model information, wherein the target vehicle pair includes a first vehicle and a second vehicle;

[0037] The key point selection module is used to select key points corresponding to the first vehicle and the second vehicle according to preset key point selection conditions, and obtain at least two first key points corresponding to the first vehicle and at least two second key points corresponding to the second vehicle.

[0038] The vanishing point acquisition module is used to acquire the vanishing point corresponding to the vehicle camera based on the first key point and the second key point, using the cross ratio invariance.

[0039] The preset key point selection criteria are as follows: from the plurality of key points, key points are selected that satisfy the condition that the line connecting the key points is parallel to the bottom or top edge of the road surface image, and in the world coordinate system, the distance between key points in the first vehicle is equal to the distance between key points in the second vehicle.

[0040] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the extrinsic parameter calibration method for a vehicle-mounted camera as described above.

[0041] The present invention also provides a vehicle including the above-described electronic equipment.

[0042] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the extrinsic parameter calibration method for vehicle-mounted cameras as described above.

[0043] The present invention provides a method, apparatus, device, vehicle, and medium for extrinsic parameter calibration of vehicle-mounted cameras. This method obtains a first key point and a second key point from a target vehicle of the same model. In the world coordinate system, the distance between the first and second key points is equal, and the straight lines passing through the first and second key points are parallel to the bottom or top edge of the road surface image. Based on these first and second key points, the vanishing point—one of the extrinsic parameters of the vehicle-mounted camera—is obtained using the cross-ratio invariance in projective transformation. The extrinsic parameter calibration method of the present invention does not rely on lane line information; it only requires obtaining key points that meet the conditions from vehicles of the same model to obtain the vanishing point. It can effectively complete the extrinsic parameter calibration of vehicle-mounted cameras in various scenarios such as complex lane lines, intersections, rural roads, and mining areas, thereby improving the generalization ability of vehicle-mounted camera extrinsic parameter calibration in various scenarios. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0045] Figure 1 This is a flowchart illustrating the external parameter calibration method for a vehicle-mounted camera provided in an embodiment of the present invention;

[0046] Figure 2 This is a schematic diagram of obtaining the vanishing point based on cross ratio invariance provided in an embodiment of the present invention;

[0047] Figure 3This is a schematic diagram of determining the vanishing point based on key points provided in an embodiment of the present invention;

[0048] Figure 4 This is a schematic diagram of the merged image provided in an embodiment of the present invention;

[0049] Figure 5 This is a schematic diagram of the external parameter calibration device for a vehicle-mounted camera provided in an embodiment of the present invention;

[0050] Figure 6 A schematic diagram of the physical structure of an electronic device is provided. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0052] Figure 1 This is a flowchart illustrating the extrinsic parameter calibration method for a vehicle-mounted camera provided in an embodiment of the present invention; as shown below. Figure 1 As shown, the extrinsic parameter calibration method for the vehicle-mounted camera includes the following steps:

[0053] S101 uses an onboard camera in the vehicle to acquire road surface images.

[0054] In this step, images of the road surface surrounding the vehicle are acquired using onboard cameras installed in the vehicle. These road surface images include multiple target vehicles, which are vehicles surrounding the vehicle. It should be noted that the onboard camera can be a front-view camera, a surround-view camera, or a rear-view camera. For surround-view cameras, distortion correction is first applied to the initial acquired road surface image, and extrinsic parameter calibration is performed based on the corrected image. For front-view and rear-view cameras, extrinsic parameter calibration can be performed directly based on the acquired road surface images.

[0055] S102, Obtain the vehicle model information and multiple key points of the target vehicle from the road surface image.

[0056] In this step, the road image is used to detect targets using existing target detection algorithms, thereby predicting the vehicle type information and bounding box of the target vehicle. At the same time, multiple key points of the target vehicle are output, such as wheel key points, headlight key points, vehicle rearview mirrors, etc.

[0057] The existing object detection algorithms can be traditional algorithms, such as the Viola Jones detector and the HOG detector, or deep learning-based algorithms, such as the R-CNN series, the YOLO series, and SSD. This invention does not limit the specific object detection algorithm. Furthermore, it should be noted that in the process of object detection using the R-CNN series and YOLO series, key points are obtained through direct regression, or separately through Stacked Hourglass Networks, or directly through the end-to-end object detection algorithm CenterNet. This invention also does not limit the acquisition of key points.

[0058] S103, Obtain a target vehicle pair from the plurality of target vehicles based on the vehicle model information.

[0059] The target vehicle pair includes a first vehicle and a second vehicle. The vehicle type information is at least one of the following: small car, microcar, compact car, mid-size car, high-end car, luxury car, sedan, CDV, MPV, or SUV.

[0060] More specifically, obtaining target vehicle pairs from the plurality of target vehicles based on the vehicle model information includes:

[0061] Target vehicles with the same model information are grouped together to obtain multiple pairs of target vehicles. That is, target vehicles with the same model are paired up to form target vehicle pairs. For example, if three target vehicles are detected, namely a small car, a small car, and an SUV, then the two small cars form one target vehicle pair. If all three detected target vehicles are small cars, then small car C and small car D form one target vehicle pair, small car C and small car E form one target vehicle pair, and small car D and small car E form one target vehicle pair, thus obtaining three pairs of target vehicle pairs. To distinguish the two cars in a target vehicle pair, they are named the first vehicle and the second vehicle, respectively.

[0062] S104, select key points corresponding to the first vehicle and the second vehicle according to preset key point selection conditions, and obtain at least two first key points corresponding to the first vehicle and at least two second key points corresponding to the second vehicle.

[0063] The preset keypoint selection criteria are as follows: from the plurality of keypoints, keypoints are selected that satisfy the condition that the line connecting the keypoints is parallel to the bottom or top edge of the road surface image, and that the distance between the first keypoints is equal to the distance between the second keypoints in the world coordinate system. It should be noted that the calculation of the distance between the first and second keypoints is not based on the pixel coordinates of the road surface image, but rather on the coordinates of each keypoint in the world coordinate system. Furthermore, a certain margin of error (within 2 degrees) is allowed for parallelism; absolute parallelism is not required.

[0064] For example, suppose that the straight lines corresponding to the two front wheel key points detected in the first vehicle are parallel to the bottom or top edge of the road surface image, and the straight lines corresponding to the two rear wheel key points detected in the second vehicle are parallel to the bottom or top edge of the road surface image, and the distance between the two front wheel key points and the distance between the two rear wheel key points are equal in the world coordinate system (the wheelbase of the same vehicle model is basically the same, and the error can be ignored in this invention), then the two front wheel key points are determined as the first key points of the first vehicle, and the two rear wheel key points are determined as the second key points of the second vehicle.

[0065] Alternatively, suppose that two rear wheel key points are detected in the first vehicle and the corresponding straight lines are parallel to the bottom or top edge of the road image, and two rear wheel key points are detected in the second vehicle and the corresponding straight lines are parallel to the bottom or top edge of the road image. Based on this, the distance between the two rear wheel key points in the first vehicle and the distance between the two rear wheel key points in the second vehicle are also equal, thus obtaining the first key point and the second key point.

[0066] In addition, at least two first and second key points are selected to obtain the lines corresponding to the first and second key points. If there is only one line corresponding to multiple first or second key points that is parallel to the bottom or top edge of the road image, then the vanishing point is obtained based on these two lines and the selected first and second key points on the line. If both the first and second key points correspond to multiple lines, then the vanishing point is obtained based on the paired lines, and the final vanishing point is determined from the multiple vanishing points. The specific determination process is described below.

[0067] It should be noted that the aforementioned equal distances are determined based on vehicle model information. Vehicles of the same model may have some identical parameters (which may have minor errors, negligible in this invention), such as the wheelbase and the distance between the side mirrors. In this invention, generally, the position of the first key point in the first vehicle corresponds to the position of the second key point in the second vehicle, such as the key point at the rear of the two vehicles. There are exceptions; the positions of the first and second key points in the first and second vehicles may not correspond. For example, when the first and second vehicles are traveling in the same direction, the rear wheel key points of the vehicles correspond, and the left and right side mirrors of the first vehicle correspond to the left and right side mirrors of the second vehicle, etc. When the first and second vehicles are traveling in opposite directions, the front wheel key point of the first vehicle (assuming the rear wheel key point of the first vehicle is obscured and not detected) corresponds to the rear wheel key point of the second vehicle, the left side mirror of the first vehicle corresponds to the right side mirror of the second vehicle, the right side mirror of the first vehicle corresponds to the left side mirror of the second vehicle, and so on.

[0068] S105, based on the first key point and the second key point, the vanishing point corresponding to the vehicle camera is obtained using cross-ratio invariance. The vanishing point is one of the extrinsic parameters of the vehicle camera. After obtaining the vanishing point, other extrinsic parameters such as the camera's pitch angle can be further determined based on it. The determination of other extrinsic parameters is implemented using commonly used vehicle camera extrinsic parameter calibration algorithms, which are not limited in this invention. Furthermore, after obtaining all the extrinsic parameter calibration data of the vehicle camera, the calibration data can be used for vehicle speed or distance measurement to complete the forward-looking task in autonomous driving.

[0069] In this step, the vanishing point is obtained based on the determined key points and through cross-ratio invariance.

[0070] The application of cross-ratio invariance in vanishing point acquisition is explained below. Figure 2 Taking points a”, b”, c”, a’, b’, and c’ (these six points are not the same as the first and second key points mentioned in this invention; they are only used to illustrate the principle of cross-ratio invariance) as an example, the vanishing point is obtained based on the length ratio. The specific steps are as follows:

[0071] 1) Identify three collinear points a', b', and c' in the image. In the world coordinate system, the ratio of the intervals between these three points is A:B.

[0072] 2) Draw a straight line l that passes through point a'. This straight line l does not coincide with the straight line n that passes through points a', b', and c'. Take points a”, b”, and c” on the straight line l, and denote point a” = a'. In the world coordinate system, the distance between point a” and point b” and the distance between point b” and point c” satisfy A:B.

[0073] 3) Obtain the line passing through points b” and b’, and the line passing through points c” and c’, and then obtain the intersection point o of the two lines.

[0074] 4) Obtain a line m that passes through the intersection point o and is parallel to the line l. Determine the intersection point v' of the line m and the line n as the vanishing point.

[0075] The vanishing point is obtained from the road surface image using the cross-ratio invariance described above. It should be noted that one vanishing point corresponds to one pair of target vehicles, and multiple vanishing points will be obtained when multiple pairs of target vehicles exist. In this case, the final vanishing point can be obtained by averaging the coordinates of multiple vanishing points; alternatively, it can be determined from multiple vanishing points using preset filtering conditions (such as the target vehicle pair closest to the vehicle); or it can be determined by combining vanishing points obtained through lane lines or other conventional vanishing point acquisition methods from the multiple vanishing points obtained by this invention. This invention does not limit the specific methods used.

[0076] According to the extrinsic parameter calibration method for vehicle-mounted cameras provided in this embodiment of the invention, a first key point and a second key point are obtained from the target vehicle of the vehicle model. In the world coordinate system, the distance between the two first key points is equal to the distance between the two second key points, and the straight line passing through the first key point and the straight line passing through the second key point are parallel to the bottom or top edge of the road surface image. Based on the above-mentioned first key points and second key points, the vanishing point is obtained by utilizing the cross ratio invariance in projective transformation. Then, the extrinsic parameter calibration of the vehicle-mounted camera is completed based on the vanishing point. The extrinsic parameter calibration method of this invention does not rely on lane line information. It only needs to obtain the vanishing point by obtaining the key points that meet the conditions from the vehicle of the same model. It can still effectively complete the extrinsic parameter calibration of vehicle-mounted cameras in various scenarios such as complex lane lines, intersections, rural roads, and mining areas, thereby improving the generalization ability of vehicle-mounted camera extrinsic parameter calibration in various scenarios.

[0077] Furthermore, based on the above embodiments, when there are two key points for both the first key point and the second key point, and the first key point is first key point a and first key point b, and the second key point is second key point c and second key point d, wherein first key point a corresponds to second key point c, and first key point b corresponds to second key point d; the distance between the first vehicle and the self-vehicle is less than the distance between the second vehicle and the self-vehicle, and the specific positional relationship is as follows: Figure 3 As shown.

[0078] In this embodiment, two key points are used for both the first and second key points. Specifically, the first key point a corresponds to the second key point c, and the first key point b corresponds to the second key point d. This means that the sequential positions of points a and b in line ab correspond to the sequential positions of points c and d in line cd. Figure 3 As shown, point a on the left corresponds to point c on the left, and point b on the right corresponds to point d on the right. It should be noted that the left and right here are relative to the positions of the two key points, not the positions of the key points in the image.

[0079] Accordingly, the step of obtaining the vanishing point corresponding to the vehicle-mounted camera based on the first key point and the second key point using cross-ratio invariance includes:

[0080] Based on the positional relationship between the second key point c and the first key point a, and the distance between the first key point a and the first key point b, select equidistant points f on the straight lines corresponding to the first key point a and the first key point b.

[0081] by Figure 3 Taking the key point position in the image as an example, according to the positional relationship between the second key point c and the first key point a, the second key point c is to the right of the first key point a. Then, take an equidistant point f to the right of the line corresponding to points a and b. This equidistant point f satisfies that the distance between point f and point b is equal to the distance between point a and point b. It should be noted that the distance here refers to the distance in the image (i.e., the distance between pixels), not the distance in the world coordinate system.

[0082] In other embodiments, if point c is to the left of point a, then an equidistant point f is taken to the left of line ab, which will not be elaborated further here.

[0083] Obtain the intersection point g between the straight line corresponding to the equidistant point f and the second key point d and the straight line corresponding to the first key point b and the second key point c.

[0084] Obtain the shadow line parallel to the bottom or top edge of the road surface image based on the intersection point g.

[0085] The vanishing point is obtained from the vanishing line based on the size information of the road surface image. Specifically, after obtaining the vanishing line, the midpoint of the vanishing line is selected based on the width of the road surface image. For example, if the width of the road surface image is 1280 pixels, the midpoint of the vanishing line parallel to the bottom edge of the image is obtained based on 1280 pixels as the vanishing point.

[0086] In addition, Figure 3 In the diagram, the intersection point of lines ad and bc is e. The world coordinates corresponding to the intersection point e are the midpoint between the first and second vehicles.

[0087] According to the cross-ratio invariance, Figure 3 In the equation, ae:ed = ab:bf, and ae:ed is actually equal to ab:cd. When ab:cd = 1:1, we take bf = ab, and based on this, we determine the equidistant point f.

[0088] It should be noted that in this embodiment, the number of the first key point and the second key point is limited to two, which can speed up the determination of the vanishing point. In other solutions of the present invention, multiple vanishing points can also be obtained by multiple first key points and multiple second key points (that is, the vanishing point of each pair of straight lines is obtained by using the cross ratio invariance based on multiple pairs of straight lines), and then the vanishing point corresponding to the target vehicle pair is determined from multiple vanishing points to improve accuracy.

[0089] According to the extrinsic parameter calibration method for vehicle-mounted cameras provided in the embodiments of the present invention, the vanishing point is obtained by a set of equal-length line segments parallel to the bottom edge of the image. The equal-length line segments are obtained based on some key points of the same vehicle model with fixed distances. Then, the extrinsic parameter calibration of the camera is completed based on the vanishing point, so that the extrinsic parameter calibration of the vehicle-mounted camera no longer relies excessively on lane line information in real-world scenarios and has better generalization.

[0090] Furthermore, based on the above embodiments, after obtaining the vehicle model information and multiple key points of the target vehicle from the road surface image, the method further includes:

[0091] When the vehicle model information of the target vehicles are all different, multiple frames of road surface images are acquired, in which the same target vehicle is located at different positions in the multiple frames of road surface images.

[0092] The multiple road surface images are merged to obtain a merged image. (Illustrative image follows) Figure 4 As shown.

[0093] The vehicle model information and several key features of the target vehicle are obtained from the merged image.

[0094] In this embodiment, if the detected target vehicles are not of the same model and cannot form a target vehicle pair, the images of the preceding and following frames can be merged, and the merged image can be used for target detection and key point detection again.

[0095] It should be noted that the above-mentioned image merging has certain requirements regarding the front-to-back position of the target vehicle, that is, the longitudinal position of the vehicle changes between the two frames. Furthermore, the differences in road surface conditions between multiple frames should be minimal. Additionally, to save computation time, only two suitable road surface images can be merged; however, to improve the accuracy of vanishing point acquisition, more than two road surface images can be merged, and this invention does not impose any limitations on this.

[0096] If there are vehicles with the same model among the detected target vehicles, the subsequent target vehicle pairing steps can be carried out directly. It should be noted that in this case, only one frame of road surface image is needed to complete the extrinsic parameter calibration.

[0097] Furthermore, in the case of obtaining vanishing points based on merged images, the first and second vehicles in the target vehicle pair are actually the same vehicle. Keypoints obtained from the same vehicle are more likely to meet the preset keypoint selection criteria, but this method has certain limitations on road conditions for multi-frame road images. In the case of obtaining vanishing points based on a single-frame road image, although there are no limitations on the road image, since the first and second vehicles are actually two vehicles of the same model, the distances between some keypoints of these two vehicles may differ. Therefore, the acquisition of the first and second keypoints is not as accurate as the aforementioned method. Both methods can be used individually or simultaneously.

[0098] According to the embodiment of the present invention, the extrinsic parameter calibration method for vehicle-mounted cameras, when there are no vehicles of the same model in the road surface image, naturally completes vehicle platooning by merging multiple frames of road surface images, further improving the generalizability of the extrinsic parameter calibration method of the present invention.

[0099] Furthermore, based on the above embodiments, the target vehicle pair is multiple target vehicle pairs, and the step of obtaining the vanishing point corresponding to the vehicle-mounted camera based on the first key point and the second key point using cross-ratio invariance includes:

[0100] For each of the plurality of target vehicle pairs, the vanishing point corresponding to the target vehicle pair is obtained by utilizing the cross ratio invariance based on the first key point and the second key point corresponding to the target vehicle pair.

[0101] The vanishing point corresponding to the vehicle-mounted camera is obtained by averaging the vanishing points corresponding to the multiple target vehicle pairs.

[0102] In this embodiment, by processing the key points of each pair of target vehicles using the aforementioned vanishing point acquisition method, a corresponding vanishing point can be obtained. For the multiple obtained vanishing points, the pixel coordinates of the final vanishing point are calculated using the average value. Then, extrinsic parameter calibration is completed based on the final vanishing point.

[0103] Alternatively, to improve computational speed, only one pair of target vehicles (e.g., the two closest vehicles of the same model) can be selected from multiple pairs of target vehicles for vanishing point acquisition. To improve the accuracy of vanishing point acquisition, the aforementioned averaging process can be applied to multiple vanishing points.

[0104] According to the extrinsic parameter calibration method for vehicle-mounted cameras provided in this embodiment of the invention, the final vanishing point is obtained by averaging multiple vanishing points, thereby improving the accuracy of vanishing point acquisition and thus improving the accuracy of extrinsic parameter calibration.

[0105] Furthermore, based on the above embodiments, the first key point and the second key point are wheel external ground contact key points. In this embodiment, the first key point is determined to be two wheel external ground contact key points in the first vehicle, and the second key point is determined to be two wheel external ground contact key points in the second vehicle. Generally, the road surface image acquired by the vehicle-mounted camera can detect one, two, or three wheel external ground contact key points of surrounding vehicles. Based on satisfying the above-mentioned preset key point selection conditions, the two wheel external ground contact key points in each vehicle are finally used as the first key point or the second key point.

[0106] Furthermore, using the outer contact point of the vehicle's wheels with the road surface as the first or second key point is not only because the outer contact point is more clearly visible, but also because the vanishing point refers to the vanishing point of the lane lines, and since lane lines are road surface information, the outer contact point of the wheels can also be considered road surface information. If other key points within the vehicle are used as the first or second key points, it is necessary to ensure that the lines connecting the key points are parallel to the road surface.

[0107] exist Figure 3 In the process, the first vehicle and the second vehicle are two vehicles traveling in the same direction. Among the key points detected, the two rear wheel contact points of the first vehicle are taken as the first key points, and the two rear wheel contact points of the second vehicle are taken as the second key points.

[0108] In another scenario, if the first vehicle and the second vehicle are traveling in different directions, then... Figure 3 For example, assuming the second vehicle is located in the opposite lane next to it, then among the detected key points, the two rear wheel contact points of the first vehicle can be used as the first key points, and the two front wheel contact points of the second vehicle can be used as the second key points.

[0109] According to the embodiment of the present invention, the external parameter calibration method for vehicle-mounted cameras improves computational efficiency by defining the first key point and the second key point as the wheel's external ground contact point. The wheel's external ground contact point is more clearly defined in the key point detection process of the road image and has a lower probability of being occluded.

[0110] In addition, to further improve the accuracy of extrinsic parameter calibration, the extrinsic parameter calibration method provided by this invention can be used in conjunction with existing extrinsic parameter calibration methods. For example, the vanishing point can be obtained both through lane lines and through key points of the target vehicle pair, and the vanishing point can be obtained by combining the two methods to complete the final extrinsic parameter calibration.

[0111] The extrinsic parameter calibration device for vehicle-mounted cameras provided by the present invention is described below. The extrinsic parameter calibration device for vehicle-mounted cameras described below can be referred to in correspondence with the extrinsic parameter calibration method for vehicle-mounted cameras described above.

[0112] Figure 5 This is a schematic diagram of the external parameter calibration device for a vehicle-mounted camera provided in an embodiment of the present invention; as shown below. Figure 5 As shown, the extrinsic parameter calibration device for the vehicle-mounted camera includes an image acquisition module 501, a target vehicle information acquisition module 502, a vehicle pair determination module 503, a key point selection module 504, and a vanishing point acquisition module 505.

[0113] Image acquisition module 501 is used to acquire road surface images using an onboard camera in the vehicle.

[0114] In this module, on-board cameras installed in the vehicle acquire road surface images surrounding the vehicle. These road surface images include multiple target vehicles, which are vehicles surrounding the vehicle. It should be noted that the on-board camera can be a front-view camera, a surround-view camera, or a rear-view camera. For surround-view cameras, distortion correction is first applied to the initial acquired road surface image, and extrinsic parameter calibration is performed based on the corrected image. Front-view and rear-view cameras can directly perform extrinsic parameter calibration based on the acquired road surface images.

[0115] The target vehicle information acquisition module 502 is used to acquire the vehicle model information and multiple key points of the target vehicle from the road surface image.

[0116] In this module, existing target detection algorithms are used to detect targets in road images, thereby predicting the vehicle type information and bounding box of the target vehicle. At the same time, multiple key points of the target vehicle are output, such as wheel key points, headlight key points, vehicle rearview mirrors, etc.

[0117] The existing object detection algorithms can be traditional algorithms, such as the Viola Jones detector and the HOG detector, or deep learning-based algorithms, such as the R-CNN series, the YOLO series, and SSD. This invention does not limit the specific object detection algorithm. Furthermore, it should be noted that in the process of object detection using the R-CNN series and YOLO series, key points are obtained through direct regression, or separately through Stacked Hourglass Networks, or directly through the end-to-end object detection algorithm CenterNet. This invention also does not limit the acquisition of key points.

[0118] The vehicle pair determination module 503 is used to obtain target vehicle pairs from the plurality of target vehicles based on the vehicle model information.

[0119] The target vehicle pair includes a first vehicle and a second vehicle. The vehicle type information is at least one of the following: small car, microcar, compact car, mid-size car, high-end car, luxury car, sedan, CDV, MPV, or SUV.

[0120] More specifically, obtaining target vehicle pairs from the plurality of target vehicles based on the vehicle model information includes:

[0121] Target vehicles with the same model information are grouped together to obtain multiple pairs of target vehicles. That is, target vehicles with the same model are paired up to form target vehicle pairs. For example, if three target vehicles are detected, namely a small car, a small car, and an SUV, then the two small cars form one target vehicle pair. If all three detected target vehicles are small cars, then small car C and small car D form one target vehicle pair, small car C and small car E form one target vehicle pair, and small car D and small car E form one target vehicle pair, thus obtaining three pairs of target vehicle pairs. To distinguish the two cars in a target vehicle pair, they are named the first vehicle and the second vehicle, respectively.

[0122] The key point selection module 504 is used to select key points corresponding to the first vehicle and the second vehicle according to preset key point selection conditions, and obtain at least two first key points corresponding to the first vehicle and at least two second key points corresponding to the second vehicle.

[0123] The preset keypoint selection criteria are as follows: from the plurality of keypoints, keypoints are selected that satisfy the condition that the line connecting the keypoints is parallel to the bottom or top edge of the road surface image, and that the distance between the first keypoints is equal to the distance between the second keypoints in the world coordinate system. It should be noted that the calculation of the distance between the first and second keypoints is not based on the pixel coordinates of the road surface image, but rather on the coordinates of each keypoint in the world coordinate system. Furthermore, a certain margin of error (within 2 degrees) is allowed for parallelism; absolute parallelism is not required.

[0124] For example, suppose that the straight lines corresponding to the two front wheel key points detected in the first vehicle are parallel to the bottom or top edge of the road surface image, and the straight lines corresponding to the two rear wheel key points detected in the second vehicle are parallel to the bottom or top edge of the road surface image, and the distance between the two front wheel key points and the distance between the two rear wheel key points are equal in the world coordinate system (the wheelbase of the same vehicle model is basically the same, and the error can be ignored in this invention), then the two front wheel key points are determined as the first key points of the first vehicle, and the two rear wheel key points are determined as the second key points of the second vehicle.

[0125] Alternatively, suppose that two rear wheel key points are detected in the first vehicle and the corresponding straight lines are parallel to the bottom or top edge of the road image, and two rear wheel key points are detected in the second vehicle and the corresponding straight lines are parallel to the bottom or top edge of the road image. Based on this, the distance between the two rear wheel key points in the first vehicle and the distance between the two rear wheel key points in the second vehicle are also equal, thus obtaining the first key point and the second key point.

[0126] In addition, at least two first and second key points are selected to obtain the lines corresponding to the first and second key points. If there is only one line corresponding to multiple first or second key points that is parallel to the bottom or top edge of the road image, then the vanishing point is obtained based on these two lines and the selected first and second key points on the line. If both the first and second key points correspond to multiple lines, then the vanishing point is obtained based on the paired lines, and the final vanishing point is determined from the multiple vanishing points. The specific determination process is described below.

[0127] It should be noted that the aforementioned equal distances are determined based on vehicle model information. Vehicles of the same model may have some identical parameters (which may have minor errors, negligible in this invention), such as the wheelbase and the distance between the side mirrors. In this invention, generally, the position of the first key point in the first vehicle corresponds to the position of the second key point in the second vehicle, such as the key point at the rear of the two vehicles. There are exceptions; the positions of the first and second key points in the first and second vehicles may not correspond. For example, when the first and second vehicles are traveling in the same direction, the rear wheel key points of the vehicles correspond, and the left and right side mirrors of the first vehicle correspond to the left and right side mirrors of the second vehicle, etc. When the first and second vehicles are traveling in opposite directions, the front wheel key point of the first vehicle (assuming the rear wheel key point of the first vehicle is obscured and not detected) corresponds to the rear wheel key point of the second vehicle, the left side mirror of the first vehicle corresponds to the right side mirror of the second vehicle, the right side mirror of the first vehicle corresponds to the left side mirror of the second vehicle, and so on.

[0128] The vanishing point acquisition module 505 is used to obtain the vanishing point corresponding to the vehicle-mounted camera based on the first key point and the second key point, utilizing cross-ratio invariance. The vanishing point is one of the extrinsic parameters of the vehicle-mounted camera. After obtaining the vanishing point, other extrinsic parameters such as the camera's pitch angle can be further determined based on it. The determination of other extrinsic parameters utilizes commonly used related vehicle-mounted camera extrinsic parameter calibration algorithms, which are not limited in this invention. Furthermore, after obtaining all the extrinsic parameter calibration data of the vehicle-mounted camera, the calibration data can be used for vehicle speed or distance measurement to complete the forward-looking task in autonomous driving.

[0129] In this module, the vanishing point is obtained based on the determined key points and through cross-ratio invariance.

[0130] The application of cross-ratio invariance in vanishing point acquisition is explained below. Figure 2 Taking points a”, b”, c”, a’, b’, and c’ (these six points are not the same as the first and second key points mentioned in this invention; they are only used to illustrate the principle of cross-ratio invariance) as an example, the vanishing point is obtained based on the length ratio. The specific steps are as follows:

[0131] 1) Identify three collinear points a', b', and c' in the image. In the world coordinate system, the ratio of the intervals between these three points is A:B.

[0132] 2) Draw a straight line l that passes through point a'. This straight line l does not coincide with the straight line n that passes through points a', b', and c'. Take points a”, b”, and c” on the straight line l, and denote point a” = a'. In the world coordinate system, the distance between the three points a” and b” is equal to the distance between point b” and c”, which satisfies A:B.

[0133] 3) Obtain the line passing through points b” and b’, and the line passing through points c” and c’, and then obtain the intersection point o of the two lines.

[0134] 4) Obtain a line m that passes through the intersection point o and is parallel to the line l. Determine the intersection point v' of the line m and the line n as the vanishing point.

[0135] The vanishing point is obtained from the road surface image using the cross-ratio invariance described above. It should be noted that one vanishing point corresponds to one pair of target vehicles, and multiple vanishing points will be obtained when multiple pairs of target vehicles exist. In this case, the final vanishing point can be obtained by averaging the coordinates of multiple vanishing points; alternatively, it can be determined from multiple vanishing points using preset filtering conditions (such as the target vehicle pair closest to the vehicle); or it can be determined by combining vanishing points obtained through lane lines or other conventional vanishing point acquisition methods from the multiple vanishing points obtained by this invention. This invention does not limit the specific methods used.

[0136] The extrinsic parameter calibration device for a vehicle-mounted camera provided according to an embodiment of the present invention obtains a first key point and a second key point from a target vehicle of the same model. In the world coordinate system, the distance between the two first key points is equal to the distance between the two second key points, and the straight line passing through the first key point and the straight line passing through the second key point are parallel to the bottom or top edge of the road surface image. Based on the above-mentioned first key point and second key point, the vanishing point is obtained by utilizing the cross ratio invariance in projective transformation. Then, the extrinsic parameter calibration of the vehicle-mounted camera is completed based on the vanishing point. The extrinsic parameter calibration method of the present invention does not rely on lane line information. It only needs to obtain key points that meet the conditions from vehicles of the same model to obtain the vanishing point. It can still effectively complete the extrinsic parameter calibration of the vehicle-mounted camera in various scenarios such as complex lane lines, intersections, rural roads, and mining areas, thereby improving the generalization ability of the extrinsic parameter calibration of the vehicle-mounted camera in various scenarios.

[0137] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logic instructions in the memory 630 to execute the extrinsic parameter calibration method for the vehicle-mounted camera provided above. This method includes: acquiring a road surface image using a vehicle-mounted camera in the vehicle; acquiring vehicle model information and multiple key points of the target vehicle from the road surface image, wherein the target vehicle is a vehicle surrounding the vehicle, and the road surface image includes multiple target vehicles; acquiring a pair of target vehicles from the multiple target vehicles based on the vehicle model information, wherein the pair of target vehicles includes a first vehicle and a second vehicle; selecting key points corresponding to the first vehicle and the second vehicle according to preset key point selection conditions to obtain at least two first key points corresponding to the first vehicle and at least two second key points corresponding to the second vehicle; and obtaining the vanishing point corresponding to the vehicle-mounted camera based on the first key points and the second key points using cross-ratio invariance; wherein the preset key point selection conditions are: selecting key points from the multiple key points that satisfy the condition that the line connecting the key points is parallel to the bottom or top edge of the road surface image, and that, in the world coordinate system, the distance between key points in the first vehicle is equal to the distance between key points in the second vehicle.

[0138] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0139] On the other hand, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the above-described extrinsic parameter calibration method for an in-vehicle camera. The method includes: acquiring a road surface image using an in-vehicle camera in a vehicle; acquiring vehicle model information and multiple key points of a target vehicle from the road surface image, wherein the target vehicle is a vehicle surrounding the vehicle, and the road surface image includes multiple target vehicles; acquiring a target vehicle pair from the multiple target vehicles based on the vehicle model information, wherein the target vehicle pair includes a first vehicle and a second vehicle; selecting key points corresponding to the first vehicle and the second vehicle according to preset key point selection conditions to obtain at least two first key points corresponding to the first vehicle and at least two second key points corresponding to the second vehicle; and acquiring the vanishing point corresponding to the in-vehicle camera based on the first key points and the second key points using cross-ratio invariance; wherein the preset key point selection conditions are that key points are selected from the multiple key points to satisfy the condition that the line connecting the key points is parallel to the bottom or top edge of the road surface image, and that the distance between the key points in the first vehicle is equal to the distance between the key points in the second vehicle in the world coordinate system.

[0140] On the other hand, embodiments of the present invention also provide a vehicle, which includes the electronic equipment provided in the foregoing embodiments. The vehicle provided in the embodiments of the present invention has the same implementation principle and the same technical effects as those in the foregoing method embodiments, and will not be repeated here.

[0141] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the above-described extrinsic parameter calibration method for an in-vehicle camera. The method includes: acquiring a road surface image using an in-vehicle camera; acquiring vehicle model information and multiple key points of a target vehicle from the road surface image, wherein the target vehicle is a vehicle surrounding the vehicle, and the road surface image includes multiple target vehicles; and acquiring a target vehicle pair from the multiple target vehicles based on the vehicle model information, wherein the target vehicle pair includes a first vehicle. The first vehicle and the second vehicle are selected according to preset key point selection conditions to obtain at least two first key points corresponding to the first vehicle and at least two second key points corresponding to the second vehicle; based on the first key points and the second key points, the vanishing point corresponding to the vehicle camera is obtained using cross ratio invariance; wherein, the preset key point selection conditions are to select key points from the plurality of key points that satisfy the condition that the line connecting the key points is parallel to the bottom edge or top edge of the road surface image, and in the world coordinate system, the distance between key points in the first vehicle is equal to the distance between key points in the second vehicle.

[0142] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0143] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for calibrating the extrinsic parameters of a vehicle-mounted camera, characterized in that, include: Use the vehicle's onboard camera to acquire road surface images; The vehicle type information and multiple key points of the target vehicle are obtained from the road surface image. The target vehicle refers to the vehicles surrounding the vehicle. The road surface image includes multiple target vehicles. When the vehicle models of the target vehicles are all different, multiple road surface images are acquired. The longitudinal position of the same target vehicle changes in the multiple road surface images, and the road surface differences between the multiple road surface images are small, so as to support target detection and key point detection in the merged image. The multiple road surface images are merged to obtain a merged image; The vehicle model information and several key points of the target vehicle are obtained from the merged image. Based on the vehicle model information, target vehicle pairs are obtained from the plurality of target vehicles. The target vehicle pairs include a first vehicle and a second vehicle. The teaming method for the target vehicle pairs is to team up two target vehicles with the same vehicle model information. Based on preset key point selection conditions, key points corresponding to the first vehicle and the second vehicle are selected to obtain at least two first key points corresponding to the first vehicle and at least two second key points corresponding to the second vehicle. Based on the first key point and the second key point, the vanishing point corresponding to the vehicle camera is obtained by utilizing the cross ratio invariance. The preset key point selection criteria are as follows: from the plurality of key points, key points are selected that satisfy the condition that the line connecting the key points is parallel to the bottom or top edge of the road surface image, and in the world coordinate system, the distance between key points in the first vehicle is equal to the distance between key points in the second vehicle. The distance between key points in the first vehicle and the distance between key points in the second vehicle are calculated based on the coordinate values ​​of each key point in the world coordinate system. The number of road surface images is at least two frames, and the first vehicle and the second vehicle in the merged image are the same vehicle; The line connecting the key points is parallel to the bottom or top edge of the road surface image, with an error of up to 2 degrees allowed. By combining the vanishing point obtained through the lane lines with the vanishing point obtained using the first key point and the second key point, the final vanishing point is obtained to complete the external parameter calibration.

2. The extrinsic parameter calibration method for a vehicle-mounted camera according to claim 1, characterized in that, When there are two key points for both the first key point and the second key point, and the first key point is a and b, and the second key point is c and d, wherein the first key point a corresponds to the second key point c, and the first key point b corresponds to the second key point d; the distance between the first vehicle and the self-vehicle is less than the distance between the second vehicle and the self-vehicle. The step of obtaining the vanishing point corresponding to the vehicle-mounted camera based on the first key point and the second key point using cross-ratio invariance includes: Based on the positional relationship between the second key point c and the first key point a, and the distance between the first key point a and the first key point b, select equidistant points f on the straight lines corresponding to the first key point a and the first key point b; Obtain the intersection point g between the line corresponding to the equidistant point f and the second key point d and the line corresponding to the first key point b and the second key point c; Obtain the shadow line parallel to the bottom or top edge of the road surface image based on the intersection point g; The vanishing point is obtained from the vanishing line based on the size information of the road surface image.

3. The extrinsic parameter calibration method for a vehicle-mounted camera according to claim 1, characterized in that, The target vehicle pair consists of multiple target vehicle pairs, and the step of obtaining the vanishing point corresponding to the vehicle-mounted camera based on the first key point and the second key point using cross ratio invariance includes: For each of the plurality of target vehicle pairs, the vanishing point corresponding to the target vehicle pair is obtained by utilizing the cross ratio invariance based on the first key point and the second key point corresponding to the target vehicle pair. The vanishing point corresponding to the vehicle-mounted camera is obtained by averaging the vanishing points corresponding to the multiple target vehicle pairs.

4. The extrinsic parameter calibration method for a vehicle-mounted camera according to claim 1, characterized in that, The step of obtaining target vehicle pairs from the plurality of target vehicles based on the vehicle model information includes: Group target vehicles with the same vehicle model information into multiple pairs of target vehicles. The vehicle type information is at least one of the following: small car, micro car, compact car, mid-range car, high-end car, luxury car, sedan, CDV, MPV, or SUV.

5. The extrinsic parameter calibration method for a vehicle-mounted camera according to any one of claims 1-4, characterized in that, The first key point and the second key point are the key points for the external ground contact of the wheel.

6. A device for calibrating the external parameters of a vehicle-mounted camera, characterized in that, include: The image acquisition module is used to acquire road surface images using the vehicle's onboard camera. The target vehicle information acquisition module is used for: The vehicle type information and multiple key points of the target vehicle are obtained from the road surface image. The target vehicle refers to the vehicles surrounding the vehicle. The road surface image includes multiple target vehicles. When the vehicle models of the target vehicles are all different, multiple road surface images are acquired. The longitudinal position of the same target vehicle changes in the multiple road surface images, and the road surface differences between the multiple road surface images are small, so as to support target detection and key point detection in the merged image. The multiple road surface images are merged to obtain a merged image; The vehicle model information and several key points of the target vehicle are obtained from the merged image. The vehicle pair determination module is used to obtain target vehicle pairs from the plurality of target vehicles based on the vehicle model information. The target vehicle pair includes a first vehicle and a second vehicle. The grouping method of the target vehicle pair is to group target vehicles with the same vehicle model information into pairs. The key point selection module is used to select key points corresponding to the first vehicle and the second vehicle according to preset key point selection conditions, and obtain at least two first key points corresponding to the first vehicle and at least two second key points corresponding to the second vehicle. The vanishing point acquisition module is used to acquire the vanishing point corresponding to the vehicle camera based on the first key point and the second key point, using the cross ratio invariance. The preset key point selection criteria are as follows: from the plurality of key points, key points are selected that satisfy the condition that the line connecting the key points is parallel to the bottom or top edge of the road surface image, and in the world coordinate system, the distance between key points in the first vehicle is equal to the distance between key points in the second vehicle. The distance between key points in the first vehicle and the distance between key points in the second vehicle are calculated based on the coordinate values ​​of each key point in the world coordinate system. The number of road surface images is at least two frames, and the first vehicle and the second vehicle in the merged image are the same vehicle; The line connecting the key points is parallel to the bottom or top edge of the road surface image, with an error of up to 2 degrees allowed. By combining the vanishing point obtained through the lane lines with the vanishing point obtained using the first key point and the second key point, the final vanishing point is obtained to complete the external parameter calibration.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the extrinsic parameter calibration method for the vehicle-mounted camera as described in any one of claims 1-5.

8. A vehicle, characterized in that, Includes the electronic device as described in claim 7.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the extrinsic parameter calibration method for vehicle-mounted cameras as described in any one of claims 1-5.

Citation Information

Patent Citations

  • Vehicle key point detection method, electronic equipment and storage medium

    CN113869323A

  • Vehicle pose measurement method and device, electronic equipment and storage medium

    CN114705121A