Camera extrinsic parameter acquisition method and device, electronic equipment and storage medium

Abstract

The application provides a camera external parameter acquisition method, which comprises the following steps: acquiring a first image including a lane line, which is shot by a vehicle-mounted camera; determining the coordinates of the lane line in the image plane coordinate system and the image blanking point coordinates of the first image in the image plane coordinate system; constructing a mapping relationship model of the first image in the world coordinate system in the image plane coordinate system, and determining the corresponding target projection point of the target point in the first image in the world coordinate system; determining a first relationship and a second relationship, wherein the first relationship is the relationship between the longitudinal distance from the target projection point to the camera optical center and the camera height; determining a third relationship according to the second relationship; and calculating the camera height by taking the ratio of the expressions of the lane widths of two different lanes and combining the first relationship. According to the method, the calculated camera height is more accurate, the process is relatively simple, and the method does not need to depend on specific targets and scenes, so that the robustness and generalization are better.

Description

Technical Field

[0001] This invention relates to the field of driver assistance system technology, and in particular to a method, apparatus, electronic device, and storage medium for acquiring camera extrinsic parameters. Background Technology

[0002] Advanced Driver Assistance Systems (ADAS) utilize various sensors (cameras, navigation systems, and radar, etc.) installed in vehicles to collect environmental data inside and outside the vehicle in a timely manner. They perform technical processing such as identification, detection, and tracking of static and dynamic objects, enabling drivers to detect potential safety hazards in the shortest possible time and thus improve driving safety.

[0003] Camera extrinsic parameters typically include rotation angles and translation vectors. In ADAS, pitch angle, yaw angle, and camera altitude are generally of primary concern. Without accurate camera extrinsic parameters, neither ranging nor inverse perspective transformation can achieve good results, leading to poor user experience and numerous false alarms in key ADAS functions. Currently, methods for obtaining camera extrinsic parameters rely heavily on specific targets, camera calibration, or training data. Target-dependent methods estimate camera extrinsic parameters using objects with known properties (e.g., length, width, height) such as buildings, vehicles, and license plates on the road. These methods fail when the driving scene does not contain these specific object types, exhibiting poor robustness. Training data-dependent methods primarily learn camera extrinsic parameters through CNN networks, heavily dependent on the scene. Significant changes in lighting, camera imaging, or scene conditions result in large errors and poor generalization. Camera calibration-dependent methods require obtaining highly accurate camera intrinsic parameters, but the calibration process is cumbersome and complex, hindering practical applications. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, and storage medium for acquiring camera extrinsic parameters, which solves the problems of large errors, poor robustness and generalization, and cumbersome operation in existing methods for acquiring camera extrinsic parameters. It achieves high-precision and simple acquisition of camera extrinsic parameters, and has good robustness and generalization.

[0005] This invention provides a method for obtaining camera extrinsic parameters, the method comprising:

[0006] Acquire the first image, including lane lines, captured by the vehicle's onboard camera;

[0007] Determine the coordinates of the lane lines in the first image in the image plane coordinate system and the coordinates of the hidden point of the first image in the image plane coordinate system;

[0008] Construct a mapping relationship model between the first image in the image plane coordinate system and the world coordinate system, and determine the target projection point corresponding to the target point in the first image in the world coordinate system, wherein the target point is any point selected in the first image;

[0009] Based on the coordinates of the target point in the image plane coordinate system, the coordinates of the center point of the first image, the coordinates of the hidden point of the image, and the camera intrinsic parameters, a first relation and a second relation are determined. The first relation is the relationship between the longitudinal distance from the target projection point to the camera optical center and the camera height. The second relation is the relationship between the lateral distance from the target projection point to the lane line and the camera height. The lateral direction is the width direction along the lane line, and the longitudinal direction is the length direction along the lane line.

[0010] A third relationship is determined based on the second relationship, wherein the third relationship is the relationship between lane width and camera height;

[0011] The camera height is calculated by comparing the expressions for the lane widths of two different lanes according to the third relation and combining them with the first relation.

[0012] According to a camera extrinsic parameter acquisition method provided by the present invention, the step of calculating the camera height by comparing two expressions for lane widths of different lanes based on the third relation and combining them with the first relation includes:

[0013] Calculate the pitch angle based on the ordinate of the hidden point in the first image and the ordinate of the center point in the first image;

[0014] The yaw angle is calculated based on the x-coordinate of the hidden point in the first image and the x-coordinate of the center point in the first image.

[0015] According to a method for acquiring camera extrinsic parameters provided by the present invention, the step of constructing a mapping relationship model of the first image in the image plane coordinate system and the world coordinate system includes:

[0016] Starting from the camera's optical center, draw lines to each point on the first image and extend them into the world coordinate system to obtain the image projection of the first image in the world coordinate system.

[0017] Determine the correspondence between points on the first image in the image plane coordinate system and points projected onto the image in the world coordinate system.

[0018] According to a method for obtaining camera extrinsic parameters provided by the present invention, determining a first relation based on the coordinates of a target point in the image plane coordinate system, the coordinates of the center point of a first image, the coordinates of the hidden point of the image, and the camera intrinsic parameters includes:

[0019] Based on the ordinates of the target point in the image coordinate system, the ordinates of the center point of the first image, the ordinates of the hidden point in the image, and the camera focal length, the first relation is determined according to the principle of inverse perspective transformation.

[0020] According to a method for obtaining camera extrinsic parameters provided by the present invention, determining a second relation based on the coordinates of a target point in the image plane coordinate system, the coordinates of the center point of a first image, the coordinates of the image hidden point, and camera intrinsic parameters includes:

[0021] Based on the coordinates of the target point in the image plane coordinate system, the coordinates of the center point of the first image, the ordinate of the camera's optical center in the world coordinate system, the first relational expression, and the camera's focal length, the second relational expression is determined according to the principle of inverse perspective transformation.

[0022] According to a method for obtaining camera extrinsic parameters provided by the present invention, the step of determining the third relation based on the second relation includes:

[0023] Based on the second relational expression, the distances from two adjacent lane lines to the target projection point are expressed respectively, and the absolute value of the difference between the two distance expressions is taken.

[0024] According to a method for acquiring camera extrinsic parameters provided by the present invention, the step of calculating the camera height by comparing the expressions for two different lane widths based on the third relation and combining them with the first relation includes:

[0025] When the camera is at the first height and the first lane is at the first width, the width of the first lane line is expressed according to the third relation. In the first image, any point is selected as the first target point, and the first longitudinal distance from the first projection point of the first target point in the world coordinate system to the optical center of the camera is calculated in combination with the first relation.

[0026] When the second lane is the second width, keep the first target point and center point in the first image unchanged, express the lane width of the second lane according to the third relation, and calculate the second longitudinal distance from the first projection point of the first target point in the world coordinate system to the optical center of the camera in combination with the first relation.

[0027] The ratio of the expression for the width of the first lane to the expression for the width of the second lane is used, and the height of the camera in the second lane is calculated by combining the known first height, first width of the first lane, first longitudinal distance, second width of the second lane, and second longitudinal distance.

[0028] The present invention also provides a camera extrinsic parameter acquisition device, the device comprising:

[0029] The acquisition module is used to acquire the first image, including lane lines, captured by the vehicle-mounted camera;

[0030] The first determining module is used to acquire a first image, and the coordinates of the lane lines in the first image in the image plane coordinate system and the coordinates of the image hidden point in the first image in the image plane coordinate system;

[0031] The construction module constructs a mapping relationship model of the first image in the image plane coordinate system and the world coordinate system, and determines the target projection point corresponding to the target point in the first image in the world coordinate system, wherein the target point is any point selected in the first image;

[0032] The second determining module determines a first relationship and a second relationship based on the coordinates of the target point in the image plane coordinate system, the coordinates of the center point of the first image, the coordinates of the image hidden point, and the camera intrinsic parameters. The first relationship is the relationship between the longitudinal distance from the target projection point to the camera optical center and the camera height. The second relationship is the relationship between the lateral distance from the target projection point to the lane line and the camera height. The lateral direction is the width direction along the lane line, and the longitudinal direction is the length direction along the lane line.

[0033] The third determining module is used to determine a third relation based on the second relation, wherein the third relation is the relationship between lane width and camera height.

[0034] The calculation module is used to compare the expressions for the lane widths of two different lanes according to the third relation, and combine them with the first relation to calculate the camera height.

[0035] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the camera extrinsic parameter acquisition method as described above.

[0036] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the camera extrinsic parameter acquisition method as described above.

[0037] The camera extrinsic parameter acquisition method provided by this invention obtains the coordinates of lane lines and hidden point coordinates of a first image in the image plane coordinate system, constructs a mapping relationship model, and combines the coordinates of the center point of the first image in the image plane coordinate system with camera intrinsic parameters to obtain the relationship between camera height and longitudinal distance, and the relationship between camera height and lane width. Then, it calculates the camera height by comparing the expressions for the lane widths of two different lanes and combining this with the relationship between camera height and longitudinal distance. The process of obtaining each coordinate and establishing the mapping relationship model is relatively simple, and the obtained coordinates are relatively accurate. Since the camera intrinsic parameters are known, the calculated camera height is relatively accurate. The process is also relatively simple, does not depend on specific targets or scenes, and has good robustness and generalization. This solves the problems of large errors in camera extrinsic parameter acquisition, poor robustness and generalization, and cumbersome and complex operation in existing technologies. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0039] Figure 1 This is one of the flowcharts illustrating the camera extrinsic parameter acquisition method provided by the present invention;

[0040] Figure 2 This is the second flowchart illustrating the camera extrinsic parameter acquisition method provided by the present invention;

[0041] Figure 3 The third flowchart illustrating the camera extrinsic parameter acquisition method provided by the present invention;

[0042] Figure 4 The fourth flowchart illustrating the camera extrinsic parameter acquisition method provided by the present invention;

[0043] Figure 5 The fifth flowchart illustrates the camera extrinsic parameter acquisition method provided by this invention.

[0044] Figure 6 This is the sixth flowchart illustrating the camera extrinsic parameter acquisition method provided by the present invention.

[0045] Figure 7 This is a model of the mapping relationship between lane lines in the first image within the image plane coordinate system and the world coordinate system.

[0046] Figure 8 for Figure 7 A magnified view of a portion of the image in the plane coordinate system;

[0047] Figure 9 This is a schematic diagram of the camera extrinsic parameter acquisition device provided by the present invention;

[0048] Figure 10 This is a schematic diagram of the structure of the electronic device provided by the present invention.

[0049] Figure label:

[0050] 910: Acquisition module; 920: First determination module; 930: Construction module; 940: Second determination module; 950: Third determination module; 960: Calculation module; 1010: Processor; 1020: Communication interface; 1030: Memory; 1040: Communication bus. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0052] The following is combined Figures 1-10 The present invention describes a method, apparatus, electronic device, and storage medium for acquiring camera extrinsic parameters.

[0053] like Figure 1 As shown, in one embodiment, the present invention provides a method for obtaining camera extrinsic parameters, comprising the following steps:

[0054] Step S110: Acquire a first image captured by the vehicle-mounted camera, including lane lines.

[0055] The first image is a virtual image of the lane lines captured by the camera. When the camera captures the lane lines, the imaging plane is behind the camera lens, and the lane line image on the imaging plane is an inverted real image. The first image is a virtual image on the equivalent imaging plane, that is, an upright lane line image located in front of the camera lens.

[0056] Step S120: Determine the coordinates of the lane lines in the first image in the image plane coordinate system and the coordinates of the image hidden point in the first image in the image plane coordinate system.

[0057] The image plane coordinate system is the plane coordinate system in which the first image is located.

[0058] Specifically, after acquiring the first image, lane lines in the first image are detected to obtain their coordinates in a planar coordinate system. After obtaining the lane line coordinates in this planar coordinate system, the coordinates of the hidden line points in the first image are calculated. Specifically, a lane line curve is fitted, and the lane line point coordinates are uniformly resampled; fitting parameters are fitted to the lane line straight line or tangent; the intersection points of any two lines are calculated based on these fitting parameters to obtain a set of intersection points, i.e., a point set; the average value of the set of intersection points is calculated to obtain the coordinates of the hidden line points.

[0059] Step S130: Construct a mapping relationship model between lane lines in the first image in the image plane coordinate system and lane lines in the world coordinate system, and determine the target projection point corresponding to the target point in the first image in the world coordinate system. The target point is any point selected in the first image.

[0060] According to this mapping model, the first image has a corresponding projection in the world coordinate system. This means that every point in the first image, including points on the lane lines, has a corresponding projection point in the world coordinate system. Conversely, within the camera's field of view, every point in the world coordinate system also has a corresponding point in the first image. The selected target point is not the center point of the first image.

[0061] Step S140: Based on the coordinates of the target point in the image plane coordinate system, the coordinates of the center point of the first image, the coordinates of the image hidden point, and the camera intrinsic parameters, determine the first relation and the second relation. The first relation is the relationship between the longitudinal distance from the target projection point to the camera optical center and the camera height. The second relation is the relationship between the lateral distance from the target projection point to the lane line and the camera height. The lateral direction is the width direction along the lane line, and the longitudinal direction is the length direction along the lane line.

[0062] The coordinates of the hidden point in the image are mainly determined by the lane lines and the camera height, while the camera intrinsic parameter is the camera focal length. The first equation allows us to represent the longitudinal distance using the camera height; similarly, the second equation allows us to represent the lateral distance using the camera height.

[0063] Step S150: Determine the third relationship based on the second relationship. The third relationship is the relationship between lane width and camera height.

[0064] Specifically, based on the second relation, the distances from each of two adjacent lane lines to the target projection point are expressed, and the absolute value of the difference between the two distance expressions is taken. The widths of two adjacent lane lines can be represented as the i-th lane line and the (i+1)-th lane line. Therefore, the lateral distance between the i-th lane line and the target projection point can be represented as D. i The lateral distance between the (i+1)th lane line and the target projection point is denoted as D. i+1Then the width of the two adjacent lane lines is |D i -D i+1 |

[0065] Step S160: Compare the expressions for the lane widths of two different lanes according to the third relation, and calculate the camera height by combining them with the first relation.

[0066] The camera extrinsic parameter acquisition method of the present invention obtains the coordinates of the lane lines and the coordinates of the hidden point of the first image in the image plane coordinate system, and constructs a mapping relationship model. Combining the coordinates of the center point of the first image in the image plane coordinate system and the camera intrinsic parameters, it obtains the relationship between the camera height and the longitudinal distance, and the relationship between the camera height and the lane width. Then, it calculates the camera height by comparing the expressions of the lane widths of two different lanes and combining the relationship between the camera height and the longitudinal distance. The process of obtaining each coordinate and establishing the mapping relationship model is relatively simple, and the obtained coordinates are relatively accurate. The camera intrinsic parameters are known, so the calculated camera height is relatively accurate. The process is relatively simple, does not depend on specific targets and scenes, and has good robustness and generalization. It solves the problems of large errors in obtaining camera extrinsic parameters, poor robustness and generalization, and cumbersome and complicated operation in the prior art.

[0067] like Figure 2 As shown, in one embodiment, the camera height is calculated by comparing two expressions for the lane widths of different lanes according to the third relation, and then combined with the first relation. The following steps are then included:

[0068] Step S210: Calculate the pitch angle based on the ordinate of the hidden point of the first image and the ordinate of the center point of the first image.

[0069] Step S220: Calculate the yaw angle based on the x-coordinate of the image hidden point of the first image and the x-coordinate of the center point of the first image.

[0070] Specifically, in ADAS, pitch angle, yaw angle, and camera altitude are generally of primary concern. After calculating the camera altitude, the pitch angle and yaw angle are calculated based on the measured coordinates of each point. The calculation results are relatively accurate, the process is relatively simple, and the robustness and generalization are good.

[0071] like Figure 3 As shown, in one embodiment, constructing a mapping model of the first image in the image plane coordinate system and the world coordinate system includes the following steps:

[0072] Step S122: Starting from the camera optical center, draw lines to each point on the first image and extend them into the world coordinate system to obtain the image projection of the first image in the world coordinate system.

[0073] Specifically, starting from the camera's optical center, lines are drawn to each point on the first image within the image plane coordinate system, and then extended to the world coordinate system. This completes the connection and extension of all points in the first image, resulting in an image projection within the world coordinate system. This image projection includes lane line projection.

[0074] Step S124: Determine the correspondence between points on the first image in the image plane coordinate system and points on the image projection in the world coordinate system.

[0075] Specifically, the image projection in the world coordinate system has corresponding points on the first image in the image plane coordinate system. The correspondence between each point on the image projection and each point on the first image is the mapping relationship model.

[0076] like Figure 4 As shown, in one embodiment, based on the coordinates of the target point in the image plane coordinate system, the coordinates of the center point of the first image, the coordinates of the image hidden point, and the camera intrinsic parameters, a first relation and a second relation are determined. The first relation is the relationship between the longitudinal distance from the target projection point to the camera optical center and the camera height. The second relation is the relationship between the lateral distance from the target projection point to the lane line and the camera height. The lateral direction is the width direction along the lane line, and the longitudinal direction is the length direction along the lane line. The process includes the following steps:

[0077] Step S142: Based on the ordinate of the target point in the image coordinate system, the ordinate of the center point of the first image, the ordinate of the hidden point of the image, and the camera focal length, determine the first relational expression according to the principle of inverse perspective transformation.

[0078] Step S144: Based on the coordinates of the target point in the image plane coordinate system, the coordinates of the center point of the first image, the ordinate of the camera optical center in the world coordinate system, the first relational expression, and the camera focal length, determine the second relational expression according to the principle of inverse perspective transformation.

[0079] The process of obtaining the coordinates of the center point and the hidden point of the first image in the image plane coordinate system, the camera coordinates in the world coordinate system, and the coordinates of the target projection point in the world coordinate system determined by the mapping relationship model is relatively simple and accurate. The camera intrinsic parameters are the original parameters of the camera, so the calculated camera extrinsic parameters are relatively accurate and the measurement process is also relatively simple.

[0080] like Figure 5 As shown, in one embodiment, the camera height is calculated by comparing the expressions for two different lane widths according to the third relation and combining them with the first relation, including the following steps:

[0081] Step S152: When the camera is at the first height and the first lane is at the first width, the width of the first lane line is expressed according to the third relation. Any point is selected in the first image as the first target point, and the first longitudinal distance from the first projection point of the first target point in the world coordinate system to the optical center of the camera is calculated in combination with the first relation.

[0082] Step S154: When the second lane is the second width, keep the first target point in the first image unchanged, express the lane width of the second lane according to the third relation, and calculate the second longitudinal distance from the first projection point of the first target point in the world coordinate system to the optical center of the camera in combination with the first relation.

[0083] Step S156: Compare the expression for the lane width of the first lane with the expression for the lane width of the second lane, and calculate the height of the camera in the second lane by combining the known first height, first width of the first lane, first longitudinal distance, second width of the second lane, and second longitudinal distance.

[0084] The first height and the width of the first lane are used as references, the width of the second lane is a known constant, and the height of the camera in the second lane is the camera height to be acquired.

[0085] like Figure 6 As shown, in a specific embodiment, the camera extrinsic parameter acquisition method of the present invention includes the following steps:

[0086] Step S610, lane line detection.

[0087] Lane detection can be achieved using segmentation algorithms, point regression-based deep learning networks, and other methods.

[0088] Specifically, this can be achieved using the Ultra Fast Lane Detection (UFLD) algorithm. This algorithm divides the image into m*n grid cells, each containing two attributes: the presence or absence of lane lines and whether they belong to the same lane. The model output is b*(m+1)*n, where b is the number of lane lines. b*1*n is used to determine if each lane line exists in each row of the image, and b*m*n is used to obtain the specific position of each lane line in each row. Lane line detection allows the acquisition of the coordinates of each lane line, facilitating the subsequent calculation of hidden point removal in the image.

[0089] Step S620: Calculate the hidden point of the image.

[0090] Specifically, this includes: 1) Fitting lane line curves to reduce the impact of outliers on lane line detection performance; 2) Uniformly resampling lane line point coordinates; 3) Fitting parameters of lane line straight lines or tangents, specifically by taking the bottom 1 / 3 points of the lane line for straight line fitting, and calculating the straight line fitting parameters k and b; 4) Obtaining the set of intersection points of any two lines, specifically by calculating all intersection points of any two lines based on the k and b parameters, thus obtaining the intersection point set, i.e., the point set; 5) Taking the average value of the point set as the image hidden point.

[0091] Step S630: Obtain the camera height based on the principle of inverse perspective transformation.

[0092] For details, see Figure 7 and Figure 8 ,in, Figure 7 This is a model of the mapping relationship between lane lines in the first image within the image plane coordinate system and the world coordinate system. Figure 8 for Figure 7 The image shows a magnified view in a plane coordinate system. Here, u and v represent the x-coordinate and y-coordinate, respectively. The symbols in the image have the following meanings:

[0093] O c : Camera optical center; o: Center of the first image; vp: Image hidden point; p: Any point on the line containing vp and q; q: Bottom center point of the first image; l1, l2: Two lane lines in the first image; e: Intersection of l1 and the bottom of the first image; h: Height of the camera optical center above the ground; L1, L2: Two lane lines in the world coordinate system; H: Hidden point in the world coordinate system; E: Projection point of e in the world coordinate system; Q: Projection point of q in the world coordinate system; G: Projection point of o in the world coordinate system; P: Projection point of p in the world coordinate system; d1: Distance between the camera and point Q in the world coordinate system; d2: Distance between the camera and point P in the world coordinate system, i.e., vertical distance; D: Distance between the bottom point of the lane line and point Q in the world coordinate system, i.e., horizontal distance; D r : True lateral distance; α, β, These are the angles between pP, oG, qQ and the horizontal plane, respectively.

[0094] Depend on Figure 6 The mapping relationship model, combined with trigonometric function relationships, yields:

[0095]

[0096]

[0097]

[0098] Among them, v o Let o be the vertical axis, and v be the vertical axis. vpv is the ordinate of the hidden line disappearance point in the image. p Let p be the ordinate of point p; height represents the image height, v o = height / 2, where f represents the camera's focal length.

[0099] From equations (2) and (3), we can obtain:

[0100]

[0101] The longitudinal distance d2 can be calculated from equation (4):

[0102]

[0103] To facilitate understanding and calculation of the lateral distance, we assume that point P coincides with point Q, and that at point O... c The EQ plane is obtained based on the property of similar triangles along parallel lines:

[0104]

[0105] Furthermore, we can consider point p as a set of points on a straight line with the same x-coordinate or y-coordinate. The trigonometric function relationship in formula (1) only involves the y-coordinate, so point p can be considered as a point on the line connecting vp and q. Similarly, e belongs to lane line l1, and the image can be considered to be composed of countless lines parallel to l1, of which l1 is any one. Point e is considered as a point with the same x-coordinate as point p, from which we can obtain:

[0106]

[0107] Among them, u p Let p be the x-coordinate, and u be the x-coordinate. o Let be the x-coordinate of point o.

[0108] Substituting equation (7) into equation (6), the lateral distance D can be obtained:

[0109]

[0110] Set h to a constant such as 2m, calculate the lateral distance from each lane line to the camera, that is, the lateral distance from the lane line to the target projection point. Let the lateral distance be Di. Therefore, the lane width is w = |Di - Di + 1|.

[0111] Since the normal lane width W in China is 3.75m, if the selected point and center point remain unchanged, adjusting the camera height H to a suitable position will result in a measured lane width of W = 3.75m. In this case, w / W can be equivalently expressed as:

[0112]

[0113] By setting w / W = 2 / 3.75, the value of the camera height H can be obtained.

[0114] The camera pitch angle γ can be calculated based on the camera focal length f and the image hidden point:

[0115]

[0116] Among them, v vp y is the ordinate of the hidden point vp in the image.

[0117] The camera yaw angle θ can be calculated based on the camera focal length f and the image hidden point:

[0118]

[0119] Among them, u vp Let vp be the x-coordinate of the hidden point in the image.

[0120] The camera extrinsic parameter acquisition device provided by the present invention is described below. The camera extrinsic parameter acquisition device described below can be referred to in correspondence with the camera extrinsic parameter acquisition method described above.

[0121] like Figure 9 As shown, in one embodiment, the present invention also provides a camera extrinsic parameter acquisition device, the device comprising:

[0122] The acquisition module 910 is used to acquire a first image, including lane lines, captured by the vehicle-mounted camera.

[0123] The first determining module 920 is used to determine the coordinates of the lane lines in the first image in the image plane coordinate system and the coordinates of the image hidden point of the first image in the image plane coordinate system.

[0124] The construction module 930 constructs a mapping relationship model of the first image in the image plane coordinate system and the world coordinate system, and determines the target projection point corresponding to the target point in the first image in the world coordinate system. The target point is any point selected in the first image.

[0125] The second determining module 940 determines a first relation and a second relation based on the coordinates of the target point in the image plane coordinate system, the coordinates of the center point of the first image, the coordinates of the image hidden point, and the camera intrinsic parameters. The first relation is the relationship between the longitudinal distance from the target projection point to the camera optical center and the camera height. The second relation is the relationship between the lateral distance from the target projection point to the lane line and the camera height. The lateral direction is the width direction along the lane line, and the longitudinal direction is the length direction along the lane line.

[0126] The third determining module 950 is used to determine a third relation based on the second relation, wherein the third relation is the relationship between lane width and camera height.

[0127] The calculation module 960 is used to compare the expressions of the lane widths of two different lanes according to the third relation, and calculate the camera height by combining the first relation.

[0128] In this embodiment, the calculation module is also used for:

[0129] Calculate the pitch angle based on the ordinate of the hidden point in the first image and the ordinate of the center point in the first image;

[0130] The yaw angle is calculated based on the x-coordinate of the hidden point in the first image and the x-coordinate of the center point in the first image.

[0131] In this embodiment, the construction module is specifically used for:

[0132] Starting from the camera's optical center, draw lines to each point on the first image and extend them into the world coordinate system to obtain the image projection of the first image in the world coordinate system.

[0133] Determine the correspondence between points on the first image in the image plane coordinate system and points projected onto the image in the world coordinate system.

[0134] In this embodiment, the first determining module is specifically used for:

[0135] Based on the ordinates of the target point in the image coordinate system, the ordinates of the center point of the first image, the ordinates of the hidden point in the image, and the camera focal length, the first relation is determined according to the principle of inverse perspective transformation.

[0136] Based on the coordinates of the target point in the image plane coordinate system, the coordinates of the center point of the first image, the ordinate of the camera optical center in the world coordinate system, the second relational expression, and the camera focal length, the second relational expression is determined according to the principle of inverse perspective transformation.

[0137] In this embodiment, the calculation module is specifically used for:

[0138] When the camera is at the first height and the first lane is at the first width, the width of the first lane line is expressed according to the third relation. In the first image, any point is selected as the first target point, and the first longitudinal distance from the first projection point of the first target point in the world coordinate system to the optical center of the camera is calculated in combination with the first relation.

[0139] When the second lane is the second width, keep the first target point and center point in the first image unchanged, express the lane width of the second lane according to the third relation, and calculate the second longitudinal distance from the first projection point of the first target point in the world coordinate system to the optical center of the camera in combination with the first relation.

[0140] The ratio of the expression for the width of the first lane to the expression for the width of the second lane is used, and the height of the camera in the second lane is calculated by combining the known first height, first width of the first lane, first longitudinal distance, second width of the second lane, and second longitudinal distance.

[0141] The camera extrinsic parameter acquisition device of the present invention acquires the coordinates of lane lines and hidden point coordinates of a first image in the image plane coordinate system, and constructs a mapping relationship model. Combining the coordinates of the center point of the first image in the image plane coordinate system and the camera intrinsic parameters, it obtains the relationship between camera height and longitudinal distance, and the relationship between camera height and lane width. Then, it calculates the ratio based on the expressions for the lane widths of two different lanes, and combines this with the relationship between camera height and longitudinal distance. The process of acquiring each coordinate and establishing the mapping relationship model is relatively simple, and the acquired coordinates are relatively accurate. Since the camera intrinsic parameters are known, the calculated camera height is relatively accurate. The process is simple, does not depend on specific targets or scenes, and has good robustness and generalization. It solves the problems of large errors in camera extrinsic parameter acquisition, poor robustness and generalization, and cumbersome operation in existing technologies.

[0142] Figure 10 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 10 As shown, the electronic device may include: a processor 1010, a communication interface 920, a memory 1030, and a communication bus 1040, wherein the processor 1010, the communication interface 1020, and the memory 1030 communicate with each other through the communication bus 1040. The processor 1010 can call logical instructions in the memory 1030 to execute a camera extrinsic parameter acquisition method, which includes:

[0143] Acquire the first image, including lane lines, captured by the vehicle's onboard camera;

[0144] Determine the coordinates of the lane lines in the first image in the image plane coordinate system and the coordinates of the hidden point of the first image in the image plane coordinate system;

[0145] Construct a mapping model between lane lines in the first image in the image plane coordinate system and lane lines in the world coordinate system, and determine the target projection point corresponding to the target point in the first image in the world coordinate system, wherein the target point is any point selected in the first image;

[0146] Based on the coordinates of the target point in the image plane coordinate system, the coordinates of the center point of the first image, the coordinates of the hidden point of the image, and the camera intrinsic parameters, a first relation and a second relation are determined. The first relation is the relationship between the longitudinal distance from the target projection point to the camera optical center and the camera height. The second relation is the relationship between the lateral distance from the target projection point to the lane line and the camera height. The lateral direction is the width direction along the lane line, and the longitudinal direction is the length direction along the lane line.

[0147] A third relationship is determined based on the second relationship, wherein the third relationship is the relationship between lane width and camera height;

[0148] The camera height is calculated by comparing the expressions for the lane widths of two different lanes according to the third relation and combining them with the first relation.

[0149] Furthermore, the logical instructions in the aforementioned memory 1030 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0150] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to execute the camera extrinsic parameter acquisition method provided by the above methods, the method comprising:

[0151] Acquire the first image, including lane lines, captured by the vehicle's onboard camera;

[0152] Determine the coordinates of the lane lines in the first image in the image plane coordinate system and the coordinates of the hidden point of the first image in the image plane coordinate system;

[0153] Construct a mapping model between lane lines in the first image in the image plane coordinate system and lane lines in the world coordinate system, and determine the target projection point corresponding to the target point in the first image in the world coordinate system, wherein the target point is any point selected in the first image;

[0154] Based on the coordinates of the target point in the image plane coordinate system, the coordinates of the center point of the first image, the coordinates of the hidden point of the image, and the camera intrinsic parameters, a first relation and a second relation are determined. The first relation is the relationship between the longitudinal distance from the target projection point to the camera optical center and the camera height. The second relation is the relationship between the lateral distance from the target projection point to the lane line and the camera height. The lateral direction is the width direction along the lane line, and the longitudinal direction is the length direction along the lane line.

[0155] A third relationship is determined based on the second relationship, wherein the third relationship is the relationship between lane width and camera height;

[0156] The camera height is calculated by comparing the expressions for the lane widths of two different lanes according to the third relation and combining them with the first relation.

[0157] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the camera extrinsic parameter acquisition method provided by the methods described above, the method comprising:

[0158] Acquire the first image, including lane lines, captured by the vehicle's onboard camera;

[0159] Determine the coordinates of the lane lines in the first image in the image plane coordinate system and the coordinates of the hidden point of the first image in the image plane coordinate system;

[0160] Construct a mapping model between lane lines in the first image in the image plane coordinate system and lane lines in the world coordinate system, and determine the target projection point corresponding to the target point in the first image in the world coordinate system, wherein the target point is any point selected in the first image;

[0161] Based on the coordinates of the target point in the image plane coordinate system, the coordinates of the center point of the first image, the coordinates of the hidden point of the image, and the camera intrinsic parameters, a first relation and a second relation are determined. The first relation is the relationship between the longitudinal distance from the target projection point to the camera optical center and the camera height. The second relation is the relationship between the lateral distance from the target projection point to the lane line and the camera height. The lateral direction is the width direction along the lane line, and the longitudinal direction is the length direction along the lane line.

[0162] A third relationship is determined based on the second relationship, wherein the third relationship is the relationship between lane width and camera height;

[0163] The camera height is calculated by comparing the expressions for the lane widths of two different lanes according to the third relation and combining them with the first relation.

[0164] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0165] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0166] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for acquiring camera extrinsic parameters, characterized in that, The method includes: Acquire the first image, including lane lines, captured by the vehicle's onboard camera; Determine the coordinates of the lane lines in the first image in the image plane coordinate system and the coordinates of the hidden point of the first image in the image plane coordinate system; Construct a mapping relationship model between the first image in the image plane coordinate system and the world coordinate system, and determine the target projection point corresponding to the target point in the first image in the world coordinate system, wherein the target point is any point selected in the first image; Based on the coordinates of the target point in the image plane coordinate system, the coordinates of the center point of the first image, the coordinates of the hidden point of the image, and the camera intrinsic parameters, a first relation and a second relation are determined. The first relation is the relationship between the longitudinal distance from the target projection point to the camera optical center and the camera height. The second relation is the relationship between the lateral distance from the target projection point to the lane line and the camera height. The lateral direction is the width direction along the lane line, and the longitudinal direction is the length direction along the lane line. A third relationship is determined based on the second relationship, wherein the third relationship is the relationship between lane width and camera height; The camera height is calculated by comparing the expressions for the lane widths of two different lanes according to the third relation and combining them with the first relation.

2. The method for obtaining camera extrinsic parameters according to claim 1, characterized in that, The step involves comparing the two expressions for lane widths of different lanes based on the third relation, and then combining them with the first relation to calculate the camera height, followed by: Calculate the pitch angle based on the ordinate of the hidden point in the first image and the ordinate of the center point in the first image; The yaw angle is calculated based on the x-coordinate of the hidden point in the first image and the x-coordinate of the center point in the first image.

3. The method for obtaining camera extrinsic parameters according to claim 1, characterized in that, The construction of the mapping relationship model between the first image in the image plane coordinate system and the world coordinate system includes: Starting from the camera's optical center, draw lines to each point on the first image and extend them into the world coordinate system to obtain the image projection of the first image in the world coordinate system. Determine the correspondence between points on the first image in the image plane coordinate system and points projected onto the image in the world coordinate system.

4. The method for obtaining camera extrinsic parameters according to claim 1, characterized in that, The step of determining the first relation based on the coordinates of the target point in the image plane coordinate system, the coordinates of the center point of the first image, the coordinates of the hidden point in the image, and the camera intrinsic parameters includes: Based on the ordinates of the target point in the image coordinate system, the ordinates of the center point of the first image, the ordinates of the hidden point in the image, and the camera focal length, the first relation is determined according to the principle of inverse perspective transformation.

5. The method for obtaining camera extrinsic parameters according to claim 4, characterized in that, The step of determining the second relation based on the coordinates of the target point in the image plane coordinate system, the coordinates of the center point of the first image, the coordinates of the hidden point in the image, and the camera intrinsic parameters includes: Based on the coordinates of the target point in the image plane coordinate system, the coordinates of the center point of the first image, the ordinate of the camera's optical center in the world coordinate system, the first relational expression, and the camera's focal length, the second relational expression is determined according to the principle of inverse perspective transformation.

6. The method for obtaining camera extrinsic parameters according to claim 1, characterized in that, The step of determining the third relation based on the second relation includes: Based on the second relational expression, the distances from two adjacent lane lines to the target projection point are expressed respectively, and the absolute value of the difference between the two distance expressions is taken.

7. The method for obtaining camera extrinsic parameters according to claim 1, characterized in that, The step of comparing the expressions for two different lane widths according to the third relation and combining them with the first relation to calculate the camera height includes: When the camera is at the first height and the first lane is at the first width, the width of the first lane line is expressed according to the third relation. In the first image, any point is selected as the first target point, and the first longitudinal distance from the first projection point of the first target point in the world coordinate system to the optical center of the camera is calculated in combination with the first relation. When the second lane is the second width, keep the first target point and center point in the first image unchanged, express the lane width of the second lane according to the third relation, and calculate the second longitudinal distance from the first projection point of the first target point in the world coordinate system to the optical center of the camera in combination with the first relation. The ratio of the expression for the width of the first lane to the expression for the width of the second lane is used, and the height of the camera in the second lane is calculated by combining the known first height, first width of the first lane, first longitudinal distance, second width of the second lane, and second longitudinal distance.

8. A camera extrinsic parameter acquisition device, characterized in that, The device includes: The acquisition module is used to acquire the first image, including lane lines, captured by the vehicle-mounted camera; The first determining module is used to determine the coordinates of the lane lines in the first image in the image plane coordinate system and the coordinates of the image hidden point in the first image in the image plane coordinate system. The construction module constructs a mapping relationship model of the first image in the image plane coordinate system and the world coordinate system, and determines the target projection point corresponding to the target point in the first image in the world coordinate system, wherein the target point is any point selected in the first image; The second determining module determines a first relationship and a second relationship based on the coordinates of the target point in the image plane coordinate system, the coordinates of the center point of the first image, the coordinates of the image hidden point, and the camera intrinsic parameters. The first relationship is the relationship between the longitudinal distance from the target projection point to the camera optical center and the camera height. The second relationship is the relationship between the lateral distance from the target projection point to the lane line and the camera height. The lateral direction is the width direction along the lane line, and the longitudinal direction is the length direction along the lane line. The third determining module is used to determine a third relation based on the second relation, wherein the third relation is the relationship between lane width and camera height. The calculation module is used to compare the expressions for the lane widths of two different lanes according to the third relation, and combine them with the first relation to calculate the camera height.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the camera extrinsic parameter acquisition method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the camera extrinsic parameter acquisition method as described in any one of claims 1 to 7.

Images