A camera calibration method and device based on road features

Abstract

The application discloses a camera calibration method and device based on road features, which solves camera parameters by extracting road feature points on an image and corresponding world feature points. The extracted road feature points include first features (point features), second features (line features) and third features (surface features). For the second features and the third features, the application introduces feature parameters to describe the matching relationship between the image feature points and the world feature points. The camera parameters and the feature parameters jointly constitute the calibration parameters of the camera. The parameters are solved by a gradient optimization algorithm to complete camera calibration. The application overcomes the dependence of traditional calibration methods on matching point pairs, realizes a multi-dimensional calibration method based on point matching, line matching and surface matching, expands the application scenarios of the calibration method, and improves the accuracy and efficiency of camera calibration.

Description

Technical Field

[0001] This invention belongs to the fields of intelligent transportation and machine vision, and in particular, it is a camera calibration method and device based on road features. Background Technology

[0002] With the development of intelligent transportation systems, the coverage of various camera devices in urban road networks is expanding, and many cities have achieved full coverage of the entire road network, enabling dynamic monitoring and analysis of traffic conditions. Roadside perception, as a crucial sensing method in vehicle-road cooperative applications, uses sensors such as cameras, millimeter-wave radar, and lidar installed on the roadside to perceive road elements, compensating for the limitations of vehicle-side perception capabilities. However, currently, most intersection cameras lack the capability to serve vehicle-road cooperative applications, and one of the most critical factors is camera calibration. Most existing intersection cameras have not been calibrated or their calibration accuracy does not meet the positioning accuracy requirements for vehicle-road cooperative applications.

[0003] Existing camera calibration methods are based on matching point pairs. This involves selecting several feature points in the calibration image, obtaining their pixel coordinates, and then finding the world coordinates for each feature point. Methods for selecting matching point pairs include manual and automatic selection. A key assumption of these methods is the existence of sufficient typical feature points in the camera's image. For typical intersection scenarios, this assumption is easily met; for example, the corners of pedestrian crossings, lane turning markers, and lane start points can all serve as matching feature points. However, when these obvious key landmarks are lacking in traffic scenarios (e.g., road segment scenarios lack lane markers, pedestrian crossings, and lane start points are difficult to pinpoint), it is difficult to obtain the matching point pairs needed for calibration. In such scenarios, traditional methods are difficult to implement and suffer from positioning accuracy deviations. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by providing a camera calibration method and apparatus based on road features. This method overcomes the dependence of traditional calibration methods on matching point pairs and realizes a multi-dimensional calibration method based on point matching, line matching, and surface matching, thereby expanding the application scenarios of the calibration method and improving the accuracy and efficiency of camera calibration.

[0005] The technical solution for achieving the objective of this invention is: a camera calibration method based on road features, the method comprising the following steps:

[0006] A road scene image is acquired as a calibration image; the calibration image contains feature subjects on the road, from which a first feature, a second feature, or a third feature can be extracted, wherein the first feature corresponds to a feature point in the first feature subject, the second feature corresponds to a feature line in the second feature subject, and the third feature corresponds to a feature surface in the third feature subject;

[0007] The coordinates of image feature points are obtained based on the calibrated image and the feature subject.

[0008] Construct a feature subject model and obtain world feature point coordinates, wherein the world feature point coordinates correspond one-to-one with the image feature point coordinates; wherein the world feature point coordinates obtained by the first feature subject are the world coordinate points corresponding to the image feature points, and the world feature point coordinates obtained by the second and third feature subjects contain feature model parameters.

[0009] The loss function is defined using image feature point coordinates and world feature point coordinates;

[0010] Camera calibration is performed using a loss function to obtain camera parameters and feature model parameters.

[0011] Furthermore, obtaining the coordinates of image feature points includes:

[0012] Extract feature points of the first feature subject in the road from the calibration image, denoted as the first image feature point, and obtain the coordinates of the first image feature point;

[0013] Alternatively, extract the feature lines of the second feature subject in the road from the calibration image, denoted as the second image feature point, and obtain the coordinates of the second image feature point;

[0014] Alternatively, extract the feature surface of the third feature subject in the road from the calibration image; denote it as the third image feature point, and obtain the coordinates of the third image feature point.

[0015] Furthermore, the step of extracting feature points of the first feature subject in the road from the calibration image, denoted as the first image feature point, and obtaining the coordinates of the first image feature point, is as follows:

[0016] Obtain N primary feature entities from the calibration image;

[0017] M feature points are extracted from the N first feature subjects to form the first image feature points; wherein, the first first feature subject includes The first feature point contains one feature point, and the second first feature body contains... The first feature point contains..., the Nth first feature subject contains One feature point, ;

[0018] Extract the pixel coordinates of M feature points, which are the coordinates of the first image feature points.

[0019] Furthermore, the step of extracting the feature lines of the second feature subject in the road from the calibration image, denoted as the second image feature point, and obtaining the coordinates of the second image feature point is as follows:

[0020] Obtain P second feature entities from the calibration image;

[0021] Q feature lines (straight lines) are extracted from the P second feature subjects to form the second image features; wherein, the first second feature subject includes The first feature line, the second feature body contains The first feature line, ..., the Pth second feature body contains One characteristic line, ;

[0022] Discretize the Q feature lines to obtain S feature points, which constitute the second image feature points; wherein, the first feature line is discretized. The first feature point, the second feature line is discrete. There are 1 feature points, ..., the Qth feature line is discrete. One feature point, ;

[0023] Extract the pixel coordinates of S feature points, which are the coordinates of the second image feature points.

[0024] Furthermore, the feature surface of the third feature subject in the road is extracted from the calibration image; denoted as the third image feature point, and the coordinates of the third image feature point are obtained, as follows:

[0025] Obtain T third-feature entities from the calibration image;

[0026] V feature surfaces (planes) are extracted from the T third feature entities to form V third image features; wherein, the first third feature entity contains The second and third feature bodies contain a plane, and the main body contains a second and third feature. A plane, ..., the Tth third feature subject contains A plane, ;

[0027] Discretize each of the V feature surfaces to obtain U feature points, which constitute U third image feature points; wherein, the first feature surface is discretized The first feature point, the second feature surface is discrete. The Vth feature point is discrete, ..., the Vth feature surface is discrete. One feature point, ;

[0028] Extract the pixel coordinates of U feature points, which are the coordinates of the third image feature points.

[0029] Furthermore, the first feature includes one or more of road traffic signs and roadside infrastructure. The road traffic signs include one or more of turning signs, waiting areas, pedestrian crossings, and stop lines. The road infrastructure includes one or more of traffic light poles, electronic police poles, green belts, and traffic fences.

[0030] The second feature includes one or more of road traffic signs and road infrastructure. The road traffic signs include one or more of lanes, stop lines, and dividing lines. The road infrastructure includes one or more of traffic light poles, electronic police poles, green belts, traffic fences, and road edges.

[0031] The third feature includes one or more of the following: road surface, signboard, and building facade.

[0032] Furthermore, obtaining the coordinates of world feature points includes:

[0033] Obtain the world feature points corresponding to the M feature points, and obtain the coordinates of the M world feature points to form the first world feature point coordinates;

[0034] Sum or minus, obtain the second-world features corresponding to Q feature lines;

[0035] Discretize the second world feature to obtain the second world feature points, and obtain the coordinates of the second world feature points;

[0036] AND or OR, obtain the third-world features corresponding to V feature surfaces;

[0037] The third-world features are discretized to obtain third-world feature points, and the coordinates of the third-world feature points are obtained.

[0038] Furthermore, the second world feature is given by the following linear model:

[0039]

[0040]

[0041] in, This represents the i-th feature line in the second world features. Let be the coordinates of any point on the i-th feature line. Let be the unit vector of the direction of the i-th feature line;

[0042] The second-world feature points are obtained by discretization using the following equation:

[0043]

[0044] in, Let m be the m-th feature point obtained by discretizing the i-th feature line in the second world feature. These are second-world feature parameters, used together with camera parameters as calibration parameters;

[0045] The characteristics of the Third World are given by the following planar model:

[0046]

[0047]

[0048] in, This represents the i-th feature surface in the Third World features. Let be the coordinates of any point on the i-th plane. Let be two non-collinear unit vectors on the i-th plane;

[0049] Third-world feature points are obtained by discretization using the following equation:

[0050]

[0051] in, Let m be the m-th feature point obtained by discretizing the i-th feature surface in the third world features. and These are third-world feature parameters, used together with camera parameters as calibration parameters.

[0052] Furthermore, the loss function is composed of multiple factors including first-world feature point projection error, second-world feature point projection error, third-world feature point projection error, second-world feature parameter constraint error, third-world feature parameter constraint error, and camera parameter constraint error.

[0053] Among them, projection error It is given by the following equation:

[0054]

[0055] In the formula, These are the pixel coordinates of the feature points. These are projected coordinates. For Huber functions, These are the weighting coefficients;

[0056] The parameter constraint error is given by the following equation:

[0057]

[0058] In the formula, For characteristic parameters, The minimum value of the characteristic parameter. The maximum value of the characteristic parameter. The range constraint penalty function is given by the following equation:

[0059] .

[0060] in, For constraint variables, The lower boundary of the constraint variable, This is the upper boundary of the constraint variable.

[0061] A road feature-based camera calibration device based on the method described above, the device comprising:

[0062] The receiving unit is used to acquire a calibration image, which includes one or more of a first feature subject, a second feature subject, and a third feature subject;

[0063] The receiving unit is also used to receive a three-dimensional road model, including one or more of feature point position measurements and high-precision map models;

[0064] The processing unit is used to construct a line model in the second world features, and to discretize the second image features and the second world features to obtain the coordinates of the second image feature points and the second world feature points.

[0065] The processing unit is also used to construct a surface model in the third-world features, and to discretize the third image features and the third-world features to obtain the coordinates of the third image feature points and the coordinates of the third-world feature points;

[0066] The processing unit is also used to construct a loss function and use the loss function to calibrate the camera.

[0067] Compared with the prior art, the significant advantages of this invention are:

[0068] 1) In the present invention, when obtaining the coordinates of world feature points, unlike traditional methods that require obtaining the coordinates of specific world feature points that correspond one-to-one with the coordinates of image feature points, the present invention obtains the world features corresponding to the image features. For example, for a straight line feature, it is only necessary to obtain the world features corresponding to the straight line. Specifically, the world features of a straight line consist of the world coordinates of any point on the straight line and the direction vector of the straight line, rather than the world coordinates corresponding to each image feature point on the straight line. For a planar feature, it is only necessary to obtain the world features corresponding to the plane. Specifically, the world features of a planar plane consist of the world coordinates of any point on the plane and the direction vectors of two complementary parallel straight lines on the plane, rather than the world coordinates corresponding to each image feature point on the plane.

[0069] 2) The present invention proposes a method for describing the linear and planar features of a subject, which can extract linear or planar elements from a subject for which it is difficult to select point-to-feature points. The corresponding world features are described by linear or planar models, replacing the traditional calibration method of selecting corresponding points. This reduces the applicable scenarios and difficulty of calibration and improves the accuracy of calibration.

[0070] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0071] Figure 1 A flowchart of a camera calibration method based on road features provided in an embodiment of the present invention.

[0072] Figure 2 This is a schematic diagram of the feature subject in an intersection scenario provided in an embodiment of the present invention.

[0073] Figure 3 This is a schematic diagram of the feature subject in a road segment scenario provided in an embodiment of the present invention.

[0074] Figure 4 A schematic diagram of the first image feature points corresponding to the left turn sign provided in an embodiment of the present invention.

[0075] Figure 5 This is a schematic diagram of the second image features corresponding to the pedestrian crossing line provided in an embodiment of the present invention.

[0076] Figure 6 This is a schematic diagram of the second image feature points corresponding to the lane lines provided in an embodiment of the present invention.

[0077] Figure 7 This is a schematic diagram of the third image feature points corresponding to the sign provided in an embodiment of the present invention.

[0078] Figure 8 This is a schematic diagram illustrating the first implementation method of the camera calibration device provided in an embodiment of the present invention.

[0079] Figure 9 This is a schematic diagram illustrating a second implementation of a camera calibration device provided in an embodiment of the present invention.

[0080] Figure 10 This is a schematic diagram illustrating a third method of implementing a camera calibration device according to an embodiment of the present invention. Detailed Implementation

[0081] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0082] In one embodiment, a camera calibration method based on road features is provided, which solves for camera parameters by extracting road feature points from an image and their corresponding world feature points. For example... Figure 1 As shown. It should be noted that the image coordinates are the coordinates of the pixels where the target is located in the image, and are two-dimensional. World coordinates correspond to three-dimensional coordinates in the physical world. The coordinate system can be any coordinate system; for example, it can be a coordinate system composed of latitude, longitude, and altitude, or a coordinate system composed of XYZ in the natural coordinate system. The world coordinates used in this application are latitude, longitude, and altitude coordinates, such as WGS84 coordinates. Feature points can be selected from one or more of the first feature point, second feature point, and third feature point. In this embodiment, the first feature point, second feature point, and third feature point are selected to form the feature points required for calibration. The specific calibration steps are as follows:

[0083] Step 1: Obtain road scene images;

[0084] The calibration image can be a road traffic photograph taken by a camera fixed to a road pole, containing information about road surface, buildings, traffic signs, infrastructure, and traffic markers. These elements constitute the main features of the calibration. The traffic photograph taken here can be a road segment image or an intersection image.

[0085] Figure 2 and Figure 3 Feature images are provided for both intersection and road segment scenarios. Specifically, 201 is a pedestrian crossing, 202 is a left-turn sign, 203 is a stop line, 204 is a straight-ahead sign, 301 is poles and signs, 302 is lane lines, 303 is a straight-ahead sign, and 304 is a guardrail. It should be noted that the examples shown are only a few examples and do not cover all features in the images.

[0086] Step 2: Extract the coordinates of the feature points in the first image of the road, and obtain the corresponding first-world feature point coordinates. The specific steps are as follows:

[0087] Step 2-1: Extract the first feature subject from the road from the calibration image;

[0088] Step 2-2: Extract feature points from each first feature subject to form the first image feature points;

[0089] Steps 2-3: Obtain the coordinates of the first image feature point corresponding to the first image feature point from the calibration image, that is, the pixel coordinates corresponding to the image feature point;

[0090] Steps 2-4: Extract the corresponding first-world feature points using the first image feature points;

[0091] Steps 2-5: Obtain the world coordinates corresponding to the first world feature point, i.e., the coordinates of the first world feature point.

[0092] The first feature subject here refers to those feature subjects that can accurately locate feature points, for example... Figure 2 Numbers 201, 202, 203, and 204 in the text. Figure 3 301 and 303 in the list. Among them, Figure 3 The reason why 302 and 304 are not used as the primary feature is because it is difficult to locate their specific position in the physical world, which is also a problem that traditional calibration methods cannot solve.

[0093] In this embodiment of the application, the selection of the first feature subject is exemplified by a left-turn sign. Figure 4 The first image feature points extracted are given, where these feature points correspond to the corner positions of the markers. The corner points can be selected manually or automatically identified using a corner detection algorithm. In this embodiment, the coordinates of the first world feature points are obtained by reading the latitude, longitude, and altitude of the corresponding feature points from a high-precision map.

[0094] Step 3: Extract the coordinates of the second image feature points of the road and obtain the corresponding second world feature point coordinates. The specific steps are as follows:

[0095] Step 3-1: Extract the second feature subject from the road from the calibration image;

[0096] Step 3-2: Extract feature lines from each second feature body. Here, each feature body may contain multiple feature lines, such as... Figure 5 As shown in 1-7;

[0097] Step 3-3: Discretize the feature lines to obtain feature points, such as... Figure 6 As shown, each feature line can extract multiple feature points, and the feature points extracted from all feature lines together form the second image feature points.

[0098] Steps 3-4: Obtain the coordinates of the second image feature points corresponding to the second image feature points from the calibration image, that is, the pixel coordinates corresponding to the image feature points;

[0099] Steps 3-5: Extract the second-world features corresponding to the second image features;

[0100] Steps 3-6: Discretize the second world features to obtain the second world feature points, and obtain the world coordinates corresponding to the second world feature points, which are the coordinates of the second world feature points.

[0101] One method for obtaining the second feature points from discrete lane lines is manual selection. This method requires manually locating the lane centerline and then selecting several points on the centerline as the second image feature points. The selection of point distribution and the number of points can vary in different embodiments.

[0102] Another method for obtaining second image feature points from discrete lane lines is automatic selection. This method extracts the lane line contour using a segmentation algorithm, then automatically calculates the lane line centerline, and finally discretizes the centerline to obtain the second image feature points. Here, the selection of point distribution and the number of points can also differ in different embodiments.

[0103] In this embodiment of the application, the second-world features are described by a linear model, as shown below:

[0104]

[0105]

[0106] in, Represents the i-th straight line in the second-world features. Let be the coordinates of any point on the i-th straight line. Let be the unit vector representing the direction of the i-th line. In constructing the line model, this embodiment uses... Figure 6 Taking lane lines in the middle as an example, Can be selected as Figure 6 The coordinates of point 1 in the world coordinate system can be selected, or the coordinates of any point on the line can be chosen. The unit vector of the line direction can be obtained by selecting the coordinates of any two points on the line.

[0107]

[0108]

[0109] in, and Let be the coordinates of any two points on the straight line. It should be noted that if latitude, longitude, and altitude are chosen as the world coordinate system, then... It needs to be converted into the corresponding projected coordinates, such as UTM coordinates.

[0110] The discretization method for second-world feature points is given by the following equation:

[0111]

[0112] in, Let m be the m-th feature point obtained by discretizing the i-th straight line in the second world feature. These are the second-world feature parameters, used together with the camera parameters as calibration parameters. Here, the number of discrete points corresponding to each straight line feature needs to be consistent with the number of feature points in the second image. For example, the first straight line discrete... Each image feature point needs to correspond to... A world feature point, the second line is discrete Each feature point needs to correspond to... There are n world feature points, ..., the Qth discrete line. Each feature point needs to correspond to... A world feature point.

[0113] Step 4: Extract the coordinates of third-world feature points from the road image and obtain the corresponding third-world feature point coordinates. The specific steps are as follows:

[0114] Step 4-1: Extract the third feature subject from the road from the calibration image;

[0115] Step 4-2: Extract feature surfaces from each third feature body. Here, each feature body may contain multiple feature surfaces.

[0116] Step 4-3: Discretize the feature surface to obtain feature points, such as... Figure 7 As shown, each feature surface can extract multiple feature points, and the feature points extracted from all feature surfaces together form the third image feature points.

[0117] Step 4-4: Obtain the coordinates of the third image feature points from the calibration image, that is, the pixel coordinates of the image feature points;

[0118] Steps 4-5: Extract the third-world features corresponding to the feature points in the third image;

[0119] Steps 4-6: Discretize the third-world features to obtain third-world feature points, and obtain the world coordinates corresponding to the third-world feature points, which are the coordinates of the third-world feature points.

[0120] In this embodiment of the invention, the third image feature points are obtained by manually discretizing the feature surface. This method requires manually locating the feature surface region, and then selecting several points on the feature surface as the third image feature points. The selection of the point distribution and the number of points can vary in different embodiments.

[0121] In this embodiment of the application, the features of the third world are described by a planar model, as shown below:

[0122]

[0123]

[0124] in, Denotes the i-th plane in the characteristics of the Third World. Let be the coordinates of any point on the i-th plane. Let be two non-collinear unit vectors on the i-th plane. In this embodiment, is used... Figure 7 Taking lane lines as an example, we can obtain Figure 7 The world coordinates corresponding to points 1, 5, and 16 are as follows: , , .

[0125] Can be selected as Alternatively, you can choose the coordinates of any point on the feature surface. It can be given by the following equation:

[0126]

[0127]

[0128]

[0129]

[0130] It should be noted here that if latitude, longitude, and altitude are chosen as the world coordinate system, then... It needs to be converted into the corresponding projected coordinates, such as UTM coordinates.

[0131] The discretization method for third-world feature points is given by the following equation:

[0132]

[0133] in, Let m be the m-th feature point obtained by discretizing the i-th plane in the third world features. and These are the third-world feature parameters, used together with the camera parameters as calibration parameters. Here, the number of discrete points corresponding to each planar feature needs to be consistent with the number of feature points in the third image. For example, the first planar discrete... Each image feature point needs to correspond to... A world feature point, the second line is discrete Each feature point needs to correspond to... There are n world feature points, ..., the Vth line is discrete. Each feature point needs to correspond to... A world feature point.

[0134] Step 5: Obtain the coordinates of road image feature points and the corresponding world feature point coordinates. In this embodiment of the invention, the first feature point, the second feature point, and the third feature point are selected to form the calibration feature points. In different embodiments, one or more of these features may be selected.

[0135] Step 6: Define a loss function using the coordinates of image feature points and world feature points. The loss function consists of multiple components, including the first world feature point projection error, the second world feature point projection error, the third world feature point projection error, the second world feature parameter constraint error, the third world feature parameter constraint error, and the camera parameter constraint error.

[0136] The projection error is given by the following equation:

[0137]

[0138] in, These are the pixel coordinates of the feature points. These are projected coordinates. For Huber functions, These are the weighting coefficients.

[0139] The parameter constraint error is given by the following equation:

[0140]

[0141] in, For characteristic parameters, The minimum value of the characteristic parameter. The maximum value of the characteristic parameter. The range constraint penalty function is given by the following equation:

[0142]

[0143] Step 7: Obtain camera parameters by optimizing the loss function. This embodiment of the invention employs a gradient-based optimization algorithm, such as the Levenberg-Marquardt method.

[0144] In one embodiment, a camera calibration device based on road features is provided, comprising: a receiving unit and a processing unit. The receiving unit is used to acquire a calibration image, including one or more of a first feature subject, a second feature subject, and a third feature subject; the receiving unit is also used to receive a three-dimensional road model, including one or more of feature point position measurements and a high-precision map model; the processing unit is used to construct a line model in the second world features, and discretize the second image features and the second world features to obtain the coordinates of the second image feature points and the second world feature points; the processing unit is also used to construct a surface model in the third world features, and discretize the third image features and the third world features to obtain the coordinates of the third image feature points and the third world feature points; the processing unit is also used to construct a loss function and use the loss function for camera calibration.

[0145] One possible implementation is to place the receiving unit and the processing unit in a single device, such as... Figure 8 As shown, the device can be a camera or other image acquisition device.

[0146] Other possible implementations include having the receiving unit and processing unit located in different devices, such as... Figure 9 As shown, the receiving device can be a camera or other image acquisition device, and the processing device can be a computer host.

[0147] Other possible implementations include having the receiving unit and processing unit located in different devices, and a single processing device connecting to multiple receiving devices, such as... Figure 10 As shown, the receiving device can be a camera or other image acquisition device, and the processing device can be a computer host.

[0148] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention without departing from its spirit and scope should be included within the protection scope of the present invention.

Claims

1. A camera calibration method based on road features, characterized in that, The method includes the following steps: A road scene image is acquired as a calibration image; the calibration image contains feature subjects on the road, from which a first feature, a second feature, or a third feature can be extracted, wherein the first feature corresponds to a feature point in the first feature subject, the second feature corresponds to a feature line in the second feature subject, and the third feature corresponds to a feature surface in the third feature subject; The coordinates of image feature points are obtained based on the calibrated image and the feature subject. Construct a feature subject model and obtain world feature point coordinates, wherein the world feature point coordinates correspond one-to-one with the image feature point coordinates; wherein the world feature point coordinates obtained by the first feature subject are the world coordinate points corresponding to the image feature points, and the world feature point coordinates obtained by the second and third feature subjects contain feature model parameters. The loss function is defined using image feature point coordinates and world feature point coordinates; Camera calibration is performed using a loss function to obtain camera parameters; The loss function is composed of multiple factors including first-world feature point projection error, second-world feature point projection error, third-world feature point projection error, second-world feature parameter constraint error, third-world feature parameter constraint error, and camera parameter constraint error. Among them, projection error It is given by the following equation: In the formula, These are the pixel coordinates of the feature points. These are projected coordinates. For Huber functions, These are the weighting coefficients; The parameter constraint error is given by the following equation: In the formula, For characteristic parameters, The minimum value of the characteristic parameter. The maximum value of the characteristic parameter. The range constraint penalty function is given by the following equation: in, For constraint variables, The lower boundary of the constraint variable, This represents the upper boundary of the constraint variable; The process of obtaining the coordinates of image feature points includes: Extract feature points of the first feature subject in the road from the calibration image, denoted as the first image feature point, and obtain the coordinates of the first image feature point; Alternatively, extract the feature lines of the second feature subject in the road from the calibration image, denoted as the second image feature point, and obtain the coordinates of the second image feature point; Alternatively, extract the feature surface of the third feature subject in the road from the calibration image; denote it as the third image feature point, and obtain the coordinates of the third image feature point; The first feature includes one or more of road traffic signs and road infrastructure. The road traffic signs include one or more of turning signs, waiting areas, pedestrian crossings, and stop lines. The road infrastructure includes one or more of traffic light poles, electronic police poles, green belts, and traffic fences. The second feature includes one or more of road traffic signs and road infrastructure. The road traffic signs include one or more of lanes, stop lines, and dividing lines. The road infrastructure includes one or more of traffic light poles, electronic police poles, green belts, traffic fences, and road edges. The third feature includes one or more of the following: road surface, signboard, and building facade; The process of obtaining the coordinates of world feature points includes: Obtain the world feature points corresponding to M feature points, and obtain the coordinates of the M world feature points to form the first world feature point coordinates; Sum or minus, obtain the second-world features corresponding to Q feature lines; Discretize the second world feature to obtain the second world feature points, and obtain the coordinates of the second world feature points; AND or OR, obtain the third-world features corresponding to V feature surfaces; Discretize the third-world features to obtain third-world feature points, and obtain the coordinates of the third-world feature points; The second world feature is given by the following linear model: in, This represents the i-th feature line in the second world features. Let be the coordinates of any point on the i-th feature line. Let be the unit vector of the direction of the i-th feature line; The second-world feature points are obtained by discretization using the following equation: in, Let m be the m-th feature point obtained by discretizing the i-th feature line in the second world feature. These are second-world feature parameters, used together with camera parameters as calibration parameters; The characteristics of the Third World are given by the following planar model: in, This represents the i-th feature surface in the Third World features. Let be the coordinates of any point on the i-th plane. Let be two non-collinear unit vectors on the i-th feature plane; Third-world feature points are obtained by discretization using the following equation: in, Let m be the m-th feature point obtained by discretizing the i-th feature surface in the third world features. and These are third-world feature parameters, used together with camera parameters as calibration parameters.

2. The camera calibration method based on road features according to claim 1, characterized in that, The step of extracting feature points of the first feature subject in the road from the calibration image, denoted as the first image feature point, and obtaining the coordinates of the first image feature point is as follows: Obtain N primary feature entities from the calibration image; M feature points are extracted from the N first feature subjects to form the first image feature points; wherein, the first first feature subject includes The first feature point contains one feature point, and the second first feature body contains... The first feature point contains..., the Nth first feature subject contains One feature point, ; Extract the pixel coordinates of M feature points, which are the coordinates of the first image feature points.

3. The camera calibration method based on road features according to claim 1, characterized in that, The step of extracting the feature lines of the second feature subject in the road from the calibration image, denoted as the second image feature point, and obtaining the coordinates of the second image feature point is as follows: Obtain P second feature entities from the calibration image; Q feature lines are extracted from the P second feature subjects to form the second image features; wherein, the first second feature subject includes The first feature line, the second feature body contains The first feature line, ..., the Pth second feature body contains One characteristic line, ; Discretize the Q feature lines to obtain S feature points, which constitute the second image feature points; wherein, the first feature line is discretized. The first feature point, the second feature line is discrete. There are 1 feature points, ..., the Qth feature line is discrete. One feature point, ; Extract the pixel coordinates of S feature points, which are the coordinates of the second image feature points.

4. The camera calibration method based on road features according to claim 1, characterized in that, The feature surface of the third feature subject in the road is extracted from the calibration image; it is denoted as the third image feature point, and the coordinates of the third image feature point are obtained, as follows: Obtain T third-feature entities from the calibration image; V feature surfaces are extracted from the T third feature subjects to form V third image features; wherein, the first third feature subject includes The second and third feature bodies contain a plane, and the main body contains a second and third feature. A plane, ..., the Tth third feature subject contains A plane, ; Discretize each of the V feature surfaces to obtain U feature points, which constitute U third image feature points; wherein, the first feature surface is discretized The first feature point, the second feature surface is discrete. The Vth feature point is discrete, ..., the Vth feature surface is discrete. One feature point, ; Extract the pixel coordinates of U feature points, which are the coordinates of the third image feature points.

5. A camera calibration device based on road features according to the method of any one of claims 1 to 4, characterized in that, The device includes: The receiving unit is used to acquire a calibration image, which includes one or more of a first feature subject, a second feature subject, and a third feature subject; The receiving unit is also used to receive a three-dimensional road model, including one or more of feature point position measurements and high-precision map models; The processing unit is used to construct a line model in the second world features, and to discretize the second image features and the second world features to obtain the coordinates of the second image feature points and the second world feature points. The processing unit is also used to construct a surface model in the third-world features, and to discretize the third image features and the third-world features to obtain the coordinates of the third image feature points and the coordinates of the third-world feature points; The processing unit is also used to construct a loss function and use the loss function to calibrate the camera.

Images