Driving parameter calibration method, device, computer equipment and storage medium for vehicle-mounted camera

By collecting motion data on the vehicle and estimating driving parameters, and automatically updating the motion trajectory and information values, efficient calibration of the on-board camera is achieved, and the problems of high cost and limited application scope in the existing technology are solved, and calibration efficiency and accuracy are improved.

CN118379368BActive Publication Date: 2024-11-12SUZHOU MAGIC VISION INTELLIGENT TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410843016.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-06-27
Publication Date
2024-11-12
Estimated Expiration
2044-06-27

AI Technical Summary

Technical Problem

The existing vehicle-mounted camera calibration methods need to be carried out in a dedicated calibration room, resulting in high labor costs and long time. The calibration results are affected by the laying of the calibration plates, and the scope of application is limited.

Method used

By determining the initial motion trajectory of the vehicle, collecting motion data, estimating the estimated value of driving parameters, and updating the motion trajectory and information values ​​using preset formulas until the conditions are met, avoiding the use of calibration rooms.

Benefits of technology

It reduces manpower and material costs, improves calibration efficiency and accuracy, reduces dependence on calibration plate laying, and is suitable for a variety of environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118379368B_ABST
    Figure CN118379368B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of camera calibration technology, and discloses a driving parameter calibration method, device, computer equipment and storage medium for a vehicle-mounted camera, which determines an initial motion trajectory corresponding to a vehicle, collects motion data of the vehicle and the vehicle-mounted camera traveling along the initial motion trajectory; determines a driving parameter estimation value according to the vehicle motion data corresponding to the vehicle and the camera motion data corresponding to the vehicle-mounted camera; determines a control variable of the vehicle and / or a preset information value according to the driving parameter estimation value using a preset formula; wherein the control variable is used to update the initial motion trajectory; when the preset information value meets a preset condition, determines the driving parameter estimation value as the driving parameter, and calibrates the vehicle and the vehicle-mounted camera according to the driving parameter. The technical solution provided by the embodiment disclosed in the present invention can reduce the high cost of calibration with the aid of calibration objects and improve the efficiency of camera calibration.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of camera calibration, and in particular to a method, device, computer equipment and storage medium for calibrating driving parameters of a vehicle-mounted camera. Background Art

[0002] With the development of unmanned driving technology, during the process of autonomous driving, calibration is needed to obtain the relative position of sensors such as cameras and vehicles. The calibration method in the existing solution needs to be assisted by a dedicated calibration room during the vehicle production process. The existing calibration method has the problems of high labor costs and long calibration time, and a better solution is needed. Summary of the invention

[0003] In view of this, the present invention provides a method, device, computer equipment and storage medium for calibrating driving parameters of a vehicle-mounted camera to solve the problem of high cost when performing camera calibration with the assistance of a calibration room.

[0004] On one hand, the present invention provides a method for calibrating driving parameters of a vehicle-mounted camera, the method comprising determining an initial motion trajectory corresponding to a vehicle, collecting motion data of the vehicle and the vehicle-mounted camera traveling along the initial motion trajectory; determining a driving parameter estimation value based on the vehicle motion data corresponding to the vehicle and the camera motion data corresponding to the vehicle-mounted camera; determining a control variable and / or a preset information value of the vehicle based on the driving parameter estimation value using a preset formula; wherein the control variable is used to update the initial motion trajectory; when the preset information value meets a preset condition, determining the driving parameter estimation value as the driving parameter, and calibrating the vehicle and the vehicle-mounted camera based on the driving parameter.

[0005] On the other hand, the present invention also provides a driving parameter calibration device for a vehicle-mounted camera, the device comprising: an acquisition module, used to determine an initial motion trajectory corresponding to a vehicle, and to collect motion data of the vehicle and the vehicle-mounted camera traveling along the initial motion trajectory; a first determination module, used to obtain a driving parameter estimation value based on the vehicle motion data corresponding to the vehicle and the camera motion data corresponding to the vehicle-mounted camera; a second determination module, used to determine a control variable and / or a preset information value of the vehicle according to the driving parameter estimation value using a preset formula; wherein the control variable is used to update the initial motion trajectory; a calibration module, used to determine the driving parameter estimation value as the driving parameter when the preset information value meets a preset condition, and to calibrate the vehicle and the vehicle-mounted camera according to the driving parameter.

[0006] On the other hand, the present invention also provides a computer device, which includes: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor implements the above-mentioned driving parameter calibration method of the vehicle-mounted camera by executing the computer instructions.

[0007] Another aspect of the present invention further provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to enable a computer to implement the above-mentioned driving parameter calibration method of a vehicle-mounted camera.

[0008] In this process, the vehicle and the on-board camera are calibrated according to the driving parameters without the aid of calibration objects such as a calibration room. This can reduce manpower and material costs, improve calibration efficiency, and ensure that the accuracy of the calibration results is not affected by the laying of the calibration board, thereby improving the accuracy of camera calibration. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0010] Figure 1 It is a flow chart of a method for calibrating driving parameters of a vehicle-mounted camera provided by an embodiment of the present invention;

[0011] Figure 2 is a schematic diagram of a coordinate system of a driving parameter calibration method of a vehicle-mounted camera provided by an embodiment of the present invention;

[0012] Figure 3 It is a flow chart of another method for calibrating driving parameters of a vehicle-mounted camera provided by an embodiment of the present invention;

[0013] Figure 4 It is a structural schematic diagram of a driving parameter calibration device for a vehicle-mounted camera provided by an embodiment of the present invention;

[0014] Figure 5 It is a structural schematic diagram of another driving parameter calibration device for a vehicle-mounted camera provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0015] Unmanned driving technology (or autonomous driving technology) refers to the technology that allows vehicles or other vehicles to perceive the driving environment through their own sensors, independently decide on the driving path, and control the vehicle to reach the desired destination along the desired path without the need for human operation; unmanned driving technology is an integration of multiple technologies, mainly including real-time perception and positioning, motion path planning, communication and data interaction, vehicle intelligent control and other technologies.

[0016] Cameras are one of the most common and widely used sensors in driverless vehicles. Using the sensor's measurement values ​​to estimate the vehicle's state requires knowing the relative position of the sensor and the vehicle. Existing calibration technologies mainly include: calibration methods with calibration targets and calibration methods without calibration targets. Among them, the calibration method with calibration targets usually enters a special calibration room when the vehicle is off the production line to complete the estimation of the relative position of the camera and the vehicle. During later maintenance, calibration boards similar to the calibration room are usually laid on a flat site to assist in completing the calibration. The calibration method without calibration targets requires the vehicle to collect surrounding environment information while moving, and the relationship between the camera's measurement values ​​and the vehicle body's position can be used to estimate the relative position of the camera to the vehicle body coordinate system.

[0017] However, the current calibration method also has the following problems:

[0018] First, in the calibration method with calibration targets, the vehicle needs to be calibrated in a calibration room or test room, which has strict requirements on the location of the vehicle, and the cost of the calibration room is too high;

[0019] Second, in the calibration method with calibration targets, the calibration effect is greatly affected by the laying effect of the calibration plate, and the laying process of the calibration plate consumes a lot of manpower and time costs;

[0020] Third, among the calibration methods without calibration targets, the lane line-based calibration method is only applicable to open roads with multiple lanes. The feature point-based calibration method is greatly affected by dynamic obstacles in the environment and is not suitable for places with fewer features. The optimization process of automatic calibration is also time-consuming and requires processing a large amount of redundant data.

[0021] To solve at least one of the above problems, various embodiments of the present invention provide a method for calibrating driving parameters of a vehicle-mounted camera, including: determining an initial motion trajectory corresponding to a vehicle, and collecting motion data of the vehicle and the vehicle-mounted camera traveling along the initial motion trajectory; determining a driving parameter estimation value based on the vehicle motion data corresponding to the vehicle and the camera motion data corresponding to the vehicle-mounted camera; determining a control variable of the vehicle and / or a preset information value using a preset formula based on the driving parameter estimation value; wherein the control variable is used to update the initial motion trajectory; when the preset information value meets a preset condition, determining the driving parameter estimation value as the driving parameter, and calibrating the vehicle and the vehicle-mounted camera based on the driving parameter.

[0022] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0023] According to an embodiment of the present invention, a method for calibrating driving parameters of a vehicle-mounted camera is provided. Figure 1 FIG. 1 is a flow chart of a method for calibrating driving parameters of a vehicle-mounted camera provided by an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0024] Step S101, determining an initial motion trajectory corresponding to the vehicle, and collecting motion data of the vehicle and the vehicle-mounted camera traveling along the initial motion trajectory;

[0025] Step S102, determining a driving parameter estimation value according to vehicle motion data corresponding to the vehicle and camera motion data corresponding to the vehicle-mounted camera;

[0026] Step S103, according to the estimated driving parameter value, a preset formula is used to determine the control variable of the vehicle and / or the preset information value; wherein the control variable is used to update the initial motion trajectory;

[0027] Step S104, when the preset information value meets the preset condition, the estimated driving parameter value is determined as the driving parameter, and the vehicle and the vehicle-mounted camera are calibrated according to the driving parameter.

[0028] Specifically, when it is determined that the vehicle is traveling along an initial motion trajectory, motion data is collected for the vehicle and a vehicle-mounted camera installed on the vehicle; based on the collected vehicle motion data and camera motion data, a driving parameter estimate corresponding to the vehicle is determined; a control variable and / or a preset information value corresponding to the vehicle for updating the initial motion trajectory is calculated by a preset formula; when the preset information value meets a preset condition, the driving parameter estimate is determined as the driving parameter corresponding to the vehicle, and the vehicle and the vehicle-mounted camera are calibrated.

[0029] In a possible implementation, the vehicle may refer to an unmanned vehicle; and determining an initial motion trajectory corresponding to the vehicle may refer to causing the unmanned vehicle to automatically drive along the initial motion trajectory.

[0030] Furthermore, the vehicle-mounted camera may refer to a camera installed on an unmanned vehicle, and the camera may be installed on the outer contour of the vehicle.

[0031] Here, the camera can be a type of vehicle-mounted sensor. By capturing images or videos of the vehicle's surroundings, the camera can provide the vehicle with the ability to perceive and identify the environment. Through the camera, the vehicle can obtain its own location information in real time and construct a map of the surrounding environment. Through the visual information provided by the camera, the vehicle can detect and avoid other pedestrians or vehicles during unmanned driving to ensure the safety of the vehicle.

[0032] In a possible implementation, collecting motion data of a vehicle and a vehicle-mounted camera traveling along an initial motion trajectory may include:

[0033] Collecting motion data of the vehicle to determine the vehicle motion data corresponding to the vehicle, and collecting motion data of the camera to determine the camera motion data corresponding to the camera;

[0034] Vehicle motion data may include, but is not limited to: vehicle speed and acceleration, direction and steering angle, vehicle posture information, and Global Positioning System (GPS) location;

[0035] The camera motion data may include, but is not limited to: the camera's position and posture information, focal length and field of view, and camera motion trajectory.

[0036] Among them, the vehicle speed can be obtained through the built-in speed sensor on the vehicle, and the acceleration can be obtained through the vehicle control system; the vehicle's driving direction and steering angle can be obtained through the vehicle's steering system or sensors such as gyroscopes; the posture information of the vehicle and camera can be obtained through the inertial measurement unit; the camera's motion trajectory can be obtained through the visual odometer.

[0037] Here, the posture information of the vehicle and the camera may include but is not limited to: pitch angle, roll angle and yaw angle; wherein the pitch angle may refer to the angle of rotation of an object around its lateral axis (i.e., X-axis), the roll angle may refer to the angle of rotation of an object around its longitudinal axis (i.e., Y-axis), and the yaw angle may refer to the angle of rotation of an object around its vertical axis (i.e., Z-axis); the pitch angle, roll angle and yaw angle can describe the posture of the vehicle and the camera in three-dimensional space, i.e., the movement direction and rotation state of the vehicle and the camera.

[0038] Furthermore, an inertial measurement unit (IMU) for measuring attitude information may refer to a device integrating multiple inertial sensors for measuring and recording acceleration and angular velocity of a vehicle and a camera; an IMU may include but is not limited to: an accelerometer and a gyroscope.

[0039] Visual Odometer (VO) is a technology that estimates the motion of a camera in three-dimensional space by analyzing continuous image sequences. It can use the changes in visual features between adjacent frames to infer the changes in the camera's posture, thereby tracking and positioning the camera's motion trajectory.

[0040] In a possible implementation, when the preset information value satisfies the preset condition, determining the estimated driving parameter as the driving parameter, and calibrating the vehicle and the vehicle-mounted camera according to the driving parameter may include:

[0041] Determine driving parameters as a conversion relationship between vehicle motion data and camera motion data;

[0042] According to the conversion relationship represented by the driving parameters, the translation and rotation of the camera relative to the vehicle are determined, and the vehicle and the camera are calibrated.

[0043] Through the above method, the relative position of the camera to the vehicle can be estimated by determining the driving parameters, so that the calibration process does not require the use of calibration objects or calibration rooms, reducing the manpower and time costs in the calibration process, and improving the efficiency of camera calibration through automated calibration.

[0044] In some embodiments, determining the estimated value of the driving parameter according to the vehicle motion data corresponding to the vehicle and the camera motion data corresponding to the vehicle-mounted camera includes:

[0045] Determine the vehicle coordinate system corresponding to the vehicle and the camera coordinate system corresponding to the onboard camera;

[0046] Determine vehicle motion data of the vehicle in the vehicle coordinate system and camera motion data of the vehicle-mounted camera in the camera coordinate system;

[0047] The driving parameter estimate is determined so that the product of the vehicle motion data and the driving parameter estimate is equal to the product of the camera motion data and the driving parameter estimate.

[0048] In one possible implementation, the vehicle coordinate system can take the center of the vehicle's rear axle as its origin, the vehicle's moving direction as the x-axis, and the left as the y-axis; the camera coordinate system can take the direction of the camera's optical center as the z-axis, and the imaging plane direction as the x-axis; wherein, the setting of the vehicle coordinate system and the camera coordinate system must follow the right-hand rule.

[0049] For example, Figure 2 Schematic diagram of a coordinate system of a method for calibrating driving parameters of a vehicle-mounted camera provided by an embodiment of the present invention. The vehicle coordinate system and the camera coordinate system are as follows: Figure 2 shown.

[0050] Here, the optical center direction of the camera may refer to the direction pointed by the optical center in the camera optical system, and the optical center is located on the optical axis of the camera lens; the imaging plane row direction is the x-axis, which may indicate that the horizontal direction on the imaging plane is parallel to the x-axis, and the imaging plane may refer to the place where the image is formed after the light is focused.

[0051] In a possible implementation, the transformation matrix from the camera coordinate system to the vehicle coordinate system can be set as , let the camera motion data between any two frames be , assuming that the vehicle motion data between any two frames is .

[0052] Specifically, It can refer to driving parameter estimation, camera motion data Vehicle motion data The following relationship is satisfied:

[0053] (1)

[0054] in, Can indicate the i Frame to j The camera motion change matrix between frames can describe the camera's position change in space, that is, the camera changes from the first i Frame to j Relative motion of frames; To estimate the driving parameters, the camera position and orientation in the vehicle coordinate system can be expressed; Can indicate the i Frame to j The motion change matrix of the vehicle between frames can describe the change of the vehicle's position in space, that is, the vehicle changes from the first i Frame to j Relative motion of frames.

[0055] Here, the motion change matrix can be a mathematical tool for describing the motion and rotation of an object in three-dimensional space. The motion change matrix can be a 4*4 matrix including translation and rotation information; posture can refer to the position and direction of an object in three-dimensional space.

[0056] In a possible implementation, determining the estimated driving parameter value so that the product of the vehicle motion data and the estimated driving parameter value is equal to the product of the camera motion data and the estimated driving parameter value may include:

[0057] Determine driving parameter estimates , so that the camera motion data Estimated parameters for left-hand driving The product of is equal to the estimated value of driving parameters Left-hand vehicle motion data The product of .

[0058] Through the above method, by determining the estimated value of driving parameters and estimating the relative pose from the camera to the vehicle, the vehicle motion data can be linked with the camera motion data, so that the calibration process can be performed without the help of a calibration board, thereby improving the efficiency of camera calibration.

[0059] In some embodiments, determining the estimated driving parameter value so that the product of the vehicle motion data and the estimated driving parameter value is equal to the product of the camera motion data and the estimated driving parameter value includes:

[0060] Measuring k sets of camera motion data between two frames according to a preset visual method, where the preset visual method includes at least one of a five-point method, an eight-point method, and a direct method; wherein k is a non-negative real number;

[0061] Measure k groups of vehicle motion data between two frames according to a preset mileage calculation method;

[0062] Determine camera motion estimation data based on vehicle motion data and driving parameter estimation values;

[0063] Determine the estimated driving parameter value corresponding to the minimum residual between the camera motion data and the camera motion estimation data.

[0064] Specifically, k groups of camera motion data and vehicle motion data between two frames are measured respectively through a preset visual method and a preset mileage calculation method, and the camera motion estimation data is determined according to the vehicle motion data and the driving parameter estimation data, and the driving parameter estimation data corresponding to the minimum residual between the camera motion data and the camera motion estimation data is determined.

[0065] Among them, the five-point method may refer to a camera motion estimation method based on feature point matching, which calculates the camera motion by using the correspondence between five pairs of feature points detected in two images; the eight-point method is similar to the five-point method, and the camera motion can be calculated by the correspondence between at least eight pairs of feature points; the direct method can directly use the pixel value information of the image for camera motion estimation without explicitly detecting and matching feature points; the preset mileage calculation method may refer to the wheel speed odometer integration method, which may be a common method for estimating vehicle motion, which may estimate the displacement and direction change of the vehicle in each time step by measuring the rotation speed of the vehicle wheels and combining the vehicle's geometric structure and motion constraints.

[0066] In a possible implementation, determining the estimated driving parameter value corresponding to the minimum residual between the camera motion data and the camera motion estimated data may include:

[0067] The following formula is used to express the estimated value of driving parameters :

[0068] (2)

[0069] in, The operator can be used to find the parameter value that minimizes a given function. It can represent k sets of camera motion data between two frames, It can be expressed based on vehicle motion data Estimated driving parameters Determined camera motion estimation data, can represent the covariance of the measurement noise w, The estimated value of driving parameters represented by The covariance of .

[0070] Here, covariance is a statistic used to measure the degree of linear correlation between two random variables. It can describe whether the changing trends of the two random variables are in the same direction or opposite directions, as well as the degree of change.

[0071] By using the above method, the estimated driving parameter corresponding to the minimum residual between the camera motion data and the camera motion estimation data is determined. , which can make the calibration results more accurate and more robust.

[0072] In some embodiments, based on the estimated driving parameter values, a preset formula is used to determine the control variables of the vehicle and / or the preset information values, including:

[0073] Determine a vector form corresponding to the estimated driving parameter value, and determine a corresponding information matrix according to the estimated driving parameter value in the vector form;

[0074] Determine the control variable corresponding to the maximum value of the information matrix and / or the preset information value.

[0075] In a possible implementation, the information matrix can be expressed as follows:

[0076] (3)

[0077] Among them, p can represent the vector form corresponding to the estimated value of the train parameters, so as to facilitate the derivation, let ,in It can be a translation, that is , can be the rotation angle, i.e. ; The conversion relationship can be .

[0078] Further, Represents the information matrix, which is used to measure the information contribution of the observation value to the estimated parameters. When the information matrix takes the maximum value, it can be said that the information contribution of the observation value to the estimated parameters is the largest, the uncertainty is the smallest, and the accuracy of the train parameter estimation is higher.

[0079] Through the above method, the vector p of the train parameter estimation value is determined, the information matrix corresponding to p is determined, and the corresponding control variables and / or preset information values ​​are determined when the information matrix takes the maximum value. This can minimize the uncertainty of the control variables and / or preset information values, and can obtain as much effective information as possible to improve the accuracy of camera positioning.

[0080] In some embodiments, determining the control variable and / or the preset information value corresponding to the maximum value of the information matrix includes:

[0081] According to the vehicle motion data, it is determined that the position and posture of the vehicle at the i-th moment is represented by the position and posture at the i-1-th moment and the control variable; wherein i is greater than or equal to 1;

[0082] Determine the inverse matrix corresponding to the information matrix, and convert the information matrix to a minimum value;

[0083] Determine the control variable and preset information value corresponding to when the inverse matrix takes the minimum value; wherein the preset information value represents the trace determined by the trace operation of the inverse matrix.

[0084] In one possible implementation, the vehicle's position at the i-th moment is From the position at the i-1th moment and control variables u OK, it can be written as .

[0085] Here, the G() function can refer to a motion model used to represent the vehicle posture, through the posture and control variables at the i-1th moment u , calculate and determine the vehicle's position and posture at the i-th moment.

[0086] Exemplarily, the motion model can be a mathematical model that describes the change of the state of an object over time during its motion, and can be used to predict the change of the vehicle's posture over time and the change of control input; the G() function as an operating model can include but is not limited to: differential drive motion model, bicycle model, Ackermann Model.

[0087] Furthermore, the vehicle motion data between any two frames Can be characterized as ;in, It can represent the vehicle motion data at the i-th moment, that is, the vehicle's position and posture, The inverse transformation of the vehicle's posture at the jth moment can be represented.

[0088] In a possible implementation, the inverse matrix corresponding to the information matrix can be expressed by the following formula:

[0089] (4)

[0090] in, Can be expressed as dependent on With parameters As shown in formula (3), the information matrix The information matrix can be expressed as The inverse matrix of ; As shown in formula (4), the information matrix is ​​calculated The inverse matrix The minimum value of is equivalent to calculating the information matrix The maximum value of , that is, the inverse matrix The minimum value of ; () can represent the trace operation of a matrix, which can determine the sum of all elements on the main diagonal of the matrix.

[0091] Here, the trace operation can refer to a linear algebraic operation that determines the sum of the diagonal elements of a matrix; the goal of formula (4) is to adjust the control variable u , so that the vehicle motion data can maximize the information matrix The effective amount of information can be increased and the constraints of vehicle motion can be satisfied at the same time.

[0092] Furthermore, formula (4) can be expressed by giving the control variable u Under the premise of input, the trace of the inverse matrix of the information matrix is ​​maximized by adjusting the estimated value p of the train parameter to be estimated to improve the accuracy of the estimated value p of the train parameter.

[0093] Through the above method, by minimizing the inverse matrix of the information matrix, as many effective control variables as possible are obtained and the next control of the vehicle is calculated, which can make the train parameter estimation more accurate.

[0094] In some embodiments, when the preset information value meets the preset condition, the estimated driving parameter value is determined as the driving parameter, and the vehicle and the vehicle-mounted camera are calibrated according to the driving parameter, including:

[0095] When the preset information value is greater than or equal to the preset threshold, the initial motion trajectory of the vehicle is updated according to the control variable, so that the vehicle travels along the updated initial motion trajectory;

[0096] When the preset information value is less than the preset threshold, the estimated driving parameter value is determined as the driving parameter, and the vehicle and the on-board camera are calibrated according to the driving parameter.

[0097] Specifically, when the trace of the information matrix is ​​greater than or equal to the preset threshold, it means that the information matrix does not contain enough information to ensure accurate estimation of the train parameter estimate. The control variable corresponding to formula (4) updates the initial motion trajectory, so that the vehicle drives along the updated initial motion trajectory and continues to collect vehicle motion data and camera motion data. When the trace of the information matrix is ​​less than the preset threshold, it means that enough information has been collected to ensure accurate estimation of the train parameter estimate. The driving parameter estimate can be determined as the driving parameter, and the vehicle and camera can be calibrated.

[0098] For example, Figure 3 FIG. 1 is a flow chart of another method for calibrating driving parameters of a vehicle-mounted camera provided by an embodiment of the present invention. Figure 3 As shown, the process includes the following steps:

[0099] Step S301, the vehicle drives according to the initial motion trajectory;

[0100] Specifically, an initial motion trajectory of the vehicle for unmanned driving is set, and the vehicle is driven along the initial motion trajectory;

[0101] Step S302, using the currently collected vehicle motion data and camera motion data, determining the estimated value of the train parameters;

[0102] Specifically, motion data of the moving vehicle and the camera installed on the vehicle are collected to determine the vehicle motion data and the camera motion data, and the train parameter estimation value is determined according to formula (2);

[0103] Step S303, using the estimated train parameter values, calculate the control variables and information matrix trace of the vehicle's next trip;

[0104] Specifically, determine the vector form p of the train parameter estimation value, determine the information matrix as in formula (3) based on p, convert the solution of the maximum value of the information matrix into the solution of the minimum value of the inverse matrix of the information matrix as in formula (4), and determine the control variables and traces corresponding to the minimum of formula (4);

[0105] Step S304, whether the trace of the information matrix is ​​less than a preset threshold; if not, proceed to step S301, update the initial motion trajectory according to the control variable; if so, obtain the driving parameters and end.

[0106] Specifically, determine whether the trace of the information matrix is ​​less than a preset threshold, that is, determine whether enough information is currently collected; if so, determine the estimated value of the train parameters as the driving parameters and end the process; if not, return to step S301 to update the initial motion trajectory and redetermine the estimated value of the driving parameters.

[0107] In a possible implementation, the embodiment of the present invention may adopt a method based on optimal experimental design to determine experimental conditions so as to obtain an accurate estimation of driving parameters under given resource constraints.

[0108] Through the above method, it is possible to automatically determine whether the currently optimized driving parameters are reasonable without the need for additional calculation and detection, thereby reducing the amount of calculation in the calibration process; calculating the vehicle's control variables through the information matrix can speed up the calibration process and save computing resources; and calibrating in an automated manner can save labor costs during the calibration process.

[0109] Figure 4 FIG. 1 is a schematic diagram of a driving parameter calibration device for a vehicle-mounted camera provided by an embodiment of the present invention. Figure 4 As shown, the device can be applied to intelligent electronic devices such as servers and computers; the device includes: a collection module 401, a first determination module 402, a second determination module 403 and a calibration module 404;

[0110] The acquisition module 401 is used to determine the initial motion trajectory corresponding to the vehicle, and to collect motion data of the vehicle and the vehicle-mounted camera traveling along the initial motion trajectory;

[0111] A first determination module 402 is used to determine a driving parameter estimation value according to vehicle motion data corresponding to the vehicle and camera motion data corresponding to the vehicle-mounted camera;

[0112] The second determination module 403 is used to determine the control variables of the vehicle and / or the preset information values ​​according to the estimated driving parameters by using a preset formula; wherein the control variables are used to update the initial motion trajectory;

[0113] The calibration module 404 is used to determine the estimated driving parameter value as the driving parameter when the preset information value meets the preset condition, and calibrate the vehicle and the vehicle-mounted camera according to the driving parameter;

[0114] A first determination module 402 is used to determine a vehicle coordinate system corresponding to the vehicle and a camera coordinate system corresponding to the vehicle-mounted camera;

[0115] Determine vehicle motion data of the vehicle in the vehicle coordinate system and camera motion data of the vehicle-mounted camera in the camera coordinate system;

[0116] Determining a driving parameter estimation value so that a product of the vehicle motion data and the driving parameter estimation value is equal to a product of the camera motion data and the driving parameter estimation value;

[0117] The second determination module 403 is used to measure k groups of camera motion data between two frames according to a preset visual method, where the preset visual method includes at least one of a five-point method, an eight-point method, and a direct method; wherein k is a non-negative real number;

[0118] Measure k groups of vehicle motion data between two frames according to a preset mileage calculation method;

[0119] Determine camera motion estimation data based on vehicle motion data and driving parameter estimation values;

[0120] Determine the estimated driving parameter value corresponding to the minimum residual between the camera motion data and the camera motion estimated data;

[0121] The second determination module 403 is used to determine the vector form corresponding to the estimated value of the driving parameter, and determine the corresponding information matrix according to the estimated value of the driving parameter in the vector form;

[0122] Determine the control variable corresponding to the maximum value of the information matrix, and / or the preset information value;

[0123] A second determination module 403 is used to determine, based on the vehicle motion data, that the posture of the vehicle at the i-th moment is represented by the posture at the i-1-th moment and the control variable; wherein i is greater than or equal to 1;

[0124] Determine the inverse matrix corresponding to the information matrix, and convert the information matrix to a minimum value;

[0125] Determine the control variable and preset information value corresponding to when the inverse matrix takes the minimum value; wherein the preset information value represents the trace determined by the trace operation of the inverse matrix.

[0126] The calibration module 404 is used to update the initial motion trajectory of the vehicle according to the control variable when the preset information value is greater than or equal to the preset threshold value, so that the vehicle drives along the updated initial motion trajectory;

[0127] When the preset information value is less than the preset threshold, the estimated driving parameter value is determined as the driving parameter, and the vehicle and the on-board camera are calibrated according to the driving parameter.

[0128] It should be noted that: the driving parameter calibration device for a vehicle-mounted camera provided in the above embodiment only uses the division of the above program modules as an example to illustrate when implementing the corresponding driving parameter calibration method for a vehicle-mounted camera. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the above-described processing. Figure 1 The embodiments of the method shown belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0129] To implement the method of the embodiment of the present invention, the embodiment of the present invention provides a driving parameter calibration device for a vehicle-mounted camera, such as Figure 5 As shown, the device includes: a processor 501 and a memory 502 for storing a computer program that can be run on the processor; wherein,

[0130] When the processor 501 is used to run the computer program, it executes: determining the initial motion trajectory corresponding to the vehicle, collecting motion data of the vehicle and the on-board camera traveling along the initial motion trajectory; determining the estimated driving parameters based on the vehicle motion data corresponding to the vehicle and the camera motion data corresponding to the on-board camera; using a preset formula to determine the vehicle's control variables and / or preset information values ​​based on the estimated driving parameters; wherein the control variables are used to update the initial motion trajectory; when the preset information values ​​meet the preset conditions, determining the estimated driving parameters as the driving parameters, and calibrating the vehicle and the on-board camera based on the driving parameters. Specifically, the device can execute the following Figure 1 The method shown, and Figure 1 The driving parameter calibration method embodiment of the vehicle-mounted camera shown belongs to the same concept, and its specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0131] In practical application, such as Figure 5 As shown, the device may also include: at least one network interface 503. The various components in the vehicle-mounted camera driving parameter calibration device are coupled together via a bus system 504. It can be understood that the bus system 504 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 504 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 5 In the figure, various buses are marked as bus system 504. There may be at least one processor 501. The network interface 503 is used for wired or wireless communication between the driving parameter calibration device of the vehicle-mounted camera and other devices.

[0132] The memory 502 in the embodiment of the present invention is used to store various types of data to support the operation of the driving parameter calibration device of the vehicle-mounted camera.

[0133] The method disclosed in the above embodiment of the present invention can be applied to the processor 501, or implemented by the processor 501. The processor 501 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 501 or an instruction in the form of software.

[0134] The above-mentioned processor 501 can be a general-purpose processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 501 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor can be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present invention, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in the memory 502. The processor 501 reads the information in the memory 502 and completes the steps of the aforementioned method in combination with its hardware.

[0135] In an exemplary embodiment, the driving parameter calibration device of the vehicle-mounted camera can be implemented by one or more application-specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD), field-programmable gate array (FPGA), general-purpose processor, controller, microcontroller (MCU), microprocessor, or other electronic components to execute the aforementioned method.

[0136] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon; when the computer program is executed by a processor, the following is performed: determining an initial motion trajectory corresponding to the vehicle, and collecting motion data of the vehicle and the on-board camera traveling along the initial motion trajectory; determining a driving parameter estimation value based on the vehicle motion data corresponding to the vehicle and the camera motion data corresponding to the on-board camera; determining the vehicle's control variables and / or preset information values ​​using a preset formula based on the driving parameter estimation values; wherein the control variables are used to update the initial motion trajectory; when the preset information values ​​meet preset conditions, determining the driving parameter estimation values ​​as driving parameters, and calibrating the vehicle and the on-board camera based on the driving parameters. Specifically, the computer program can also perform the following operations: Figure 1 The method shown, and Figure 1 The method embodiments shown belong to the same concept, and their specific implementation processes are detailed in the method embodiments, which will not be repeated here.

[0137] In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0138] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0139] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0140] A person skilled in the art can understand that: all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, etc. Various media that can store program codes.

[0141] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0142] It should be noted that: "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0143] In addition, the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

[0144] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for calibrating driving parameters of a vehicle-mounted camera, characterized in that: The method comprises: Determine an initial motion trajectory corresponding to the vehicle, and collect motion data of the vehicle and the vehicle-mounted camera traveling along the initial motion trajectory; Determining a driving parameter estimation value according to the vehicle motion data corresponding to the vehicle and the camera motion data corresponding to the vehicle-mounted camera includes: Determine the vehicle coordinate system corresponding to the vehicle and the camera coordinate system corresponding to the onboard camera; Determine vehicle motion data of the vehicle in the vehicle coordinate system and camera motion data of the vehicle-mounted camera in the camera coordinate system; Determining a driving parameter estimation value so that a product of the vehicle motion data and the driving parameter estimation value is equal to a product of the camera motion data and the driving parameter estimation value; According to the estimated driving parameter value, a preset formula is used to determine the control variable and the preset information value of the vehicle; wherein the control variable is used to update the initial motion trajectory; When the preset information value meets the preset conditions, the driving parameter estimation value is determined as the driving parameter, and the vehicle and the vehicle-mounted camera are calibrated according to the driving parameters, including: when the preset information value is greater than or equal to a preset threshold, the initial motion trajectory of the vehicle is updated according to the control variable, so that the vehicle drives along the updated initial motion trajectory; when the preset information value is less than the preset threshold, the driving parameter estimation value is determined as the driving parameter, and the vehicle and the vehicle-mounted camera are calibrated according to the driving parameters.

2. The method according to claim 1, characterized in that The determining of the estimated driving parameter value so that the product of the vehicle motion data and the estimated driving parameter value is equal to the product of the camera motion data and the estimated driving parameter value includes: Measuring k groups of camera motion data between two frames according to a preset visual method, wherein the preset visual method includes at least one of a five-point method, an eight-point method, and a direct method; wherein k is a non-negative real number; Measure k groups of vehicle motion data between two frames according to a preset mileage calculation method; Determining camera motion estimation data based on the vehicle motion data and the driving parameter estimation value; Determine the driving parameter estimation value corresponding to when the residual difference between the camera motion data and the camera motion estimation data is minimum.

3. The method according to claim 1, characterized in that The step of determining the control variable of the vehicle and / or the preset information value by using a preset formula according to the estimated driving parameter value includes: Determine a vector form corresponding to the estimated value of the driving parameter, and determine a corresponding information matrix according to the estimated value of the driving parameter in the vector form; Determine the control variable and / or preset information value corresponding to the maximum value of the information matrix.

4. The method according to claim 3, characterized in that The determining of the control variable corresponding to the maximum value of the information matrix and / or the preset information value includes: Determining, based on the vehicle motion data, that the posture of the vehicle at the i-th moment is represented by the posture and the control variable at the i-1-th moment; wherein i is greater than or equal to 1; Determine an inverse matrix corresponding to the information matrix, and convert the determination of the maximum value of the information matrix into the determination of the minimum value of the inverse matrix; Determine the control variable and preset information value corresponding to when the inverse matrix takes the minimum value; wherein the preset information value represents the trace determined by performing a trace operation on the inverse matrix.

5. A driving parameter calibration device for a vehicle-mounted camera, characterized in that: The device comprises: A collection module, used to determine an initial motion trajectory corresponding to the vehicle, and collect motion data of the vehicle and the vehicle-mounted camera traveling along the initial motion trajectory; A first determination module is used to obtain vehicle motion data corresponding to the vehicle and camera motion data corresponding to the vehicle-mounted camera to determine a driving parameter estimation value; A second determination module is used to determine the control variable of the vehicle and / or the preset information value according to the estimated driving parameter value by using a preset formula; wherein the control variable is used to update the initial motion trajectory; A calibration module, configured to determine the estimated driving parameter value as a driving parameter when the preset information value satisfies a preset condition, and calibrate the vehicle and the vehicle-mounted camera according to the driving parameter; Wherein, the first determination module is specifically used to determine a vehicle coordinate system corresponding to the vehicle and a camera coordinate system corresponding to the vehicle-mounted camera; Determine vehicle motion data of the vehicle in the vehicle coordinate system and camera motion data of the vehicle-mounted camera in the camera coordinate system; Determining a driving parameter estimation value so that a product of the vehicle motion data and the driving parameter estimation value is equal to a product of the camera motion data and the driving parameter estimation value; The calibration module is specifically used to update the initial motion trajectory of the vehicle according to the control variable when the preset information value is greater than or equal to the preset threshold value, so that the vehicle drives along the updated initial motion trajectory; When the preset information value is less than the preset threshold, the driving parameter estimated value is determined as the driving parameter, and the vehicle and the vehicle-mounted camera are calibrated according to the driving parameter.

6. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the driving parameter calibration method of the vehicle-mounted camera according to any one of claims 1 to 4 by executing the computer instructions.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the driving parameter calibration method for a vehicle-mounted camera according to any one of claims 1 to 4.