On-line calibration method for pitch angle of vehicle-mounted camera and medium

Abstract

The invention discloses a vehicle-mounted camera pitch angle online calibration method and a medium, and the method comprises the steps: responding to a current frame image collected by a vehicle-mounted camera, and determining lane line information according to the image; determining the correction of the pitch angle of the camera according to the lane line information and the equal-width characteristics of the lane; and determining the calibration value of the camera pitch angle according to the correction amount and the initial value of the camera pitch angle. And the calibration precision is ensured, and the method is wide in application scene, is not limited by straightway scenes and can also be applied to curve scenes.

Description

Technical Field

[0001] The present application relates to the technical field of online calibration of pitch angles of vehicle-mounted cameras, and in particular to a method and medium for online calibration of pitch angles of vehicle-mounted cameras. Background Art

[0002] As a core module in autonomous driving technology, the accuracy of the perception module's results is crucial for downstream planning and control modules. Cameras, widely used sensors, can obtain obstacle location information through camera imaging and camera extrinsic parameters. During vehicle motion, factors such as vehicle load, tire pressure, and bumpy roads can cause inaccurate camera extrinsic parameters, leading to inaccurate obstacle position estimation. Therefore, accurately estimating camera extrinsic parameters in real time is crucial.

[0003] Methods for extrinsic parameter calibration for vehicle-mounted cameras primarily include offline and online calibration. Offline calibration primarily involves calibration in a calibration room upon vehicle manufacture. While this method is stable and mature, it places special demands on the calibration environment. Online calibration, performed while the vehicle is in motion, optimizes extrinsic parameters based on road information. This method has relatively low environmental requirements and is convenient to use.

[0004] Camera extrinsic parameters include roll, yaw, and pitch. The pitch has a greater impact on the estimation of obstacle position information, so real-time estimation of the pitch angle of the vehicle-mounted camera is an urgent problem to be solved.

[0005] In view of this, this application is hereby filed. Summary of the Invention

[0006] The following is a brief summary of one or more aspects to provide a basic understanding of these aspects. This summary is not an exhaustive overview of all conceivable aspects and is neither intended to identify key or critical elements of all aspects nor to define the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description that will be provided later.

[0007] The present application provides a method and medium for online calibration of the pitch angle of a vehicle-mounted camera, which ensures calibration accuracy and is applicable to a wide range of scenarios. It is not limited to straight road scenarios and can also be applied to curved scenarios.

[0008] In a first aspect, the present application provides a method for online calibration of pitch angles of a vehicle-mounted camera, comprising:

[0009] In response to the vehicle-mounted camera capturing a current frame image, determining lane line information based on the image;

[0010] Determining a correction amount for the camera pitch angle based on the lane line information and the lane equal width feature;

[0011] A calibration value of the camera pitch angle is determined according to the correction amount and an initial value of the camera pitch angle.

[0012] Furthermore, after obtaining the calibration value of the camera pitch angle, the method further includes:

[0013] updating the initial value using the calibration value, and in response to the vehicle-mounted camera capturing a next frame of image, iteratively calculating the calibration value based on the next frame of image;

[0014] The final calibration value of the camera pitch angle is determined based on the calibration values obtained through multiple iterative calculations.

[0015] Furthermore, determining a final calibration value of the camera pitch angle based on calibration values obtained through multiple iterative calculations includes:

[0016] The calibration values obtained by multiple iterative calculations are smoothed to obtain the final calibration value of the camera pitch angle.

[0017] Furthermore, determining a correction amount for the camera pitch angle according to the lane line information and the lane equal width feature includes:

[0018] Determine the width difference at different positions of the same lane according to the lane line information;

[0019] Substituting the width difference into a preset equation to obtain a correction value for the camera pitch angle, wherein the preset equation is constructed based on the lane width feature. Further, the method further includes:

[0020] According to the lane line information, an initial value of the camera pitch angle is determined by an extended Kalman filter.

[0021] Furthermore, the lane line information includes a point set of at least three lane lines, and determining the initial value of the camera pitch angle by using an extended Kalman filter based on the lane line information includes:

[0022] Calculate the vanishing point based on the point sets of two different lane lines;

[0023] Verifying the vanishing point based on a point set of other lane lines except the two different lane lines;

[0024] In response to the verification being passed, determining a reference value of the camera pitch angle according to the vanishing point, wherein the vanishing point is an intersection point formed on the image by lane lines parallel to each other in the physical world;

[0025] Initializing an extended Kalman filter according to a reference value of the camera pitch angle;

[0026] An initial value of the camera pitch angle is determined according to the lane line information using an extended Kalman filter.

[0027] Furthermore, the calculating of the vanishing point based on the point sets of two different lane lines includes:

[0028] Determine equations of the two different lane lines based on the point sets of the two different lane lines respectively;

[0029] An intersection point of the two different lane lines is calculated according to equations of the two different lane lines, and the intersection point is determined as the vanishing point.

[0030] Furthermore, verifying the vanishing point based on a point set of other lane lines except the two different lane lines includes:

[0031] Determine the midpoint of the other lane line based on the point set of the other lane line;

[0032] Determine the angle between the line connecting the vanishing point and the midpoint and the other lane lines;

[0033] If the angle is smaller than the preset value, it is determined that the verification is passed;

[0034] Determining a reference value of the camera pitch angle according to the vanishing point includes:

[0035] The reference value of the camera pitch angle is calculated by using a relationship between the vanishing point and the camera pitch angle.

[0036] Furthermore, the initializing the extended Kalman filter according to the reference value of the camera pitch angle includes:

[0037] Initializing the camera pitch angle using the reference value, wherein the camera pitch angle and the camera pitch angle change rate are state quantities of an extended Kalman filter, and the initial value of the camera pitch angle change rate is zero;

[0038] Determining the initial value of the camera pitch angle according to the lane line information by using an extended Kalman filter includes:

[0039] According to the state equation of the extended Kalman filter, the state quantity is predicted to obtain the predicted value of the pitch angle;

[0040] The predicted value is updated according to the Kalman gain and the endpoint of the lane line to obtain the initial value, wherein the lane line information includes the endpoint of the lane line.

[0041] In a second aspect, the present application also provides an online calibration device for the pitch angle of a vehicle-mounted camera, comprising:

[0042] A first determining module is configured to determine lane line information based on a current frame image captured by the vehicle-mounted camera;

[0043] A second determination module is used to determine a correction amount of the camera pitch angle according to the lane line information and the lane equal width feature;

[0044] The correction module is used to determine a calibration value of the camera pitch angle according to the correction amount and an initial value of the camera pitch angle.

[0045] In a third aspect, the present application further provides an electronic device, comprising:

[0046] one or more processors;

[0047] a storage device for storing one or more programs;

[0048] When the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned on-line calibration method for the pitch angle of the vehicle-mounted camera.

[0049] In a fourth aspect, the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned online calibration method for the pitch angle of a vehicle-mounted camera.

[0050] The online pitch angle calibration method for an on-board camera disclosed in this application ensures calibration accuracy and is applicable to a wide range of scenarios, not limited to straight roads but also applicable to curved roads. It also avoids strong dependence on and sensitivity to vanishing points, improving the practicality of the online calibration method. Specifically, it includes the following steps: in response to the on-board camera capturing a current frame image, determining lane line information based on the image; determining a correction for the camera's pitch angle based on the lane line information and lane width characteristics; and determining a calibration value for the camera's pitch angle based on the correction and the initial value of the camera's pitch angle. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0052] Figure 1 A schematic diagram of the process of online calibration method of pitch angle of vehicle-mounted camera provided in the embodiment of the present application Figure 1 ;

[0053] Figure 2 A schematic diagram of correcting a lane line using a correction value provided in an embodiment of the present application;

[0054] Figure 3 A schematic diagram of the angle between the line connecting the vanishing point and the midpoint and the other lane lines provided in an embodiment of the present application;

[0055] Figure 4 A schematic diagram of the process of online calibration method of pitch angle of vehicle-mounted camera provided in the embodiment of the present application Figure 2 ;

[0056] Figure 5 A schematic diagram of the structure of an online calibration method for the pitch angle of a vehicle-mounted camera provided in an embodiment of the present application;

[0057] Figure 6 This is a structural diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0058] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the invention are shown in the accompanying drawings.

[0059] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0060] Figure 1 This is a flow chart of an online pitch angle calibration method for a vehicle-mounted camera proposed in this application. The online pitch angle calibration method for a vehicle-mounted camera includes the following steps:

[0061] S110 : In response to the vehicle-mounted camera capturing a current frame image, determine lane line information based on the image.

[0062] In other words, the essence of online calibration is that the vehicle uses the on-board camera to capture the surrounding roads and environment while driving, obtains real-time images, and then recognizes lane line information based on the real-time collected images, and corrects the pitch angle of the on-board camera according to the recognized lane line information.

[0063] For example, an image can be fed into a neural network model to identify lane line instances, i.e., lane line information. Lane line information is specifically a set of points that make up the lane line, with each point in the set carrying coordinate information.

[0064] In some embodiments, in order to improve the calibration accuracy and efficiency, the lane line information is further filtered and processed, for example, lane lines with shorter lengths (lane lines with shorter lengths will disappear in the images captured by the camera as the vehicle travels, which is not conducive to subsequent continuous calibration), lane lines with farther starting points (for lane lines with farther starting points, the recognition error is usually larger, so it is recommended to filter them out to further improve the calibration accuracy), and lane lines with closer end points (which are actually also shorter lane lines, which are not conducive to subsequent continuous calibration, so they are filtered out). Furthermore, lane line points within a certain distance range are filtered according to the curvature to improve the quality of the lane lines, lay the foundation for subsequent calibration, and ensure the final effect of the calibration. Furthermore, noise points outside the lane lines can be eliminated to ultimately obtain a high-quality point set of lane lines that are conducive to calibration.

[0065] S220: Determine a correction amount for the camera pitch angle according to the lane line information and the lane equal width feature.

[0066] On highways, overpasses, expressways and other roads, lane lines and lanes are relatively standardized, and the lanes have equal width characteristics. Based on this, this application corrects the pitch angle of the on-board camera based on the lanes captured by the on-board camera, assuming that the lanes are equal width.

[0067] Specifically, determining the width difference at different positions of the same lane according to the lane line information;

[0068] The width difference is substituted into a preset relationship to obtain a correction value of the camera pitch angle, wherein the preset relationship is constructed based on a lane equal width feature.

[0069] Exemplarily, the preset relationship is the following relationship (1):

[0070]

[0071] Where i represents the number of iterative calculations, ΔP i Indicates the correction amount of the camera pitch angle in the i-th iteration calculation, asin() represents the arc sine function, W f,i and W c,i Respectively represent the width of the same lane at different positions in the image, H represents the height of the vehicle-mounted camera, and L represents the length of the shorter lane line of the two lane lines that make up the same lane. The role of is to limit the correction step size, and the role of asin() is to limit the final result to a certain range. f,i -W c,i Indicates the difference in lane width at different locations. When W f,i With W c,i When they become the same, the correction amount tends to 0.

[0072] Furthermore, in order to improve the degree of single correction and accelerate the correction iteration process, W f,i and W c,i Set to the lane width at the beginning and end of the lane in the image, respectively.

[0073] For example, see Figure 2 A schematic diagram of correcting lane lines by using a correction amount is shown, wherein lane lines 210a and 210b represent the two lane lines before correction, and it can be seen that the two are not parallel. Lane lines 220a and 220b represent the two lane lines after correction, and it can be clearly seen that lane lines 220a and 220b are closer to parallel.

[0074] By determining the correction amount of the camera pitch angle according to the lane line information based on the feature of equal lane width, the method is not limited to straight road scenes and can also be applied to curved scenes. It avoids the dependence on the vanishing point and improves the applicability of the calibration method.

[0075] S230: Determine a calibration value of the camera pitch angle according to the correction amount and the initial value of the camera pitch angle.

[0076] In some embodiments, the sum of the correction amount and the initial value of the camera pitch angle is determined as the calibration value of the camera pitch angle. The initial value of the camera pitch angle can be obtained through offline calibration or other mature calibration methods.

[0077] In some implementations, in order to improve calibration accuracy, the initial value of the camera pitch angle is limited. The initial value of the camera pitch angle is obtained as follows:

[0078] According to the lane line information, an initial value of the camera pitch angle is determined by an extended Kalman filter.

[0079] Specifically, the lane line information includes a point set of at least three lane lines, and determining the initial value of the camera pitch angle by using an extended Kalman filter based on the lane line information includes:

[0080] A vanishing point is calculated based on a point set of two different lane lines; the vanishing point is verified based on a point set of other lane lines other than the two different lane lines; in response to passing the verification, a reference value of a camera pitch angle is determined based on the vanishing point, wherein the vanishing point is an intersection point formed on the image by lane lines parallel to each other in the physical world; an extended Kalman filter is initialized based on the reference value of the camera pitch angle; and an initial value of the camera pitch angle is determined based on the lane line information using the extended Kalman filter.

[0081] The calculating of the vanishing point based on the point sets of two different lane lines includes:

[0082] Based on the point sets of the two different lane lines, the equations of the two different lane lines are respectively determined (for example, the lane line equations can be obtained by fitting the point sets of the lane lines using the least squares method); the intersection of the two different lane lines is calculated according to the equations of the two different lane lines, and the intersection is determined as the vanishing point.

[0083] The verifying the vanishing point based on a point set of other lane lines except the two different lane lines includes:

[0084] Determine the midpoint of the other lane line based on the point set of the other lane line; determine the angle between the line connecting the vanishing point and the midpoint and the other lane line; if the angle is less than a preset value, determine that the verification is passed. Figure 3 A schematic diagram of the angle between the line connecting the vanishing point and the midpoint and the other lane lines is shown, where reference numeral 310 represents the vanishing point, and reference numeral 320 represents the midpoint of the lane line not involved in the vanishing point calculation. If the angle θ is less than a preset value, it indicates that the error in the vanishing point solution is within the allowable range and verification is passed.

[0085] Determining a reference value of the camera pitch angle according to the vanishing point includes:

[0086] The reference value of the camera pitch angle is calculated by using a relationship between the vanishing point and the camera pitch angle.

[0087] For example, the relationship between the vanishing point and the camera pitch angle is expressed by the following equations (2)-(4):

[0088]

[0089] Among them, K represents the camera intrinsic parameter, (u, v) represents the coordinates of the vanishing point, pitch represents the pitch angle, which is the unknown quantity to be solved, and the obtained pitch is the initial value, and yaw represents the yaw angle, which is a known quantity. In summary, according to the above relationship, pitch can be solved as the initial value.

[0090] Exemplarily, the initializing the extended Kalman filter according to the reference value of the camera pitch angle includes:

[0091] The camera pitch angle is initialized using the reference value, wherein the camera pitch angle and the camera pitch angle change rate are state quantities of an extended Kalman filter, and an initial value of the camera pitch angle change rate is zero.

[0092] Determining the initial value of the camera pitch angle according to the lane line information by using an extended Kalman filter includes:

[0093] According to the state equation of the extended Kalman filter, the state quantity is predicted to obtain a predicted value of the pitch angle; the predicted value is updated according to the Kalman gain and the endpoint of the lane line to obtain the initial value, wherein the lane line information includes the endpoint of the lane line.

[0094] Specifically, the process of initializing the Kalman filter, predicting the state quantity, and updating the predicted value is expressed by the following equations (5)-(12):

[0095] X p =[θ ω θ ] T (5)

[0096]

[0097] Among them, X p Represents the state quantity of filter optimization, state quantity θ, ω θ denote the camera pitch angle and the angular velocity of the camera pitch angle respectively. represents the predicted value of the state quantity at time t, Indicates the updated value of the state quantity at time t-1. When t=1, is the reference value of the state quantity, that is, the value given during initialization. Equation (6) represents the state quantity prediction process, where f p (X p ) is the state transfer function, and equation (7) is the specific form of the state transfer function, W p is the noise variance that conforms to the normal distribution. Equation (8) represents the covariance of the state quantity after prediction, represents the predicted covariance, E t is the Jacobian matrix corresponding to the state transfer function, W t is the process noise.

[0098]

[0099] After the filter is updated, the state quantity can be obtained and the corresponding covariance P t , where H t is the Jacobian matrix of the observation equation, K is the Kalman gain, h p (X p ) represents the observation equation, which can be obtained by the following equations (11) and (12).

[0100]

[0101] n=(K -1 p1)×(K -1 p2)(12)

[0102] In equation (11), Represents the direction vector of the X axis in the world coordinate system, R is the external parameter matrix of the camera, q p is the noise variance of the measurement model, which obeys the normal distribution. In equation (12), p1 and p2 are the two endpoints of the lane line.

[0103] By determining the initial value of the camera pitch angle through an extended Kalman filter according to the lane line information, the final calibration accuracy and calibration efficiency can be improved, and the calibration convergence process can be accelerated.

[0104] On the basis of the above embodiment, in order to further improve the calibration accuracy, calibration values of multiple camera pitch angles may be determined through multiple iterative calculations, and then a final calibration value may be determined based on the multiple calibration values.

[0105] Exemplarily, after the calibration value of the camera pitch angle is calculated through S110-S130, the calibration value is used to update the initial value, that is, the initial value is modified to the calibration value, and the lane line information is recognized based on the new image collected in real time by the on-board camera, and S120-S130 are repeated to obtain a calibration value again. The initial value is updated with the latest calibration value, and the lane line information is recognized based on the new image collected in real time by the on-board camera, and S120-S130 are repeated to obtain a calibration value again. In this way, the loop is iterated to obtain multiple calibration values, and the final calibration value of the camera pitch angle is determined based on the calibration values obtained by multiple iterative calculations. The iterative termination condition can be that the number of iterations reaches a preset value, or the correction amount obtained after the iteration is less than the set value.

[0106] Furthermore, determining a final calibration value of the camera pitch angle based on calibration values obtained through multiple iterative calculations includes:

[0107] The calibration values obtained by multiple iterative calculations are smoothed to obtain the final calibration value of the camera pitch angle.

[0108] For example, the average value of calibration values obtained through multiple iterative calculations is determined as the final calibration value of the camera pitch angle.

[0109] Alternatively, the calibration values obtained by multiple iterative calculations are smoothed by other smoothing methods such as window smoothing method and histogram smoothing method to obtain the final calibration value of the camera pitch angle.

[0110] Based on the above embodiments, Figure 4The flowchart of a method for online calibration of the pitch angle of a vehicle-mounted camera shown in FIG. 1 includes the following steps:

[0111] S1. The vehicle-mounted camera collects road images.

[0112] S2. Analyze the image and identify lane line information.

[0113] Specifically, an image can be fed into a neural network model, which then identifies lane line instances, or lane line information. Lane line information is specifically a set of points that make up a lane line, with each point in the set carrying coordinate information.

[0114] S3. Determine whether the number of lane lines reaches a threshold. If not, execute S4; otherwise, execute S5.

[0115] S4. Exit the calibration process.

[0116] S5. Lane line information preprocessing.

[0117] Lane line information preprocessing includes: filtering out lane lines with shorter lengths (lane lines with shorter lengths will disappear in the images captured by the camera as the vehicle travels, which is not conducive to subsequent continuous calibration), lane lines with farther starting points (for lane lines with farther starting points, the recognition error is usually larger, so it is recommended to filter them out to further improve the calibration accuracy), and lane lines with closer end points (which are actually shorter lane lines, which are not conducive to subsequent continuous calibration, so they are filtered out). Furthermore, lane line points within a certain distance range are filtered according to the curvature to improve the quality of the lane lines, lay the foundation for subsequent calibration, and ensure the final effect of the calibration. Furthermore, noise points outside the lane lines can be removed to ultimately obtain a high-quality point set of lane lines that are conducive to calibration.

[0118] S6. Calculate a vanishing point based on the preprocessed lane line, and calculate a reference value of the camera pitch angle based on the vanishing point.

[0119] S7. Initialize the extended Kalman filter using the reference value.

[0120] S8. Update the extended Kalman filter according to the lane line information to obtain the initial value of the camera pitch angle.

[0121] S9. Based on the lane width feature, calculate the correction amount of the pitch angle, and determine the calibration value in combination with the initial value of the camera pitch angle.

[0122] S10: Update the initial value using the calibration value.

[0123] S11. Recognize the real-time image captured by the camera to obtain lane line information.

[0124] S12: Preprocess lane line information.

[0125] S13. Based on the lane width feature, calculate the correction value of the pitch angle, combine it with the initial value to obtain a calibration value, and return to execute S10.

[0126] S14. Store the calibration value obtained by each iterative calculation.

[0127] S15: Smoothing the multiple calibration values to obtain a final calibration value.

[0128] Based on the above embodiments, Figure 5 This is a schematic diagram of the structure of an on-board camera pitch angle online calibration device provided by an embodiment of the present application. Figure 5 As shown, the device includes: a first determination module 510, which is used to determine lane line information based on the current frame image captured by the vehicle-mounted camera; a second determination module 520, which is used to determine the correction amount of the camera pitch angle based on the lane line information and the lane equal width feature; and a correction module 530, which is used to determine the calibration value of the camera pitch angle based on the correction amount and the initial value of the camera pitch angle.

[0129] Furthermore, it also includes: an updating module, used to update the initial value using the calibration value, and in response to the vehicle-mounted camera capturing the next frame image, iteratively calculate the calibration value based on the next frame image; a third determination module, used to determine the final calibration value of the camera pitch angle based on the calibration values obtained by multiple iterative calculations.

[0130] Furthermore, the third determination module is specifically configured to: perform smoothing processing on calibration values respectively obtained through multiple iterative calculations to obtain a final calibration value of the camera pitch angle.

[0131] Furthermore, the second determination module 520 is specifically configured to: determine a width difference at different positions of the same lane based on the lane line information; and substitute the width difference into a preset relationship to obtain a correction value for the camera pitch angle, wherein the preset relationship is constructed based on the equal width feature of the lanes, for example:

[0132]

[0133] Where i represents the number of iterative calculations, ΔP i Indicates the correction amount of the camera pitch angle in the i-th iteration calculation, asin() represents the arc sine function, W f,i and W c,i They represent the width of the same lane at different positions in the image, H represents the height of the vehicle-mounted camera, and L represents the length of the shorter lane line of the two lane lines that make up the same lane.

[0134] Furthermore, it also includes: a fourth determination module, used to determine the initial value of the camera pitch angle through an extended Kalman filter according to the lane line information.

[0135] Furthermore, the fourth determination module includes: a vanishing point calculation unit, used to calculate the vanishing point based on a point set of two different lane lines; a verification unit, used to verify the vanishing point based on a point set of other lane lines other than the two different lane lines; a reference value determination unit, used to determine a reference value of the camera pitch angle based on the vanishing point in response to passing the verification, wherein the vanishing point is an intersection point formed on the image by lane lines parallel to each other in the physical world; an initialization unit, used to initialize the extended Kalman filter based on the reference value of the camera pitch angle; and an initial value determination unit, used to determine the initial value of the camera pitch angle based on the lane line information through the extended Kalman filter.

[0136] Furthermore, the vanishing point calculation unit is specifically used to: determine the equations of the two different lane lines based on the point sets of the two different lane lines respectively; calculate the intersection of the two different lane lines according to the equations of the two different lane lines, and determine the intersection as the vanishing point.

[0137] Furthermore, the verification unit is specifically used to: determine the midpoint of the other lane line based on the point set of the other lane line; determine the angle between the line connecting the vanishing point and the midpoint and the other lane line; if the angle is less than a preset value, determine that the verification is passed.

[0138] Furthermore, the reference value determining unit is specifically configured to calculate a reference value of the camera pitch angle according to a relationship between a vanishing point and a camera pitch angle.

[0139] Furthermore, the initialization unit is specifically used to: initialize the camera pitch angle using the reference value, wherein the camera pitch angle and the camera pitch angle change rate are state quantities of the extended Kalman filter, and the initial value of the camera pitch angle change rate is zero; the initial value determination unit is specifically used to: predict the state quantity according to the state equation of the extended Kalman filter to obtain a predicted value of the pitch angle; update the predicted value according to the Kalman gain and the endpoint of the lane line to obtain the initial value, wherein the lane line information includes the endpoint of the lane line.

[0140] The on-board camera pitch angle online calibration device provided by the embodiment of the present disclosure can execute the steps of the on-board camera pitch angle online calibration method provided by the method embodiment of the present disclosure, and the execution steps and beneficial effects are no longer repeated here.

[0141] Figure 6 This is a schematic diagram of the structure of an electronic device in the embodiment of the present disclosure. Figure 6 , which shows a structural diagram of an electronic device 500 suitable for implementing the embodiments of the present disclosure. Figure 6 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0142] like Figure 6 As shown, the electronic device 500 may include a processing device (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes to implement the method of the embodiment as described in the present disclosure according to the program stored in the read-only memory (ROM) or the program loaded from the storage device 508 into the random access memory (RAM). In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the ROM 502 and the RAM 503 are connected to each other via a bus 504. The I / O interface 505 is also connected to the bus 504. The input device 506, the output device 507, the storage device 508 and the communication device 509 are all connected to the I / O interface 505.

[0143] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart, thereby implementing the above-mentioned vehicle-mounted camera pitch angle online calibration method. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0144] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0145] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device. The computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device performs the method steps of the present application.

[0146] Optionally, when the above one or more programs are executed by the electronic device, the electronic device may also execute other steps described in the above embodiments.

[0147] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0148] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0149] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. The above is only the preferred implementation method of this application. It should be pointed out that due to the limitations of textual expression, there are objectively infinite specific structures. For ordinary technicians in this technical field, without departing from the principles of this application, they can also make several improvements, modifications or changes, and can also combine the above technical features in an appropriate manner; these improvements, modifications, changes or combinations, or the direct application of the inventive concept and technical solution to other occasions without improvement, should be regarded as the scope of protection of this application.

Claims

1. A method for online calibration of pitch angle of a vehicle-mounted camera, characterized in that: include: In response to the vehicle-mounted camera capturing a current frame image, determining lane line information based on the image; Determining a correction amount for the camera pitch angle based on the lane line information and the lane equal width feature; A calibration value of the camera pitch angle is determined according to the correction amount and an initial value of the camera pitch angle.

2. The on-line calibration method for the pitch angle of a vehicle-mounted camera according to claim 1, characterized in that: After determining the calibration value of the camera pitch angle according to the correction amount and the initial value of the camera pitch angle, the method further includes: updating the initial value using the calibration value, and in response to the vehicle-mounted camera capturing a next frame of image, iteratively calculating the calibration value based on the next frame of image; The final calibration value of the camera pitch angle is determined based on the calibration values obtained through multiple iterative calculations.

3. The on-line calibration method for the pitch angle of a vehicle-mounted camera according to claim 2, characterized in that: The determining of a final calibration value of the camera pitch angle based on calibration values respectively obtained through multiple iterative calculations includes: The calibration values obtained by multiple iterative calculations are smoothed to obtain the final calibration value of the camera pitch angle.

4. The on-line calibration method for the pitch angle of a vehicle-mounted camera according to claim 1, characterized in that: The determining of the correction amount of the camera pitch angle according to the lane line information and the lane equal width feature includes: Determine the width difference at different positions of the same lane according to the lane line information; The width difference is substituted into a preset relationship to obtain a correction value of the camera pitch angle, wherein the preset relationship is constructed based on a lane equal width feature.

5. The on-line calibration method for the pitch angle of a vehicle-mounted camera according to claim 1, characterized in that: The method further comprises: According to the lane line information, an initial value of the camera pitch angle is determined by an extended Kalman filter.

6. The on-line calibration method for the pitch angle of a vehicle-mounted camera according to claim 5, characterized in that: The lane line information includes a point set of at least three lane lines, and determining the initial value of the camera pitch angle by using an extended Kalman filter based on the lane line information includes: Calculating a vanishing point based on a point set of two different lane lines, wherein the vanishing point is an intersection point formed on the image by lane lines that are parallel to each other in the physical world; Verifying the vanishing point based on a point set of other lane lines except the two different lane lines; In response to the verification being passed, determining a reference value of a camera pitch angle according to the vanishing point; Initializing an extended Kalman filter according to a reference value of the camera pitch angle; An initial value of the camera pitch angle is determined according to the lane line information using an extended Kalman filter.

7. The on-line calibration method for the pitch angle of a vehicle-mounted camera according to claim 6, characterized in that: The calculating of the vanishing point based on the point sets of two different lane lines includes: Determine equations of the two different lane lines based on the point sets of the two different lane lines respectively; An intersection point of the two different lane lines is calculated according to equations of the two different lane lines, and the intersection point is determined as the vanishing point.

8. The on-line calibration method for pitch angle of a vehicle-mounted camera according to claim 6, characterized in that: The verifying the vanishing point based on a point set of other lane lines except the two different lane lines includes: Determining the midpoint of the other lane line based on the point set of the other lane line; Determine the angle between the line connecting the vanishing point and the midpoint and the other lane lines; If the angle is smaller than the preset value, it is determined that the verification is passed; Determining a reference value of the camera pitch angle according to the vanishing point includes: The reference value of the camera pitch angle is calculated by using a relationship between the vanishing point and the camera pitch angle.

9. The on-line calibration method for pitch angle of a vehicle-mounted camera according to claim 6, characterized in that: Initializing the extended Kalman filter according to the reference value of the camera pitch angle includes: Initializing the camera pitch angle using the reference value, wherein the camera pitch angle and the camera pitch angle change rate are state quantities of an extended Kalman filter, and the initial value of the camera pitch angle change rate is zero; Determining the initial value of the camera pitch angle according to the lane line information by using an extended Kalman filter includes: According to the state equation of the extended Kalman filter, the state quantity is predicted to obtain the predicted value of the pitch angle; The predicted value is updated according to the Kalman gain and the endpoint of the lane line to obtain the initial value, wherein the lane line information includes the endpoint of the lane line.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the online calibration method for the pitch angle of a vehicle-mounted camera according to any one of claims 1 to 8 is implemented.

Images