Onboard camera pitch angle online calibration method and medium

Abstract

The application discloses a vehicle-mounted camera pitch angle online calibration method and a medium. The vehicle-mounted camera pitch angle online calibration method comprises the following steps: in response to the fact that a vehicle-mounted camera collects a current frame image, determining lane line information according to the image; determining a correction amount of a camera pitch angle according to the lane line information and lane equal-width characteristics; and determining a calibration value of the camera pitch angle according to the correction amount and an initial value of the camera pitch angle. The calibration accuracy is ensured, and the method is suitable for a wide range of scenes and is not limited to straight road scenes, and can also be applied to curved road scenes.

Description

Technical Field

[0001] This application relates to the field of online calibration technology for vehicle-mounted camera pitch angle, specifically to a method and medium for online calibration of vehicle-mounted camera pitch angle. Background Technology

[0002] As a core module of autonomous driving technology, the accuracy of the perception results from the perception module is crucial for downstream planning and control modules. Cameras, as widely used sensors, obtain obstacle location information through image capture and the analysis of camera intrinsic and extrinsic parameters. During vehicle movement, factors such as vehicle load, tire pressure, and bumpy road surfaces can cause inaccurate camera extrinsic parameters, leading to inaccurate obstacle location estimations. Therefore, real-time and accurate estimation of camera extrinsic parameters is essential.

[0003] External parameter calibration methods for vehicle-mounted cameras mainly include offline calibration and online calibration. Offline calibration refers to calibration performed in a calibration room during vehicle manufacturing. This method is stable and mature, but it has specific requirements for the calibration environment. Online calibration, on the other hand, is performed while the vehicle is in motion, optimizing external parameters based on road information. This method has relatively lower requirements for the calibration environment and is convenient to use.

[0004] Camera extrinsic parameters include roll, yaw, and pitch. Among them, pitch has a significant impact on the estimation of obstacle position information, so real-time estimation of the pitch angle of the vehicle-mounted camera is an urgent problem to be solved.

[0005] In view of the above, this application is hereby submitted. Summary of the Invention

[0006] The following provides a brief overview of one or more aspects to offer a basic understanding of them. This overview is not an exhaustive summary of all conceived aspects, nor is it intended to identify key or decisive elements of all aspects, nor to define the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form to prepare for the more detailed descriptions that follow.

[0007] This application provides an online calibration method and medium for the pitch angle of a vehicle-mounted camera, which ensures calibration accuracy and has a wide range of applications, not limited to straight road scenarios, but also applicable to curved road scenarios.

[0008] Firstly, this application provides an online calibration method for the pitch angle of a vehicle-mounted camera, including:

[0009] In response to the vehicle-mounted camera capturing the current frame image, lane line information is determined based on the image;

[0010] The correction amount for the camera pitch angle is determined based on the lane line information and lane width characteristics.

[0011] The calibration value of the camera pitch angle is determined based on the correction amount and the initial value of the camera pitch angle.

[0012] Furthermore, after obtaining the calibration value of the camera pitch angle, the process also includes:

[0013] The initial value is updated using the calibration value, and in response to the vehicle-mounted camera acquiring the next frame image, the calibration value is iteratively calculated based on the next frame image;

[0014] The final calibration value of the camera pitch angle is determined based on the calibration values ​​obtained from multiple iterative calculations.

[0015] Furthermore, determining the final calibration value of the camera pitch angle based on the calibration values ​​obtained from multiple iterative calculations includes:

[0016] The calibration values ​​obtained from multiple iterations are smoothed to obtain the final calibration value of the camera pitch angle.

[0017] Furthermore, determining the correction amount for the camera pitch angle based on the lane line information and lane width characteristics includes:

[0018] Determine the width difference at different locations within the same lane based on the lane line information;

[0019] Substituting the width difference into a preset formula yields the correction amount for the camera pitch angle, wherein the preset formula is constructed based on lane width equalization features. Furthermore, the method also includes:

[0020] Based on the lane line information, the initial value of the camera pitch angle is determined by an extended Kalman filter.

[0021] Furthermore, the lane line information includes a point set of at least three lane lines, and the step of determining the initial value of the camera pitch angle using an extended Kalman filter based on the lane line information includes:

[0022] Calculate the vanishing point based on the point sets of two different lane lines;

[0023] The vanishing point is verified based on the point set of other lane lines besides the two different lane lines.

[0024] In response to successful verification, a reference value for the camera pitch angle is determined based on the vanishing point, wherein the vanishing point is the intersection point formed by parallel lane lines in the physical world on the image.

[0025] The extended Kalman filter is initialized based on the reference value of the camera pitch angle;

[0026] The initial value of the camera pitch angle is determined based on the lane line information using an extended Kalman filter.

[0027] Furthermore, the calculation of the vanishing point based on the point set of two different lane lines includes:

[0028] Determine the equations of the two different lane lines based on the point sets of the two different lane lines respectively;

[0029] The intersection point of the two different lane lines is calculated based on their equations, and the intersection point is determined as the vanishing point.

[0030] Furthermore, the verification of the vanishing point based on the point set of other lane lines besides the two different lane lines includes:

[0031] Determine the midpoint of the other lane lines based on the point set of the other lane lines;

[0032] Determine the angle between the line connecting the vanishing point and the midpoint and the other lane lines;

[0033] If the included angle is less than the preset value, the verification is considered successful.

[0034] The step of determining the reference value for the camera pitch angle based on the vanishing point includes:

[0035] The reference value of the camera pitch angle is calculated using the relationship between the vanishing point and the camera pitch angle.

[0036] Furthermore, the initialization of the extended Kalman filter based on the reference value of the camera pitch angle includes:

[0037] The camera pitch angle is initialized using the reference value, wherein the camera pitch angle and the rate of change of the camera pitch angle are state variables of the extended Kalman filter, and the initial value of the rate of change of the camera pitch angle is zero;

[0038] The step of determining the initial value of the camera pitch angle based on the lane line information using an extended Kalman filter includes:

[0039] Based on the state equation of the extended Kalman filter, the state variables are predicted to obtain the predicted pitch angle.

[0040] The predicted value is updated based on the Kalman gain and the endpoints of the lane lines to obtain the initial value, wherein the lane line information includes the endpoints of the lane lines.

[0041] Secondly, this application also provides an online calibration device for the pitch angle of a vehicle-mounted camera, comprising:

[0042] The first determining module is used to determine lane line information based on the image captured by the vehicle-mounted camera in response to the current frame image being acquired.

[0043] The second determining module is used to determine the correction amount of the camera pitch angle based on the lane line information and lane width characteristics.

[0044] The correction module is used to determine the calibration value of the camera pitch angle based on the correction amount and the initial value of the camera pitch angle.

[0045] Thirdly, this application also provides an electronic device, the electronic device comprising:

[0046] One or more processors;

[0047] Storage device for storing one or more programs;

[0048] When the one or more programs are executed by the one or more processors, the one or more processors implement the online calibration method for the pitch angle of the vehicle-mounted camera as described above.

[0049] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the online calibration method for the pitch angle of a vehicle-mounted camera as described above.

[0050] The online calibration method for the pitch angle of a vehicle-mounted camera disclosed in this application ensures calibration accuracy and has wide applicability, not limited to straight road scenarios but also applicable to curved road scenarios. It avoids strong dependence on and sensitivity to vanishing points, thus improving the practicality of the online calibration method. Specifically, it includes the following steps: in response to the vehicle-mounted camera acquiring the current frame image, determining lane line information based on the image; determining a correction amount for the camera pitch angle based on the lane line information and lane width equalization features; and determining the calibration value of the camera pitch angle based on the correction amount and the initial value of the camera pitch angle. Attached Figure Description

[0051] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 A flowchart illustrating an online calibration method for the pitch angle of a vehicle-mounted camera provided in this application embodiment. Figure 1 ;

[0053] Figure 2 This is a schematic diagram illustrating the correction of lane lines using a correction amount, as provided in an embodiment of this application.

[0054] Figure 3 A schematic diagram illustrating the angle between the line connecting the vanishing point and the midpoint and the other lane lines, provided as an embodiment of this application;

[0055] Figure 4 A flowchart illustrating an online calibration method for the pitch angle of a vehicle-mounted camera provided in this application embodiment. Figure 2 ;

[0056] Figure 5 A schematic diagram of a method for online calibration of the pitch angle of a vehicle-mounted camera provided in this application embodiment;

[0057] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0058] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0059] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0060] Figure 1 This is a flowchart illustrating an online calibration method for the pitch angle of a vehicle-mounted camera proposed in this application. The online calibration method for the pitch angle of a vehicle-mounted camera includes the following steps:

[0061] S110. In response to the vehicle-mounted camera capturing the current frame image, determine lane line information based on the image.

[0062] In other words, online calibration essentially involves the vehicle taking pictures of the surrounding road and environment through an onboard camera while driving, obtaining real-time images, and then identifying lane line information based on the real-time acquired images, and correcting the pitch angle of the onboard camera according to the identified lane line information.

[0063] For example, an image can be input into a neural network model, which can then identify lane line instances, i.e., lane line information. Lane line information specifically consists of a set of points that make up the lane lines, with each point in the set carrying coordinate information.

[0064] In some implementations, to improve calibration accuracy and efficiency, the lane line information is further filtered. For example, lane lines that are too short (short lane lines disappear from the camera's image as the vehicle moves, hindering subsequent calibration), have distant starting points (lane lines with distant starting points typically have larger recognition errors, so filtering them is recommended to further improve calibration accuracy), or have close ending points (which are essentially also short lane lines, hindering subsequent calibration, so they are filtered out) are filtered out. Furthermore, lane line points within a certain distance range are filtered based on curvature to improve lane line quality, laying the foundation for subsequent calibration and ensuring the final calibration effect. Additionally, noise outside the lane lines can be removed to ultimately obtain a high-quality set of lane line points conducive to calibration.

[0065] S220. Determine the correction amount for the camera pitch angle based on the lane line information and lane width characteristics.

[0066] On highways, viaducts, expressways, and other roads, lane lines and lanes are relatively standardized, and lanes have equal width characteristics. Based on this, this application corrects the pitch angle of the vehicle-mounted camera based on the lanes captured by the vehicle-mounted camera, assuming that the lanes are of equal width.

[0067] Specifically, the width difference at different locations within the same lane is determined based on the lane line information;

[0068] Substituting the width difference into a preset formula yields the correction amount for the camera pitch angle, wherein the preset formula is constructed based on lane width characteristics.

[0069] For example, the preset relation is as follows (1):

[0070]

[0071] Where i represents the number of iterations, ΔP i This represents the correction amount of the camera pitch angle in the i-th iteration calculation, asin() represents the arcsine function, and W f,i and W c,i Let H and L represent the width of the same lane at different locations in the image, respectively; H represent the height of the vehicle-mounted camera; and L represent the length of the shorter lane line among the two lane lines that make up the same lane. In the above formula... The function of `asin()` is to limit the correction step size, and the function of `asin()` is to limit the final result to a certain range. Among these, W... f,i -W c,i This represents the difference in lane width at different locations, when W f,i With W c,i When they tend to be the same, the correction amount tends to 0.

[0072] Furthermore, in order to improve the degree of a single correction and accelerate the correction iteration process, W can be... f,i and W c,i Set the lane width at the beginning of the lane and the lane width at the end of the lane in the image, respectively.

[0073] For example, see references to Figure 2 The diagram shows a method for correcting lane lines using a correction amount. Lane lines 210a and 210b represent two lane lines before correction, and it can be seen that they are not parallel. Lane lines 220a and 220b represent two lane lines after correction, and it is clear that lane lines 220a and 220b are closer to being parallel.

[0074] By determining the camera pitch angle correction based on the lane line information according to the equal width characteristics of the lanes, this method is not limited to straight road scenarios and can also be applied to curved road scenarios. It also avoids dependence on the vanishing point and improves the applicability of the calibration method.

[0075] S230. Determine the calibration value of the camera pitch angle based on the correction amount and the initial value of the camera pitch angle.

[0076] In some implementations, the sum of the correction amount and the initial value of the camera pitch angle is determined as the calibration value of the camera pitch angle. The initial value of the camera pitch angle can be obtained through offline calibration or through other mature calibration methods.

[0077] In some implementations, to improve calibration accuracy, the initial value of the camera pitch angle is limited. The initial value of the camera pitch angle is obtained in the following way:

[0078] Based on the lane line information, the initial value of the camera pitch angle is determined by an extended Kalman filter.

[0079] Specifically, the lane line information includes a point set of at least three lane lines, and determining the initial value of the camera pitch angle using an extended Kalman filter based on the lane line information includes:

[0080] The vanishing point is calculated based on the point sets of two different lane lines; the vanishing point is verified based on the point sets of other lane lines besides the two different lane lines; in response to successful verification, a reference value for the camera pitch angle is determined based on the vanishing point, wherein the vanishing point is the intersection point formed by parallel lane lines in the physical world on the image; the extended Kalman filter is initialized based on the reference value for the camera pitch angle; and the initial value of the camera pitch angle is determined based on the lane line information using the extended Kalman filter.

[0081] The calculation of the vanishing point based on the point set of two different lane lines includes:

[0082] The equations of the two different lane lines are determined based on the point sets of the two different lane lines respectively (for example, the lane line equations can be obtained by fitting the point sets of the lane lines using the least squares method); the intersection point of the two different lane lines is calculated based on the equations of the two different lane lines, and the intersection point is determined as the vanishing point.

[0083] The step of verifying the vanishing point based on the point set of other lane lines besides the two different lane lines includes:

[0084] Based on the point set of the other lane lines, determine the midpoint of the other lane lines; determine the angle between the line connecting the vanishing point and the midpoint and the other lane lines; if the angle is less than a preset value, the verification is successful. For example, refer to... Figure 3 The diagram shows the angle between the line connecting the vanishing point and the midpoint and the other lane lines. In this diagram, 310 represents the vanishing point and 320 represents the midpoint of a lane line that is not involved in the vanishing point calculation. If the angle θ is less than a preset value, it means that the error in solving the vanishing point is within the allowable range, and the verification is successful.

[0085] The step of determining the reference value for the camera pitch angle based on the vanishing point includes:

[0086] The reference value of the camera pitch angle is calculated using the relationship between the vanishing point and the camera pitch angle.

[0087] For example, the relationship between the vanishing point and the camera pitch angle is expressed by the following equations (2)-(4):

[0088]

[0089] Where K represents the camera intrinsic parameters, (u, v) represents the coordinates of the vanishing point, pitch represents the pitch angle, which is an unknown quantity to be solved, and the obtained pitch is the initial value. yaw represents the yaw angle, which is a known quantity. In summary, the pitch can be solved as the initial value based on the above relationship.

[0090] For example, initializing the extended Kalman filter based on the reference value of the camera pitch angle includes:

[0091] The camera pitch angle is initialized using the reference value, wherein the camera pitch angle and the rate of change of the camera pitch angle are state variables of the extended Kalman filter, and the initial value of the rate of change of the camera pitch angle is zero.

[0092] The step of determining the initial value of the camera pitch angle based on the lane line information using an extended Kalman filter includes:

[0093] Based on the state equation of the extended Kalman filter, the state variables are predicted to obtain the predicted pitch angle; the predicted value is updated based on the Kalman gain and the endpoints of the lane lines to obtain the initial value, wherein the lane line information includes the endpoints of the lane lines.

[0094] Specifically, the process of initializing the Kalman filter, predicting the state variables, and updating the predicted values ​​is expressed by the following equations (5)-(12):

[0095] X p =[θ ω θ ] T (5)

[0096]

[0097] Among them, X p The state variables θ and ω represent the state variables used in filter optimization. θ These represent the angular velocities of the camera pitch angle and the camera tilt angle, respectively. This represents the predicted value of the state variable at time t. This represents the updated value of the state variable at time t-1. When t=1, The reference value for the state variable is the value given during initialization. Relation (6) represents the state variable prediction process, where f p (X p ) is the state transition function, and relation (7) is the specific form of the state transition function, W p It is the noise variance that conforms to a normal distribution. Relationship (8) represents the covariance of the predicted state variables. E represents the predicted covariance. t W is the Jacobian matrix corresponding to the state transition function. t This is process noise.

[0098]

[0099] The state variables can be obtained after the filter is updated. and the corresponding covariance P t H t Let $h$ be the Jacobian matrix of the observation equation, $K$ be the Kalman gain, and $h$ be the value of $h$. p (X p ) represents the observation equation, which can be obtained from the following equations (11) and (12).

[0100]

[0101] n = (K -1 p1)×(K -1 p2)(12)

[0102] In relation (11), q represents the direction vector of the X-axis in the world coordinate system, R is the extrinsic parameter matrix of the camera, and q p The noise variance of the measurement model follows a normal distribution. In equation (12), p1 and p2 are the two endpoints of the lane line.

[0103] By determining the initial value of the camera pitch angle using an extended Kalman filter based on the lane line information, the final calibration accuracy and efficiency can be improved, and the calibration convergence process can be accelerated.

[0104] Based on the above embodiments, in order to further improve the calibration accuracy, multiple camera pitch angle calibration values ​​can be determined through multiple iterative calculations, and then the final calibration value can be determined based on these multiple calibration values.

[0105] For example, after the calibration value of the camera pitch angle is calculated through S110-S130, the initial value is updated using the calibration value, that is, the initial value is modified to the calibration value. Based on the new image acquired in real time by the vehicle-mounted camera, lane line information is recognized, and S120-S130 is repeated to obtain another calibration value. The initial value is updated using the latest obtained calibration value, and S120-S130 is repeated to obtain another calibration value. This process is repeated iteratively to obtain multiple calibration values. The final calibration value of the camera pitch angle is determined based on the calibration values ​​obtained from multiple iterations. The iteration termination condition can be that the number of iterations reaches a preset value, or the correction amount obtained after the iteration is less than a set value.

[0106] Furthermore, determining the final calibration value of the camera pitch angle based on the calibration values ​​obtained from multiple iterative calculations includes:

[0107] The calibration values ​​obtained from multiple iterations are smoothed to obtain the final calibration value of the camera pitch angle.

[0108] For example, the average of the calibration values ​​obtained from multiple iterations can be used as the final calibration value for the camera pitch angle.

[0109] Alternatively, other smoothing methods such as window smoothing or histogram smoothing can be used to smooth the calibration values ​​obtained from multiple iterations to obtain the final calibration value of the camera pitch angle.

[0110] Based on the above embodiments, refer to as follows Figure 4The flowchart shown illustrates an online calibration method for the pitch angle of a vehicle-mounted camera, including the following steps:

[0111] S1. Vehicle-mounted camera captures road images.

[0112] S2. Analyze the image and identify the lane line information within it.

[0113] Specifically, an image can be input into a neural network model, which then identifies lane line instances, i.e., lane line information. Lane line information consists of a set of points that make up the lane lines, and each point in the set carries coordinate information.

[0114] S3. Determine if the number of lane lines has reached the threshold. If it has not reached the threshold, proceed to S4; otherwise, proceed to S5.

[0115] S4. Exit calibration process.

[0116] S5, Lane line information preprocessing.

[0117] Lane line information preprocessing includes: filtering out lane lines that are too short (short lane lines disappear from the camera's image as the vehicle moves, hindering subsequent calibration), lane lines with distant starting points (lane lines with distant starting points usually have larger recognition errors, so filtering them is recommended to further improve calibration accuracy), and lane lines with close ending points (essentially also short lane lines, which are also detrimental to subsequent calibration, so they are filtered out). Furthermore, lane line points within a certain distance range are selected based on curvature to improve lane line quality, laying the foundation for subsequent calibration and ensuring the final calibration effect. Additionally, noise outside the lane lines can be removed to ultimately obtain a high-quality set of lane line points conducive to calibration.

[0118] S6. Calculate the vanishing point based on the preprocessed lane lines, and calculate the reference value of the camera pitch angle based on the vanishing point.

[0119] S7. Initialize the extended Kalman filter using the reference value.

[0120] S8. Update the extended Kalman filter based on the lane line information to obtain the initial value of the camera pitch angle.

[0121] S9. Based on the equal width characteristics of the lanes, calculate the correction amount of the pitch angle, and determine the calibration value by combining the initial value of the camera pitch angle.

[0122] S10. Update the initial value using the calibration value.

[0123] S11. Recognize the real-time images captured by the camera to obtain lane line information.

[0124] S12. Preprocess the lane line information.

[0125] S13. Based on the equal width characteristics of the lanes, calculate the correction amount of the pitch angle, and obtain the calibration value by combining the initial value, and return to execute S10.

[0126] S14. Store the calibration values ​​obtained from each iteration.

[0127] S15. Smooth multiple calibration values ​​to obtain the final calibration value.

[0128] Based on the above embodiments, Figure 5 This is a schematic diagram of the structure of an online tilt angle calibration device for a vehicle-mounted camera provided in an embodiment of this application. Figure 5 As shown, the device includes: a first determining module 510, used to determine lane line information based on the image captured by the vehicle-mounted camera in response to the current frame image; a second determining module 520, used to determine a correction amount for the camera pitch angle based on the lane line information and lane width equalization features; and a correction module 530, used to determine a calibration value for the camera pitch angle based on the correction amount and the initial value of the camera pitch angle.

[0129] Furthermore, it also includes: an update module, used to update the initial value using the calibration value, and in response to the vehicle-mounted camera acquiring the next frame image, to perform iterative calculation on the calibration value based on the next frame image; and a third determination module, used to determine the final calibration value of the camera pitch angle based on the calibration values ​​obtained from multiple iterative calculations.

[0130] Furthermore, the third determining module is specifically used to: smooth the calibration values ​​obtained from multiple iterative calculations to obtain the final calibration value of the camera pitch angle.

[0131] Furthermore, the second determining module 520 is specifically used for: determining the width difference at different locations within the same lane based on the lane line information; substituting the width difference into a preset formula to obtain the correction amount for the camera pitch angle, wherein the preset formula is constructed based on lane width equalization features, for example:

[0132]

[0133] Where i represents the number of iterations, ΔP i This represents the correction amount of the camera pitch angle in the i-th iteration calculation, asin() represents the arcsine function, and W f,i and W c,i The widths at different locations within the same lane in the image are represented by H, the height of the vehicle-mounted camera is represented by H, and the length of the shorter lane line among the two lane lines that make up the same lane is represented by L.

[0134] Furthermore, it also includes a fourth determining module, used to determine the initial value of the camera pitch angle based on the lane line information using an extended Kalman filter.

[0135] Furthermore, the fourth determining module includes: a vanishing point calculation unit, used to calculate the vanishing point based on the point sets of two different lane lines; a verification unit, used to verify the vanishing point based on the point sets of other lane lines besides the two different lane lines; a reference value determining unit, used to determine a reference value of the camera pitch angle based on the vanishing point in response to successful verification, wherein the vanishing point is the intersection point formed by parallel lane lines in the physical world on the image; an initialization unit, used to initialize the extended Kalman filter based on the reference value of the camera pitch angle; and an initial value determining unit, used to determine the initial value of the camera pitch angle based on the lane line information using the extended Kalman filter.

[0136] Furthermore, the vanishing point calculation unit is specifically used to: determine the equations of the two different lane lines based on the point sets of the two different lane lines respectively; calculate the intersection point of the two different lane lines according to the equations of the two different lane lines, and determine the intersection point as the vanishing point.

[0137] Furthermore, the verification unit is specifically used to: determine the midpoint of the other lane lines based on the point set of the other lane lines; determine the angle between the line connecting the vanishing point and the midpoint and the other lane lines; if the angle is less than a preset value, determine that the verification is successful.

[0138] Furthermore, the reference value determination unit is specifically used to: calculate the reference value of the camera pitch angle through the relationship between the vanishing point and the camera pitch angle.

[0139] Furthermore, the initialization unit is specifically used to: initialize the camera pitch angle using the reference value, wherein the camera pitch angle and the rate of change of the camera pitch angle are state variables of the extended Kalman filter, and the initial value of the rate of change of the camera pitch angle is zero; the initial value determination unit is specifically used to: predict the state variables according to the state equation of the extended Kalman filter to obtain a predicted value of the pitch angle; update the predicted value according to the Kalman gain and the endpoints of the lane lines to obtain the initial value, wherein the lane line information includes the endpoints of the lane lines.

[0140] The vehicle-mounted camera pitch angle online calibration device provided in this embodiment can execute the steps in the vehicle-mounted camera pitch angle online calibration method provided in this embodiment, and has the execution steps and beneficial effects, which will not be repeated here.

[0141] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. See below for details. Figure 6 It shows a schematic diagram of a structure suitable for implementing the electronic device 500 in the embodiments of this disclosure. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0142] like Figure 6 As shown, electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes to implement the methods of the embodiments described herein, based on a program stored in read-only memory (ROM) or a program loaded from storage device 508 into random access memory (RAM). RAM 503 also stores various programs and data required for the operation of electronic device 500. Processing device 501, ROM 502, and RAM 503 are interconnected via bus 504. I / O interface 505 is also connected to bus 504. Input device 506, output device 507, storage device 508, and communication device 509 are all connected to I / O interface 505.

[0143] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts, thereby implementing the online calibration method for the pitch angle of a vehicle-mounted camera as described above. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, it performs the functions defined in the methods of embodiments of this disclosure.

[0144] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0145] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the method steps of this application.

[0146] Optionally, when one or more of the above-described procedures are executed by the electronic device, the electronic device may also perform other steps described in the above embodiments.

[0147] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0148] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0149] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. The above descriptions are only preferred embodiments of this application. It should be noted that due to the limitations of written expression, while there are objectively infinite specific structures, those skilled in the art can make several improvements, modifications, or changes without departing from the principles of this application, and can also combine the above technical features in an appropriate manner. These improvements, modifications, changes, or combinations, or the direct application of the inventive concept and technical solution to other situations without modification, should all be considered within the scope of protection of this application.

Claims

1. A method for online calibration of the pitch angle of a vehicle-mounted camera, characterized in that, include: In response to the vehicle-mounted camera capturing the current frame image, lane line information is determined based on the image; The correction amount for the camera pitch angle is determined based on the lane line information and lane width characteristics, including: The width difference at different locations within the same lane is determined based on the lane line information; the width difference is then substituted into a preset formula to obtain the correction amount for the camera pitch angle, wherein the preset formula is constructed based on lane width equalization features; the preset formula is: Where i represents the number of iterations. This represents the correction amount of the camera pitch angle in the i-th iteration calculation, and asin() represents the arcsine function. and These represent the widths at different locations within the same lane in the image, H represents the height of the vehicle-mounted camera, and L represents the length of the shorter lane line among the two lane lines that make up the same lane. The calibration value of the camera pitch angle is determined based on the correction amount and the initial value of the camera pitch angle.

2. The online calibration method for the pitch angle of a vehicle-mounted camera according to claim 1, characterized in that, After determining the calibration value of the camera pitch angle based on the correction amount and the initial value of the camera pitch angle, the method further includes: The initial value is updated using the calibration value, and in response to the vehicle-mounted camera acquiring the next frame image, the calibration value is iteratively calculated based on the next frame image; The final calibration value of the camera pitch angle is determined based on the calibration values ​​obtained from multiple iterative calculations.

3. The online calibration method for the pitch angle of a vehicle-mounted camera according to claim 2, characterized in that, The determination of the final calibration value of the camera pitch angle based on the calibration values ​​obtained from multiple iterative calculations includes: The calibration values ​​obtained from multiple iterations are smoothed to obtain the final calibration value of the camera pitch angle.

4. The online calibration method for the pitch angle of a vehicle-mounted camera according to claim 1, characterized in that, The method further includes: Based on the lane line information, the initial value of the camera pitch angle is determined by an extended Kalman filter.

5. The online calibration method for the pitch angle of a vehicle-mounted camera according to claim 4, characterized in that, The lane line information includes a point set of at least three lane lines. The step of determining the initial value of the camera pitch angle using an extended Kalman filter based on the lane line information includes: The vanishing point is calculated based on the point set of two different lane lines, wherein the vanishing point is the intersection point formed by parallel lane lines in the physical world on the image. The vanishing point is verified based on the point set of other lane lines besides the two different lane lines. In response to successful verification, a reference value for the camera pitch angle is determined based on the vanishing point; The extended Kalman filter is initialized based on the reference value of the camera pitch angle; The initial value of the camera pitch angle is determined based on the lane line information using an extended Kalman filter.

6. The online calibration method for the pitch angle of a vehicle-mounted camera according to claim 5, characterized in that, The calculation of the vanishing point based on the point set of two different lane lines includes: Determine the equations of the two different lane lines based on the point sets of the two different lane lines respectively; The intersection point of the two different lane lines is calculated based on their equations, and the intersection point is determined as the vanishing point.

7. The online calibration method for the pitch angle of a vehicle-mounted camera according to claim 5, characterized in that, The step of verifying the vanishing point based on the point set of other lane lines besides the two different lane lines includes: Determine the midpoint of the other lane lines based on the point set of the other lane lines; Determine the angle between the line connecting the vanishing point and the midpoint and the other lane lines; If the included angle is less than the preset value, the verification is considered successful. The step of determining the reference value for the camera pitch angle based on the vanishing point includes: The reference value of the camera pitch angle is calculated using the relationship between the vanishing point and the camera pitch angle.

8. The online calibration method for the pitch angle of a vehicle-mounted camera according to claim 5, characterized in that, The initialization of the extended Kalman filter based on the reference value of the camera pitch angle includes: The camera pitch angle is initialized using the reference value, wherein the camera pitch angle and the rate of change of the camera pitch angle are state variables of the extended Kalman filter, and the initial value of the rate of change of the camera pitch angle is zero; The step of determining the initial value of the camera pitch angle based on the lane line information using an extended Kalman filter includes: Based on the state equation of the extended Kalman filter, the state variables are predicted to obtain the predicted pitch angle. The predicted value is updated based on the Kalman gain and the endpoints of the lane lines to obtain the initial value, wherein the lane line information includes the endpoints of the lane lines.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the online calibration method for the pitch angle of the vehicle-mounted camera as described in any one of claims 1-7.

Images