Vehicle front-view camera yaw angle calibration method, device, equipment and medium

Abstract

The invention relates to the field of camera calibration, in particular to a calibration method, device and equipment for a yaw angle of a vehicle foresight camera and a medium. The method comprises the following steps: acquiring a driving direction information sequence of a vehicle in a straight driving state; according to the driving direction information sequence, obtaining a reference driving direction of the vehicle driving along a straight line; determining the actual driving direction of the vehicle at the target moment based on the driving direction information at the target moment in the driving direction information sequence; determining a compensation yaw angle according to the deviation between the actual driving direction and the reference driving direction; and on the basis of the compensation yaw angle, calibrating an estimated value of the external parameter yaw angle of the foresight camera at the target moment, and obtaining a calibration value of the external parameter yaw angle of the foresight camera.

Description

Technical Field

[0001] This invention relates to the field of camera calibration, and specifically to a method, apparatus, equipment, and medium for calibrating the yaw angle of an external parameter of a vehicle forward-looking camera. Background Technology

[0002] The forward-facing camera is a core perception sensor in autonomous driving and driver assistance systems, responsible for acquiring rich information about the road, vehicles, pedestrians, and traffic signs ahead. To accurately map the two-dimensional image information perceived in the camera's coordinate system to the vehicle's coordinate system and even the world coordinate system for critical decisions such as distance estimation and path planning, the relative position and attitude relationship between the camera and the vehicle must be determined. This relationship is known as the camera's extrinsic parameters. The yaw angle is a key rotational parameter among the extrinsic parameters; it defines the angle between the camera's optical axis and the vehicle's longitudinal axis (i.e., the direction of travel) on the horizontal plane. If the yaw angle is incorrect, the system will misjudge the lateral position of targets ahead, potentially leading to misjudgments by lane-keeping systems, adaptive cruise control, and other functions, thus affecting driving safety.

[0003] Currently, the calibration of extrinsic parameters for vehicle forward-looking cameras typically relies on the lane vanishing point method. Firstly, this method cannot adapt to the slow changes in extrinsic parameters caused by bumps, collisions, or maintenance during vehicle use. Secondly, the accuracy of this method is often limited by whether the vehicle is parallel to the lane lines. Since it is difficult for a vehicle to be perfectly parallel to the lane lines during actual driving, the yaw angle calibrated using the vanishing point method generally has errors, significantly reducing its reliability. Therefore, a method is needed that can accurately calibrate the yaw angle of extrinsic parameters under normal vehicle driving conditions. Summary of the Invention

[0004] The method for calibrating the extrinsic yaw angle of a vehicle's forward-looking camera provided in this disclosure can accurately calibrate the extrinsic yaw angle of the forward-looking camera.

[0005] According to a first aspect of this disclosure, a method for calibrating the yaw angle of a vehicle's forward-looking camera is provided, comprising: Obtain the sequence of driving direction information of the vehicle in a straight-line driving state; Based on the driving direction information sequence, the reference driving direction of the vehicle when driving in a straight line is obtained; Based on the driving direction information at the target time in the driving direction information sequence, the actual driving direction of the vehicle at the target time is determined; The compensation yaw angle is determined based on the deviation between the actual driving direction and the reference driving direction; Based on the compensated yaw angle, the estimated value of the extrinsic yaw angle of the forward-looking camera at the target time is calibrated, and the calibration value of the extrinsic yaw angle of the forward-looking camera is obtained.

[0006] According to a second aspect of this disclosure, a calibration device for the extrinsic yaw angle of a vehicle's forward-looking camera is provided, comprising: an acquisition module for acquiring a sequence of driving direction information of the vehicle in a straight-line driving state; a reference direction determination module for obtaining a reference driving direction of the vehicle along a straight line based on the sequence of driving direction information; an actual direction determination module for determining the actual driving direction of the vehicle at a target time based on the driving direction information at a target time in the sequence of driving direction information; a compensation determination module for determining a compensation yaw angle based on the deviation between the actual driving direction and the reference driving direction; and a calibration module for calibrating the estimated value of the extrinsic yaw angle of the forward-looking camera at the target time based on the compensation yaw angle, thereby obtaining a calibration value of the extrinsic yaw angle of the forward-looking camera.

[0007] According to a third aspect of this disclosure, an electronic device is provided, including a processor and a memory, the memory storing computer instructions that, when executed by the processor, implement the method described in either the first or second aspect.

[0008] According to a fourth aspect of this disclosure, a storage medium is provided that stores computer instructions thereon, which, when executed by a processor, implement the steps of the method as described in any one of the first or second aspects of this disclosure.

[0009] One beneficial effect of this disclosure is that it provides a method for calibrating the yaw angle of a vehicle's forward-looking camera. By using the vehicle's own driving direction information, a reference driving direction representing the straight-line driving direction, i.e., the direction of the road and lane lines, is obtained. Then, by comparing the deviation between the vehicle's real-time driving direction and this reference direction, a compensation angle for calibrating the yaw angle of the forward-looking camera's extrinsic parameters is calculated. This method effectively overcomes the limitations of traditional calibration methods such as the vanishing point method that rely on lane line detection, significantly reducing the calibration requirement of complete parallelism with the lane lines and improving calibration robustness. Simultaneously, this method can compensate for extrinsic parameter deviations caused by changes in vehicle attitude in real time, achieving higher precision and stronger adaptability in calibration, thereby ensuring the long-term accuracy of perception and positioning in automatic and assisted driving systems.

[0010] Other features and advantages of the embodiments of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0011] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of the present disclosure and, together with their description, serve to explain the principles of the embodiments of the present disclosure.

[0012] Figure 1 A flowchart illustrating a method for calibrating the yaw angle of a vehicle's forward-looking camera according to an embodiment of this disclosure is shown. Figure 2A schematic diagram of a vehicle traveling in a lane according to an embodiment of the present disclosure is shown; Figure 3 A schematic diagram of a calibration device for the yaw angle of a vehicle forward-looking camera according to an embodiment of the present disclosure is shown; Figure 4 A schematic diagram of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0013] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the invention.

[0014] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0015] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0016] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0017] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0018] This application provides a method for calibrating the yaw angle of a vehicle's forward-looking camera. For example... Figure 1 As shown, the method includes steps S11-S13.

[0019] Step S11: Obtain the sequence of driving direction information of the vehicle in a straight-line driving state.

[0020] In one example, the vehicle's yaw angle and other data can be received in real time from the onboard odometer or inertial measurement unit (IMU) via the vehicle's CAN bus, and used as the vehicle's driving direction information. Simultaneously, pre-set criteria can be used to determine whether the vehicle is traveling straight, acquiring only the driving direction information when the vehicle is traveling straight. This information is then arranged and filtered according to time to obtain a sequence of driving direction information that meets the requirements for straight-line driving. The number of driving direction information entries in the sequence can be set according to actual needs.

[0021] Step S12: Based on the driving direction information sequence, obtain the reference driving direction for the vehicle to travel in a straight line.

[0022] In one example of this embodiment, the reference driving direction of the vehicle traveling in a straight line is obtained based on the driving direction information sequence, including: determining the average driving direction of the vehicle in a straight-line driving state based on the driving direction information sequence, and using the average driving direction as the reference driving direction of the vehicle traveling in a straight line.

[0023] In one example, after acquiring a sequence of driving direction information that meets the conditions for straight-line driving, such as a sequence of vehicle yaw angle data, the vehicle yaw angle data within this sequence are averaged. The resulting mean is taken as the average driving direction of the vehicle during this straight-line driving process, which is equivalent to the direction of road extension or lane extension. This direction is used as the reference direction for the vehicle's straight-line driving. This reference direction effectively filters out minor directional fluctuations during straight-line driving, thus providing a stable reference for subsequent calculations of the deviation between the vehicle's real-time direction and this reference.

[0024] Step S13: Based on the driving direction information at the target time in the driving direction information sequence, determine the actual driving direction of the vehicle at the target time.

[0025] In one example of this embodiment, the actual driving direction of the vehicle at the target time is directly represented by the driving direction information sequence corresponding to the target time. The vehicle yaw angle data, measured and output in real time by the onboard odometer or IMU at that target time, can be directly read from the odometer data. This yaw angle value represents the vehicle's actual orientation relative to the global coordinate system or initial direction at that target time, which is the vehicle's actual driving direction at that time.

[0026] Step S14: Determine the compensation yaw angle based on the deviation between the actual driving direction and the reference driving direction.

[0027] In one example, the compensated yaw angle is determined based on the deviation between the actual driving direction and the reference driving direction. This is achieved by calculating the difference between the actual vehicle yaw angle at the target time and the average vehicle yaw angle in the reference driving direction. This difference is the compensated yaw angle, which reflects the angle of deviation of the current vehicle's instantaneous driving direction from the road reference direction, that is, the angle between the vehicle's longitudinal axis and the direction of lane line extension.

[0028] Step S15: Based on the estimated value of the extrinsic yaw angle of the forward-looking camera at the target time of the compensation yaw angle calibration, obtain the calibration value of the extrinsic yaw angle of the forward-looking camera.

[0029] In one example, the estimated value of the forward-looking camera's extrinsic yaw angle at the target time, based on the compensated yaw angle calibration, can be obtained by subtracting the compensated yaw angle calculated in the preceding steps from the preliminary estimated value of the forward-looking camera's extrinsic yaw angle obtained through the vanishing point method of the lane lines in the image or other methods. This subtraction operation directly cancels out the deviation caused by the angle between the actual driving direction of the vehicle and the direction of the lane lines, ultimately outputting a more accurate calibrated value of the forward-looking camera's extrinsic yaw angle after compensation calibration.

[0030] This example presents a method for calibrating the yaw angle of a vehicle's forward-looking camera. By using the vehicle's own driving direction information, a reference driving direction representing the straight-line driving direction (i.e., the direction of the road and lane lines) is obtained. Then, by comparing the deviation between the vehicle's real-time driving direction and this reference direction, a compensation angle for calibrating the yaw angle of the forward-looking camera's extrinsic parameters is calculated. This method effectively overcomes the limitations of traditional calibration methods such as the vanishing point method, which rely on lane line detection. It significantly reduces the calibration requirement of being completely parallel to the lane lines, improving calibration robustness. Simultaneously, this method can compensate for extrinsic parameter deviations caused by changes in vehicle attitude in real time, achieving higher accuracy and stronger adaptability in calibration, thereby ensuring the long-term accuracy of perception and positioning in automated and assisted driving systems.

[0031] In one example of this embodiment, before calibrating the estimated value of the extrinsic yaw angle of the forward-looking camera at the target time based on the compensated yaw angle, the method further includes: mapping lane line information in the image captured by the vehicle-mounted camera to the vehicle coordinate system, and calculating the distances from the vehicle to the left and right lane lines respectively; determining the amount of change in the distance between the vehicle and the left and right lane lines within a first time window; determining that the vehicle is traveling in a straight line along the lane lines when the amount of change in the distance between the vehicle and the left and right lane lines is less than a predetermined change threshold; and obtaining the estimated value of the extrinsic yaw angle of the forward-looking camera at the target time based on the vanishing point of the lane lines when the vehicle is traveling in a straight line along the lane lines.

[0032] In one example, before calibrating the estimated yaw angle of the forward-looking camera's extrinsic parameters at the target time based on the compensated yaw angle, it is necessary to determine the straight-ahead scenario along the lane lines and obtain an initial estimate. For example, using images captured by perception modules such as forward-looking or other onboard cameras, lane line information is obtained. Combined with the camera's intrinsic parameters and the initial extrinsic parameters, the lane lines in the image are mapped to the vehicle coordinate system through inverse projection transformation. Based on this, the lateral distances from the vehicle's rear axle center to the left and right lane lines are calculated respectively. Subsequently, within a data window of a preset time length, such as 10 consecutive valid frames, the variance of the distances from the vehicle to the left and right lane lines is calculated, which is equivalent to the change in the distance between the vehicle and the left and right lane lines. When the variances of the distances on both sides are less than a preset threshold, such as 0.05 meters, it is determined that the vehicle is traveling in a straight line along the lane line direction during this period. Only at the target time that this straight-ahead condition is met, the vanishing point is calculated based on the intersection of the fitted straight lines of the left and right lane lines in the image, and the vanishing point is projected onto the vehicle coordinate system. Then, by calculating the angle between specific vectors, the initial estimate of the yaw angle of the forward-looking camera's extrinsic parameters at that target time is obtained. In this way, the estimated value of the camera yaw angle obtained by the vanishing point method is relatively accurate and has reference value, so that the estimated value can be calibrated later.

[0033] In another example, the change in distance between the vehicle and the left and right lane lines can also be determined by calculating the average rate of change. Within a preset time window, the distances from the vehicle to the left lane line and the right lane line are fitted separately to obtain the slope of the distance change over time on both sides. The absolute value of this slope represents the average rate of distance change. When the absolute values ​​of the slopes of the distance changes on both sides are less than a predetermined rate of change threshold, it can also be determined that the vehicle is traveling in a straight line along the lane lines.

[0034] In one example of this embodiment, obtaining an estimated value of the extrinsic yaw angle of the forward-looking camera at the target time based on the vanishing point of the lane line includes: fitting the lane line to the forward-looking camera image at the target time to obtain the vanishing point of the lane line in the image; projecting the vanishing point onto the vehicle coordinate system, and determining the target direction vector based on the coordinates of the vanishing point and the coordinates of the camera's optical center in the vehicle coordinate system; and using the angle between the target direction vector and the reference direction of the vehicle coordinate system as an estimated value of the extrinsic yaw angle of the forward-looking camera at the target time.

[0035] In this embodiment, lane lines can be detected and fitted on the image captured by the forward-looking camera at the target time to obtain the equations of two straight lines representing the lane lines, and their intersection point in the image plane, i.e., the vanishing point, can be calculated. Then, using the camera's intrinsic parameter matrix and initial extrinsic parameter matrix, the vanishing point in the image coordinate system is projected onto the vehicle coordinate system through coordinate transformation to obtain its three-dimensional coordinates. Combined with the known position of the camera's optical center in the vehicle coordinate system, the target direction vector from the optical center to the vanishing point is calculated. Finally, the angle between this target direction vector and a preset reference direction vector in the vehicle coordinate system is calculated. This reference direction vector is defined as the direction passing through the camera's optical point and parallel to the vehicle's longitudinal axis. The calculated angle is used as the initial estimate of the forward-looking camera's extrinsic yaw angle at the target time.

[0036] In one example of this embodiment, obtaining the driving direction information sequence of a vehicle in a straight-line driving state includes: determining a second time window in which the vehicle is in a straight-line driving state based on the vehicle's motion information, wherein the vehicle's motion information includes driving speed, yaw rate and driving direction information; and filtering the driving direction information within the second time window to form a driving direction information sequence.

[0037] By acquiring the sequence of vehicle direction information during straight-line driving, segments representing true straight-line travel can be filtered from continuous vehicle motion data. Vehicle motion information primarily includes speed, yaw rate, and yaw angle provided by the vehicle bus. A set of dynamic threshold conditions for determining straight-line driving status can be initially set. For example, the vehicle might be required to have a certain speed to avoid interference from sensor noise during low-speed creep or when stationary. Simultaneously, the absolute value of the vehicle's yaw rate must consistently remain below an upper limit to ensure the vehicle is not performing significant turning or lane-changing maneuvers.

[0038] During real-time data processing, a dynamic sliding time window can be created to continuously monitor the incoming velocity and yaw rate data. Only when all data points within the window simultaneously meet the aforementioned velocity and yaw rate conditions is the time period considered a valid "second time window." The length of this window can be fixed or dynamic, starting from the moment the conditions are first met and continuing until the conditions are broken, thus ensuring that the entire time period within the window represents stable straight-line travel.

[0039] After determining the second time window, further filtering can be performed to improve the data quality of the driving direction information. For example, a statistical outlier detection algorithm can be applied to the yaw angle data within the window to remove abnormal jumps caused by road bumps or instantaneous sensor interference. Finally, the cleaned and filtered, temporally continuous vehicle yaw angle data are arranged in chronological order to form a driving direction information sequence. This lays a high-quality data foundation for calculating a benchmark driving direction that accurately reflects the road direction.

[0040] In one example of this embodiment, obtaining the driving direction information sequence of a vehicle in a straight-line driving state includes: determining a second time window in which the vehicle is in a straight-line driving state based on the vehicle's motion information, wherein the vehicle's motion information includes driving speed, yaw rate and driving direction information; and filtering the driving direction information within the second time window to form a driving direction information sequence.

[0041] To obtain the sequence of driving direction information of a vehicle in a straight-line driving state, it is necessary to determine a second time window that can truly reflect the vehicle's straight-line driving state based on the vehicle's motion information. The vehicle motion information mainly includes the driving speed, yaw rate, and vehicle yaw angle, which are obtained in real time from the vehicle bus, as well as the vehicle body yaw angle, which is the driving direction information itself.

[0042] In this embodiment, yaw rate is a physical quantity describing how quickly a vehicle's direction changes; it refers to the angular velocity of the vehicle rotating around an axis perpendicular to the ground. It has a direct correspondence with the steering wheel angle. When the driver turns the steering wheel, the steering system changes the front wheel angle, thereby generating a torque that causes the vehicle to rotate around its vertical axis. This torque leads to a change in the vehicle's yaw rate. The larger the steering wheel angle and the faster the input, the greater the yaw rate is typically generated. By monitoring whether the yaw rate is close to zero, it is possible to effectively determine whether the vehicle is in a straight-line driving state without steering intention. After determining a valid second time window, a data segment of straight-line vehicle travel has been selected. From this second time window, all travel direction information that meets the criteria is extracted, i.e., a series of continuous vehicle yaw angle data. This data can then undergo further smoothing and filtering, and finally arranged in chronological order, forming a high-quality sequence of travel direction information for subsequent calculation of the reference travel direction.

[0043] In another example of this embodiment, determining the second time window in which the vehicle is in a straight-line driving state based on the vehicle's motion information includes: determining the time period in which the driving speed is greater than a speed threshold, the absolute value of the yaw rate is less than the angular rate threshold, and the difference between the maximum and minimum values ​​of the driving direction information is less than the direction difference threshold.

[0044] In this example, to determine if the vehicle is traveling in a straight line, vehicle motion information for each frame can be acquired in real time and filtered to determine a second time window indicating that the vehicle is traveling in a straight line. For example, the vehicle motion information in a single frame within the window must satisfy the following conditions: the vehicle speed must be greater than a certain threshold to ensure that the vehicle is in valid motion rather than stationary or creeping. Its yaw rate must be less than a certain threshold to exclude scenarios where the vehicle is clearly turning. Furthermore, if the difference between the maximum and minimum yaw angles of all vehicles within the window exceeds a threshold, it indicates a change in the vehicle's direction of travel. To ensure the accuracy of the direction of travel information in the second time window—that is, to ensure the reference direction of travel correctly reflects the vehicle's straight-line direction—the latest frame of data can be deleted if the above conditions are not met, or the second time window can be redefined.

[0045] In one example of this embodiment, the sequence of driving direction information must satisfy the following conditions: the distance between the vehicle positions corresponding to any two adjacent driving direction information in the sequence satisfies the first constraint condition; the distance between the vehicle positions corresponding to the driving direction information at the beginning and end of the sequence satisfies the second constraint condition; and the number of driving direction information in the sequence satisfies the third constraint condition.

[0046] In one example of this embodiment, the construction of the driving direction information sequence needs to meet several constraints to ensure that it can accurately reflect a stable and effective straight-line driving process. First, the straight-line distance between the vehicle position points corresponding to any two adjacent driving direction information in the sequence must meet the first constraint, that is, the distance must be controlled between a preset minimum and maximum value. For example, the minimum distance can be set to 0.1 meters to avoid data redundancy and oversampling; the maximum distance can be set to 20 meters to ensure the timeliness of the data and avoid missing subtle changes in the vehicle's driving state due to excessively long sampling intervals, thereby ensuring that the sequence can accurately represent the vehicle's driving trajectory.

[0047] The total distance between the vehicle position points corresponding to the driving direction information at the beginning and end of the entire sequence must meet the second constraint condition, that is, the total distance must not exceed a set upper limit value or fall below a set lower limit value. The purpose of setting this condition is to limit the distance span used to calculate the reference direction, ensuring that the extracted reference direction can represent a local, approximate straight road segment.

[0048] The number of driving direction information points contained in the sequence, i.e., the total number of data points, must satisfy the third constraint. This number must be between a set minimum and maximum value. For example, the sequence must contain at least 10 frames to ensure sufficient data for reliable statistical analysis and to calculate a representative average driving direction. Furthermore, the total number of frames must not exceed 300 frames to prevent an excessively long data window from including too much outdated historical information and to maintain the sequence's ability to represent the vehicle's recent driving state. These conditions work together to use the selected driving direction information sequence as the data basis for calculating a reliable reference driving direction.

[0049] In a complete embodiment, a continuous time window conforming to straight-line driving standards is dynamically identified by continuously analyzing the vehicle's real-time motion data. Within this time window, a sequence of driving direction information that reliably reflects the overall driving trend of the vehicle is collected and constructed. Through statistical analysis of this sequence, an average direction is calculated and established as the reference driving direction followed by the vehicle during this period, such as... Figure 2 As shown, this direction can represent the direction of the road and lane lines. At the same time, since it is difficult for a vehicle to be absolutely parallel to the lane lines when it is driving on the road, the driver usually makes small left and right steering wheel controls to keep the vehicle driving in a straight line. At a certain moment, the actual driving direction of the vehicle will have an angle of a few degrees or even a fraction of a degree with the reference driving direction. This angle is the compensation yaw angle.

[0050] Simultaneously, a forward-looking camera is used to perceive the road environment. It analyzes lane lines in the image and combines this with changes in the relative position of the vehicle and lane lines to cross-verify whether the vehicle is traveling in a straight line along the lane lines. Only when the vehicle is also traveling in a straight line along the lane lines is the image processed for that moment. Based on the geometric features of the lane lines in the image, an initial estimate of the camera's extrinsic yaw angle is calculated using the vanishing point principle. Then, the actual driving direction directly measured by the vehicle's motion sensor at the same target moment is compared with the previously established baseline driving direction; the difference between the two is quantified as a compensated yaw angle. This angle essentially reflects the angle between the vehicle body and the lane line direction. Finally, by subtracting this compensated yaw angle from the image-based initial estimate, the error caused by the vehicle body not being parallel to the lane lines is accurately eliminated, thus outputting a calibrated and more accurate extrinsic yaw angle calibration value.

[0051] In another example of this embodiment, the calibration values ​​of the camera's extrinsic yaw angle at multiple target times can be obtained, and mean filtering can be performed to obtain more accurate calibration values ​​of the camera's extrinsic yaw angle.

[0052] This application embodiment also provides a calibration device 100 for the yaw angle of a vehicle's forward-looking camera, such as... Figure 3 As shown, it includes: an acquisition module 101, used to acquire a sequence of driving direction information of the vehicle in a straight-line driving state; a reference direction determination module 102, used to obtain a reference driving direction of the vehicle along a straight line based on the driving direction information sequence; an actual direction determination module 103, used to determine the actual driving direction of the vehicle at the target time based on the driving direction information at the target time in the driving direction information sequence; a compensation determination module 104, used to determine a compensation yaw angle based on the deviation between the actual driving direction and the reference driving direction; and a calibration module 105, used to calibrate the estimated value of the extrinsic yaw angle of the forward-looking camera at the target time based on the compensation yaw angle, and obtain the calibration value of the extrinsic yaw angle of the forward-looking camera.

[0053] Optionally, the reference direction determination module is specifically used to: determine the average driving direction of the vehicle in a straight-line driving state based on the driving direction information sequence, and use the average driving direction as the reference driving direction of the vehicle in a straight line.

[0054] Optionally, the device further includes: an estimation module, used to map lane line information in images captured by the vehicle-mounted camera to the vehicle coordinate system, and calculate the distances from the vehicle to the left and right lane lines respectively; determine the change in the distance between the vehicle and the left and right lane lines within a first time window; determine that the vehicle is traveling in a straight line along the lane lines when the change in the distance between the vehicle and the left and right lane lines is less than a predetermined change threshold; and obtain an estimated value of the extrinsic yaw angle of the forward-looking camera at the target time based on the vanishing point of the lane lines when the vehicle is traveling in a straight line along the lane lines.

[0055] Optionally, based on the vanishing point of the lane line, the estimated value of the extrinsic yaw angle of the forward-looking camera at the target time is obtained, including: fitting the lane line based on the forward-looking camera image at the target time to obtain the vanishing point of the lane line in the image; projecting the vanishing point onto the vehicle coordinate system, and determining the target direction vector according to the coordinates of the vanishing point and the coordinates of the camera optical center in the vehicle coordinate system; and using the angle between the target direction vector and the reference direction of the vehicle coordinate system as the estimated value of the extrinsic yaw angle of the forward-looking camera at the target time.

[0056] Optionally, the acquisition module is specifically used to determine a second time window in which the vehicle is in a straight-line driving state based on the vehicle's motion information, wherein the vehicle's motion information includes driving speed, yaw rate and driving direction information; and to filter the driving direction information within the second time window to form a driving direction information sequence.

[0057] Optionally, determining a second time window based on the vehicle's motion information, where the vehicle is in a straight-line driving state, includes: defining the time period in which the driving speed is greater than a speed threshold, the absolute value of the yaw rate is less than an angular rate threshold, and the difference between the maximum and minimum values ​​of the driving direction information is less than a direction difference threshold.

[0058] Optionally, the sequence of driving direction information must meet the following conditions: the distance between the vehicle positions corresponding to any two adjacent driving direction information in the sequence satisfies the first constraint condition; the distance between the vehicle positions corresponding to the driving direction information at the beginning and end of the sequence satisfies the second constraint condition; and the number of driving direction information in the sequence satisfies the third constraint condition.

[0059] This application also provides an electronic device 200, such as... Figure 4 As shown, it includes a processor 201 and a memory 202. The memory 202 stores computer instructions. When the computer instructions are executed by the processor 201, they implement any one of the above embodiments of the calibration method for the yaw angle of the vehicle's forward-looking camera and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0060] This application provides a storage medium storing a program or instructions that, when executed by a processor, implement the steps of the vehicle forward-looking camera yaw angle calibration method of any of the foregoing embodiments, and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0061] The various embodiments in this disclosure are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the control method and storage medium embodiments are basically similar to the device embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0062] The foregoing has described specific embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0063] Embodiments of this disclosure may be systems, methods, and / or computer program products. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the embodiments of this disclosure.

[0064] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0065] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0066] Computer program instructions used to perform the operations of embodiments of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on a user's computer, partially on a user's computer, as a standalone software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of embodiments of this disclosure.

[0067] Various aspects of embodiments of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0068] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0069] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0070] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation in a combination of software and hardware are equivalent.

[0071] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for calibrating the yaw angle of a vehicle's forward-looking camera, characterized in that, The method further includes: Obtain the sequence of driving direction information of the vehicle in a straight-line driving state; Based on the driving direction information sequence, the reference driving direction of the vehicle when driving in a straight line is obtained; Based on the driving direction information at the target time in the driving direction information sequence, the actual driving direction of the vehicle at the target time is determined; The compensation yaw angle is determined based on the deviation between the actual driving direction and the reference driving direction; Based on the compensated yaw angle, the estimated value of the extrinsic yaw angle of the forward-looking camera at the target time is calibrated, and the calibration value of the extrinsic yaw angle of the forward-looking camera is obtained.

2. The method according to claim 1, characterized in that, The step of obtaining the reference driving direction of the vehicle traveling in a straight line based on the driving direction information sequence includes: Based on the driving direction information sequence, the average driving direction of the vehicle in the straight-line driving state is determined, and the average driving direction is used as the reference driving direction of the vehicle in the straight line.

3. The method according to claim 1, characterized in that, Before calibrating the estimated value of the extrinsic yaw angle of the forward-looking camera at the target time based on the compensated yaw angle, the method further includes: The lane line information in the images captured by the vehicle-mounted camera is mapped to the vehicle coordinate system, and the distances from the vehicle to the left and right lane lines are calculated respectively. Determine the change in the distance between the vehicle and the left and right lane lines within the first time window; If the change in distance between the vehicle and the left and right lane lines is less than a predetermined change threshold, it is determined that the vehicle is traveling in a straight line along the lane lines. When the vehicle is traveling in a straight line along the lane line, the estimated value of the extrinsic yaw angle of the forward-looking camera at the target time is obtained based on the vanishing point of the lane line.

4. The method according to claim 3, characterized in that, The step of obtaining an estimated value of the yaw angle of the forward-looking camera at the target time based on the vanishing point of the lane line includes: Based on the forward-looking camera image at the target time, the lane lines are fitted to obtain the vanishing points of the lane lines in the image; The vanishing point is projected onto the vehicle coordinate system, and the target direction vector is determined based on the coordinates of the vanishing point and the coordinates of the camera optical center in the vehicle coordinate system. The angle between the target direction vector and the reference direction of the vehicle coordinate system is used as the estimated value of the yaw angle of the extrinsic parameter of the forward-looking camera at the target time.

5. The method according to claim 1, characterized in that, The acquisition of the vehicle's driving direction information sequence in a straight-line driving state includes: The second time window for determining whether a vehicle is in a straight-line driving state is determined based on the vehicle's motion information, wherein the vehicle's motion information includes driving speed, yaw rate and driving direction information; The driving direction information within the second time window is filtered to form the driving direction information sequence.

6. The method according to claim 5, characterized in that, The second time window for determining whether a vehicle is traveling in a straight line based on its motion information includes: The time period in which the driving speed is greater than the speed threshold, the absolute value of the yaw rate is less than the angular rate threshold, and the difference between the maximum and minimum values ​​of the driving direction information is less than the direction difference threshold is determined as the second time window.

7. The method according to claim 5, characterized in that, The driving direction information sequence must meet the following conditions: The distance between the vehicle positions corresponding to any two adjacent driving direction information in the sequence satisfies the first constraint condition. The distance between the vehicle positions corresponding to the driving direction information at the head and tail of the sequence satisfies the second constraint condition. The number of driving direction information in the sequence satisfies the third constraint condition.

8. A calibration device for the yaw angle of a vehicle forward-looking camera, characterized in that, include: The acquisition module is used to acquire the sequence of driving direction information of the vehicle in a straight-line driving state; The reference direction determination module is used to obtain the reference driving direction of the vehicle traveling in a straight line based on the driving direction information sequence; The actual direction determination module is used to determine the actual driving direction of the vehicle at the target time based on the driving direction information at the target time in the driving direction information sequence; The compensation determination module is used to determine the compensation yaw angle based on the deviation between the actual driving direction and the reference driving direction; The calibration module is used to calibrate the estimated value of the extrinsic yaw angle of the forward-looking camera at the target time based on the compensated yaw angle, and obtain the calibration value of the extrinsic yaw angle of the forward-looking camera.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing computer instructions that, when executed by the processor, implement the method according to any one of claims 1-7.

10. A storage medium, characterized in that, It includes the step of storing computer instructions thereon, which, when executed by a processor, implement the method described in any one of claims 1-7.

Images