Vehicle-mounted camera extrinsic parameter calibration method, device, medium, equipment and driving system

Abstract

The present application discloses a method, apparatus, medium, equipment and driving system for extrinsic parameter calibration of an on-board camera, which belongs to the field of autonomous driving or assisted driving technology. The method mainly includes: using a forward-facing on-board camera to obtain an environmental image of the vehicle's environment, and sensing the environmental image to obtain two or more lane lines on both sides of the vehicle; obtaining the point on the environmental image with the minimum distance to the two or more lane lines as a forward reference point, and obtaining the pixel position of the forward reference point; calculating the pixel offset of the distortion center point relative to the forward reference point based on the pixel position of the forward reference point and the pixel position of the distortion center point on the environmental image; and calculating the angle between the lens coordinate system of the forward-facing on-board camera and the body coordinate system of the vehicle based on the pixel offset. This method can simplify the extrinsic parameter calibration process, improve calibration efficiency, and has strong practicality.

Description

Technical Field

[0001] The present application relates to the field of assisted driving or autonomous driving technology, and in particular to a calibration of a vehicle-mounted camera. Background Art

[0002] When using on-board cameras for environmental perception for assisted or autonomous driving, it is necessary to calibrate the camera's extrinsic parameters to confirm the position of the camera lens relative to the vehicle, thereby determining the position of the vehicle relative to the ground lane lines, and then obtaining the relative position relationship between other vehicles or obstacles captured by the camera and the vehicle.

[0003] Existing technology uses a checkerboard grid to perform offline calibration of the rotational relationship between the lens and the ground. However, this operation is complex and specialized, making it difficult for ordinary personnel to perform, and the calibration time is long. A simple and efficient camera extrinsic calibration solution is urgently needed. Summary of the Invention

[0004] In response to the problems existing in environmental perception of vehicle-mounted cameras, this application mainly provides a vehicle-mounted camera extrinsic parameter calibration method, device, medium, equipment and driving system.

[0005] In a first aspect, an embodiment of the present application provides a method for calibrating extrinsic parameters of a vehicle-mounted camera, comprising:

[0006] A forward-facing vehicle-mounted camera is used to obtain an environmental image of the vehicle's environment, and the environmental image is sensed to obtain two or more lane line straight lines on both sides of the vehicle; a point on the environmental image with the minimum distance to the two or more lane line straight lines is obtained as a forward reference point, and the pixel position of the forward reference point is obtained; based on the pixel position of the forward reference point and the pixel position of the distortion center point on the environmental image, a pixel offset of the distortion center point relative to the forward reference point is calculated; and, based on the pixel offset, an angle between a lens coordinate system of the forward-facing vehicle-mounted camera and a body coordinate system of the vehicle is calculated.

[0007] In a second aspect, an embodiment of the present application provides an extrinsic parameter calibration device for a vehicle-mounted camera, comprising:

[0008] The lane line acquisition module is used to use the forward-facing vehicle-mounted camera to acquire an environmental image of the vehicle's environment, and perceive the environmental image to acquire two or more lane line lines on both sides of the vehicle; the forward reference point pixel position acquisition module is used to obtain the point on the environmental image with the minimum distance to two or more lane line lines as the forward reference point, and obtain the pixel position of the forward reference point; the pixel offset acquisition module is used to calculate the pixel offset of the distortion center point relative to the forward reference point based on the pixel position of the forward reference point and the pixel position of the distortion center point on the environmental image; and the coordinate system angle acquisition module is used to calculate the angle between the lens coordinate system of the forward-facing vehicle-mounted camera and the body coordinate system of the vehicle based on the pixel offset.

[0009] In a third aspect, an embodiment of the present application provides a driving system, which includes the above-mentioned vehicle-mounted camera extrinsic parameter calibration device.

[0010] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing computer instructions, which are operated to execute the vehicle-mounted camera extrinsic parameter calibration method in the above-mentioned scheme.

[0011] In a fifth aspect, an embodiment of the present application provides a computer device comprising a processor and a memory, wherein the memory stores computer instructions, which are operated to execute the vehicle-mounted camera extrinsic parameter calibration method in the above-mentioned scheme.

[0012] In combination with the above embodiments, the technical solution of the present application can simplify the calibration process, improve the calibration efficiency, and has strong practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present application, a brief introduction will be given below to the drawings required for describing the embodiments. Obviously, the drawings in the following description exemplarily show some embodiments of the present application.

[0014] Figure 1 This is a flowchart of a specific implementation of a method for calibrating extrinsic parameters of a vehicle-mounted camera in the present application;

[0015] Figure 2 This is a schematic diagram of a vehicle coordinate system in a specific embodiment of a method for calibrating extrinsic parameters of a vehicle-mounted camera of the present application;

[0016] Figure 3 This is a schematic diagram of a lens coordinate system in a specific embodiment of a method for calibrating extrinsic parameters of a vehicle-mounted camera of the present application;

[0017] Figure 4 This is a schematic diagram of an environmental image of a vehicle in a specific embodiment of a method for calibrating extrinsic parameters of a vehicle-mounted camera of the present application;

[0018] Figure 5 This is a schematic diagram of the pixel positions of the forward reference point and the distortion center point in a specific embodiment of the extrinsic parameter calibration method for a vehicle-mounted camera of the present application;

[0019] Figure 6 This is a schematic diagram of a specific implementation of an extrinsic parameter calibration device for a vehicle-mounted camera of the present application;

[0020] The above drawings illustrate specific embodiments of the present disclosure, which will be described in more detail below. These drawings and textual descriptions are not intended to limit the scope of the present disclosure in any way, but rather to illustrate the concepts of the present disclosure to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0021] Some embodiments of the present application are described in detail below in conjunction with the accompanying drawings so that the advantages and features of the present application can be more easily understood by those skilled in the art, thereby making a clearer and more definite definition of the protection scope of the present application.

[0022] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, the elements defined by the phrase "comprising..." do not exclude the presence of other identical elements in the process, method, article, or device comprising the elements.

[0023] In vehicle-mounted application scenarios, the lens of a vehicle-mounted camera can be fixed to the vehicle body by a bracket, bonding or other connection methods. The optical axis of the lens of the vehicle-mounted camera may not be parallel to the direction of the front of the vehicle. For example, the vehicle-mounted camera and the front of the vehicle are deflected upward, downward, left or right. This application does not limit the position of the vehicle-mounted camera. The camera can be located inside or outside the vehicle, for example: in the vehicle cabin, at the front windshield, on the A-pillar of the vehicle, on the top of the vehicle, etc. When using a vehicle-mounted camera as an image or environmental perception device to realize the assisted driving or automatic driving function of the vehicle, it is necessary to calibrate the camera extrinsic parameters to confirm the position relationship of the camera lens relative to the vehicle, thereby determining the position relationship of the vehicle relative to the ground lane line, and then obtaining the relative position relationship between other vehicles or obstacles photographed by the camera and the vehicle. In some technologies, the relative position relationship between the lens and the ground is calibrated offline using a checkerboard. This operation is professional, complex, and takes a long time to calibrate. There is an urgent need for a simple and efficient camera extrinsic parameter calibration solution.

[0024] It can be understood that the camera extrinsic parameters referred to in the embodiments of the present application refer to the posture of the vehicle-mounted camera lens relative to the vehicle body, that is, the angle between the lens coordinate system and the vehicle body coordinate system.

[0025] The present application provides a method for calibrating the external parameters of a vehicle-mounted camera, which can automatically calibrate the relative position relationship between the vehicle-mounted camera lens and the vehicle during the normal movement of the vehicle without the need for additional operation by professionals. The calibration process is simple, can improve the calibration efficiency, and is highly practical.

[0026] The following specific embodiments are combined with the accompanying drawings to describe the technical solution of the present application in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0027] Figure 1 A specific implementation of a vehicle-mounted camera extrinsic parameter calibration method of the present application is shown.

[0028] exist Figure 1The specific implementation of the extrinsic parameter calibration method for a vehicle-mounted camera of the present application is shown, including process S101, using a forward-facing vehicle-mounted camera to obtain an environmental image of the vehicle's environment, and sensing the environmental image to obtain two or more lane line straight lines on both sides of the vehicle; process S102, obtaining the point on the environmental image with the minimum distance to two or more lane line straight lines as a forward reference point, and obtaining the pixel position of the forward reference point; process S103, calculating the pixel offset of the distortion center point relative to the forward reference point based on the pixel position of the forward reference point and the pixel position of the distortion center point on the environmental image; and process S104, calculating the angle between the lens coordinate system of the forward-facing vehicle-mounted camera and the body coordinate system of the vehicle based on the pixel offset.

[0029] This application uses the distortion center point and the forward reference point to automatically calibrate the external parameters of the vehicle-mounted camera's rotation angle relative to the vehicle, which can simplify the calibration process, improve calibration efficiency, and has strong practicality.

[0030] Process S101 represents the process of using a forward-facing vehicle-mounted camera to obtain an environmental image of the vehicle's environment, and sensing the environmental image to obtain two or more lane line straight lines on both sides of the vehicle, which can facilitate the direct acquisition of a forward reference point based on the above two or more lanes.

[0031] In an optional embodiment of the present application, the process of using a forward-facing vehicle-mounted camera to acquire an environmental image of the vehicle's environment and sensing and determining two or more lane lines on either side of the vehicle includes determining whether the two or more lane lines on either side of the vehicle are straight. Specifically, lane lines may not all be straight, so determining whether the lane lines are straight is necessary.

[0032] In an optional specific embodiment of the present application, the process of using a forward-facing vehicle-mounted camera to obtain an environmental image of the environment in which the vehicle is located, and sensing the environmental image to obtain two or more lane line lines on both sides of the vehicle includes:

[0033] A first environmental image of the vehicle's environment at time t1 is acquired using a forward-facing vehicle-mounted camera, and first road section lane lines located on both sides of the vehicle in the first environmental image are sensed; the current road section lane lines are fitted, and based on the fitting results, it is determined whether each first road section lane line is a straight line.

[0034] Specifically, when judging whether a lane line is a straight line, the straight line equation can be used to fit the lane line in the acquired environment image on a road section according to a straight line, and the corresponding lane line can be judged as a straight line based on the fitting residual, which can ensure the accuracy of the judgment.

[0035] In an optional specific embodiment of the present application, the process of using a forward-facing vehicle-mounted camera to obtain an environmental image of the environment in which the vehicle is located, and sensing the environmental image to obtain two or more lane line lines on both sides of the vehicle includes:

[0036] If the number of straight lane lines in the first road section is less than two, or the straight lane lines in the first road section are located on the same side of the vehicle, a second environmental image of the vehicle's environment at time t2 is acquired using the forward-facing onboard camera. This second environmental image is then sensed to acquire two or more straight lane lines located on either side of the vehicle. If the environmental image of a specific road section fails to meet the judgment requirements, the environmental image of the next road section is used to ensure the accuracy of the ultimately derived parameters.

[0037] In an optional specific embodiment of the present application, the above-mentioned process of using a forward-facing vehicle-mounted camera to obtain an environmental image of the environment in which the vehicle is located, and sensing the environmental image to obtain two or more lane line straight lines located on both sides of the vehicle includes: if the number of straight lane lines of the current road section is less than two, or the straight lane lines of the current road section are located on the same side of the vehicle, then an image with two or more lane line straight lines located on both sides of the vehicle is screened from the historical environmental image of the vehicle obtained by the forward-facing vehicle-mounted camera.

[0038] Specifically, only when there are straight lane lines on both sides of the vehicle can the calibration requirements of this application be met. Therefore, during the calibration process, it is necessary to screen the scenes that meet the requirements.

[0039] Process S102 represents the process of obtaining the point on the environment image with the minimum distance to two or more lane lines as the forward reference point, and obtaining the pixel position of the forward reference point. This can facilitate calculating the pixel offset of the distortion center point based on the pixel position of the forward reference point, and further calculating the angle between the lens coordinate system of the forward vehicle-mounted camera and the body coordinate system of the vehicle.

[0040] Specifically, this application is based on the vehicle body coordinate system, such as Figure 2 As shown, and the lens coordinate system of the vehicle-mounted camera is as follows Figure 3 As shown in the figure, the main optical axis coincides with the Z axis of the lens coordinate system. The rotation relationship between the vehicle camera lens and the vehicle is automatically calibrated. The essence of this is to calculate the angle between the X axis of the vehicle coordinate system and the Z axis of the lens coordinate system. When the vehicle is driving normally, the X axis of the vehicle coordinate system is parallel to the lane line direction, as shown in the figure. Figure 4The distance from the two or more lane lines and the smallest forward reference point are exactly points on a line parallel to the X-axis of the vehicle coordinate system. Therefore, it is only necessary to calculate the angle between the ray from the optical center of the lens of the forward-facing vehicle-mounted camera mounted on the vehicle corresponding to the actual point on the forward reference point and the Z-axis of the lens coordinate system.

[0041] In an optional specific embodiment of the present application, the above-mentioned process of obtaining the point with the minimum sum of distances to two or more lane line straight lines on the environmental image as the forward reference point, and obtaining the pixel position of the forward reference point includes obtaining the intersection of at least two or more lane line straight lines or the intersection of the extension lines of two or more lane line straight lines as the forward reference point.

[0042] Specifically, the forward reference point in this embodiment is the lane line and the vanishing point corresponding to the X-axis of the vehicle coordinate system. In the actual scene corresponding to the environmental image, the line connecting any point on the vehicle and the vanishing point must be parallel to the corresponding lane line, that is, parallel to the X-axis in the vehicle coordinate system. Therefore, it is only necessary to calculate the angle between the light ray captured by the optical center of the lens of the forward-facing vehicle-mounted camera installed on the vehicle at the actual point corresponding to the vanishing point and the Z-axis of the lens coordinate system.

[0043] Process S103 represents the process of calculating the pixel offset of the distortion center point relative to the forward reference point based on the pixel position of the forward reference point and the pixel position of the distortion center point on the environmental image. This process can facilitate automatic calibration of the rotational relationship between the vehicle-mounted camera lens and the vehicle based on the pixel offset between the distortion center point and the forward reference point.

[0044] Specifically, the Z axis of the lens coordinate system is parallel to the main optical axis of the camera lens. When calculating the angle between the light ray from the actual point corresponding to the forward reference point and the lens coordinate system, what is actually calculated is the angle between the light ray from the actual point corresponding to the forward reference point and the lens and the main optical axis. According to the camera imaging principle, the angle between the light ray from the optical center of the forward vehicle-mounted camera lens from the actual point corresponding to the forward reference point and the lens and the main optical axis is equal to the angle between the optical center ray passing through the pixel position where the forward reference point is located and the optical center ray passing through the pixel position where the distortion center is located, as shown in the figure below: Figure 5 shown.

[0045] In an optional specific embodiment of the present application, the process of calculating the pixel offset of the distortion center point relative to the forward reference point based on the pixel position of the forward reference point and the pixel position of the distortion center point on the environment image includes:

[0046] The horizontal pixel offset and vertical pixel offset of the distortion center point relative to the forward reference point are calculated respectively, so as to calculate the horizontal and vertical angles between the lens coordinate system and the vehicle coordinate system respectively based on the horizontal pixel offset and the vertical pixel offset.

[0047] Process S104 represents the process of calculating the angle between the lens coordinate system of the forward-facing vehicle-mounted camera and the body coordinate system of the vehicle based on the pixel offset. This process can ultimately automatically and efficiently calculate the angle between the lens coordinate system of the forward-facing vehicle-mounted camera and the body coordinate system of the vehicle based on the pixel offset of the distortion center point relative to the forward reference point, thereby completing the external parameter calibration of the forward-facing vehicle-mounted camera.

[0048] In an optional specific embodiment of the present application, the above-mentioned process of calculating the angle between the lens coordinate system of the forward-facing vehicle-mounted camera and the body coordinate system of the vehicle based on pixel offset includes calculating the horizontal angle between the lens coordinate system and the body coordinate system based on the horizontal pixel offset, and calculating the vertical angle between the lens coordinate system and the body coordinate system based on the vertical pixel offset.

[0049] Figure 6 A specific embodiment of the vehicle-mounted camera extrinsic calibration device of the present application is shown

[0050] exist Figure 6 The specific embodiment of the vehicle-mounted camera extrinsic parameter calibration device of the present application shown includes: a lane line acquisition module 601, which is used to use the forward vehicle-mounted camera to acquire an environmental image of the vehicle's environment, and perceive the environmental image to acquire two or more lane line straight lines on both sides of the vehicle; a forward reference point pixel position acquisition module 602, which is used to obtain the point on the environmental image with the minimum distance to two or more lane line straight lines as the forward reference point, and obtain the pixel position of the forward reference point; a pixel offset acquisition module 603, which is used to calculate the pixel offset of the distortion center point relative to the forward reference point based on the pixel position of the forward reference point and the pixel position of the distortion center point on the environmental image; and a coordinate system angle acquisition module 604, which is used to calculate the angle between the lens coordinate system of the forward vehicle-mounted camera and the body coordinate system of the vehicle based on the pixel offset.

[0051] This application uses the distortion center point and the forward reference point to automatically calibrate the external parameters of the vehicle-mounted camera's rotation angle relative to the vehicle, which can simplify the calibration process, improve calibration efficiency, and has strong practicality.

[0052] The lane line acquisition module 601 is used to obtain the environmental image of the vehicle's environment using the forward-facing vehicle-mounted camera, and to sense the environmental image to obtain two or more lane line straight lines on both sides of the vehicle. It can obtain two or more lane line straight lines on both sides of the vehicle so as to directly obtain the forward reference point based on the above two or more lanes.

[0053] In an optional embodiment of the present application, the lane line acquisition module 601, which is configured to use a forward-facing vehicle-mounted camera to acquire an environmental image of the vehicle's environment and sense and acquire two or more straight lane lines on either side of the vehicle, can be configured to determine whether the two or more lane lines on either side of the vehicle are straight. Specifically, lane lines are not necessarily straight, so it is necessary to determine whether the lane lines are straight.

[0054] In an optional specific embodiment of the present application, the lane line acquisition module 601 for acquiring an environmental image of the environment in which the vehicle is located using a forward-facing vehicle-mounted camera, and sensing the environmental image to acquire two or more lane line straight lines located on both sides of the vehicle, can be used to acquire the current environmental image of the environment in which the vehicle is currently located using a forward-facing vehicle-mounted camera, and sense the lane lines of the current road section located on both sides of the vehicle in the current environmental image; perform straight line fitting on the lane lines of the current road section using a straight line equation, calculate the fitting residual, and determine whether each lane line of the current road section is a straight line based on the fitting residual. If it is a straight line, the corresponding lane line of the current road section is determined as a lane line straight line. Specifically, when determining whether a lane line is a straight line, it is preferred to use a straight line equation to fit the lane line of the current road section as a straight line, and determine whether the corresponding lane line is a straight line based on the fitting residual, so as to ensure the accuracy of the judgment.

[0055] In an optional specific embodiment of the present application, the lane line acquisition module 601 for acquiring an environmental image of the environment in which the vehicle is located using a forward-facing vehicle-mounted camera, and sensing the environmental image to acquire two or more lane line straight lines located on both sides of the vehicle, can be used to acquire the next environmental image of the environment in which the vehicle is located at the next moment using the forward-facing vehicle-mounted camera if the number of straight lane lines in the current road section is less than two, or the straight lane lines in the current road section are located on the same side of the vehicle, and sense the next environmental image to acquire two or more lane line straight lines located on both sides of the vehicle.

[0056] In an optional specific embodiment of the present application, the lane line acquisition module 601 for acquiring an environmental image of the vehicle's environment using a forward-facing vehicle-mounted camera and sensing the environmental image to acquire two or more straight lane lines on both sides of the vehicle can be used to filter out an image with two or more straight lane lines on both sides of the vehicle from the historical environmental image of the vehicle acquired by the forward-facing vehicle-mounted camera if the number of straight lane lines of the current road section is less than two, or the straight lane lines of the current road section are located on the same side of the vehicle.

[0057] Specifically, only when there are straight lane lines on both sides of the vehicle can the calibration requirements of this application be met. Therefore, during the calibration process, it is necessary to screen the scenes that meet the requirements.

[0058] The forward reference point pixel position acquisition module 602 is used to obtain the point with the minimum distance from two or more lane lines on the environment image as the forward reference point, and obtain the pixel position of the forward reference point. This module can be used to calculate the pixel offset of the distortion center point based on the obtained pixel position of the forward reference point, and further calculate the angle between the lens coordinate system of the forward vehicle-mounted camera and the body coordinate system of the vehicle.

[0059] Specifically, this application is based on the vehicle body coordinate system, such as Figure 2 As shown, and the lens coordinate system of the vehicle-mounted camera is as follows Figure 3 As shown in the figure, the main optical axis coincides with the Z axis of the lens coordinate system. The rotation relationship between the vehicle camera lens and the vehicle is automatically calibrated. The essence of this is to calculate the angle between the X axis of the vehicle coordinate system and the Z axis of the lens coordinate system. When the vehicle is driving normally, the X axis of the vehicle coordinate system is parallel to the lane line direction, as shown in the figure. Figure 4 The distance from the two or more lane lines and the smallest forward reference point are exactly points on a line parallel to the X-axis of the vehicle coordinate system. Therefore, it is only necessary to calculate the angle between the ray from the optical center of the lens of the forward-facing vehicle-mounted camera mounted on the vehicle corresponding to the actual point on the forward reference point and the Z-axis of the lens coordinate system.

[0060] In an optional specific embodiment of the present application, the forward reference point pixel position acquisition module 602, which is used to obtain the point with the minimum distance from two or more lane line straight lines on the environmental image as the forward reference point and obtain the pixel position of the forward reference point, can be used to obtain the intersection of the above-mentioned two or more lane line straight lines as the forward reference point.

[0061] The pixel offset acquisition module 603 is used to calculate the pixel offset of the distortion center point relative to the forward reference point based on the pixel position of the forward reference point and the pixel position of the distortion center point on the environment image. It can facilitate the automatic calibration of the rotation relationship between the vehicle camera lens and the vehicle based on the obtained pixel offset between the distortion center point and the forward reference point.

[0062] Specifically, the Z axis of the lens coordinate system is parallel to the main optical axis of the camera lens. When calculating the angle between the light ray from the actual point corresponding to the forward reference point and the lens coordinate system, what is actually calculated is the angle between the light ray from the actual point corresponding to the forward reference point and the lens and the main optical axis. According to the camera imaging principle, the angle between the light ray from the optical center of the forward vehicle-mounted camera lens from the actual point corresponding to the forward reference point and the lens and the main optical axis is equal to the angle between the optical center ray passing through the pixel position where the forward reference point is located and the optical center ray passing through the pixel position where the distortion center is located, as shown in the figure below: Figure 5 shown.

[0063] In an optional specific embodiment of the present application, the above-mentioned pixel offset acquisition module 603, which is used to calculate the pixel offset of the distortion center point relative to the forward reference point based on the pixel position of the forward reference point and the pixel position of the distortion center point on the environmental image, can respectively calculate the horizontal pixel offset and the vertical pixel offset of the distortion center point relative to the forward reference point, so as to calculate the horizontal and vertical angles between the lens coordinate system and the vehicle coordinate system based on the horizontal pixel offset and the vertical pixel offset.

[0064] The coordinate system angle acquisition module 604 is used to calculate the angle between the lens coordinate system of the forward vehicle-mounted camera and the body coordinate system of the vehicle based on the pixel offset. It can ultimately automatically and efficiently calculate the angle between the lens coordinate system of the forward vehicle-mounted camera and the body coordinate system of the vehicle based on the pixel offset of the distortion center point relative to the forward reference point, thereby completing the external parameter calibration of the forward vehicle-mounted camera.

[0065] In an optional specific embodiment of the present application, the above-mentioned module for calculating the angle between the lens coordinate system of the forward vehicle-mounted camera and the body coordinate system of the vehicle based on pixel offset can calculate the horizontal angle between the lens coordinate system and the body coordinate system based on the horizontal pixel offset, and calculate the vertical angle between the lens coordinate system and the body coordinate system based on the vertical pixel offset.

[0066] In another specific embodiment of the present application: a driving system is provided, wherein the driving system includes an on-board camera extrinsic parameter calibration device, optionally, the on-board camera calibration device includes a processor and a memory, and the processor and the memory are coupled; the memory stores computer instructions, and when the computer instructions are executed, the on-board camera extrinsic parameter calibration device is used to execute the on-board camera extrinsic parameter calibration method in any of the above embodiments.

[0067] In a specific embodiment of the present application, each functional module in the vehicle-mounted camera extrinsic parameter calibration device of the present application can be directly in hardware, in a software module executed by a processor, or in a combination of the two.

[0068] The software modules may reside in RAM memory, flash memory, ROM memory, EPRO0M memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium.

[0069] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination thereof. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. In the alternative, the storage medium may be integral to the processor. The processor and storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In the alternative, the processor and storage medium may reside as discrete components in the user terminal.

[0070] In another specific embodiment of the present application, a computer-readable storage medium stores computer instructions. When the computer instructions are executed, a vehicle-mounted camera extrinsic parameter calibration method in any of the above embodiments is implemented.

[0071] In another specific embodiment of the present application, a computer device includes a processor and a memory, wherein the memory stores computer instructions. When the computer instructions are executed, the vehicle-mounted camera extrinsic parameter calibration method in any of the above embodiments is implemented.

[0072] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0073] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0074] The above are merely embodiments of the present application and are not intended to limit the patent scope of the present application. Any equivalent structural transformations made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for calibrating extrinsic parameters of a vehicle-mounted camera, characterized in that: include: Acquire an environmental image of the vehicle's environment using a forward-facing vehicle-mounted camera, and perceive the environmental image to acquire two or more lane line lines located on both sides of the vehicle; Obtaining a point on the environment image with the minimum distance from the two or more lane line straight lines as a forward reference point, and obtaining a pixel position of the forward reference point; Calculating a pixel offset of the distortion center point relative to the forward reference point based on a pixel position of the forward reference point and a pixel position of the distortion center point on the environment image; as well as, The angle between the lens coordinate system of the forward vehicle-mounted camera and the body coordinate system of the vehicle is calculated according to the pixel offset.

2. The vehicle-mounted camera extrinsic parameter calibration method according to claim 1, characterized in that: The method of acquiring an environmental image of an environment in which the vehicle is located by using a forward-facing vehicle-mounted camera and sensing the environmental image to acquire two or more lane line straight lines located on both sides of the vehicle includes: Acquire a first environmental image of the environment in which the vehicle is located at time t1 using a forward-facing vehicle-mounted camera, and sense lane lines of a first road section located on both sides of the vehicle in the first environmental image; The lane lines of the current road section are fitted, and whether each lane line of the first road section is a straight line is determined according to the fitting result.

3. The vehicle-mounted camera extrinsic parameter calibration method according to claim 2, characterized in that: The method of acquiring an environmental image of the environment in which the vehicle is located using a forward-facing vehicle-mounted camera and sensing the environmental image to acquire two or more lane line lines located on both sides of the vehicle further includes: If the number of straight lane lines in the first road section is less than two, or the straight lane lines in the first road section are located on the same side of the vehicle, the forward-facing vehicle-mounted camera is used to obtain a second environmental image of the environment in which the vehicle is located at time t2, and the second environmental image is perceived to obtain two or more straight lane lines located on both sides of the vehicle.

4. The method for calibrating extrinsic parameters of a vehicle-mounted camera according to claim 1, wherein: The obtaining, as a forward reference point, a point on the environment image having the smallest sum of distances from the two or more lane line straight lines, includes: An intersection point of the two or more lane line straight lines or an intersection point of extension lines of the two or more lane line straight lines is obtained as the forward reference point.

5. The vehicle-mounted camera extrinsic parameter calibration method according to claim 1, characterized in that: The calculating the pixel offset of the distortion center point relative to the forward reference point includes: Calculate the horizontal pixel offset and the vertical pixel offset of the distortion center point relative to the forward reference point respectively.

6. The vehicle-mounted camera extrinsic parameter calibration method according to claim 5, characterized in that: Calculating the angle between the lens coordinate system of the forward vehicle-mounted camera and the body coordinate system of the vehicle according to the pixel offset includes: The horizontal angle between the lens coordinate system and the vehicle coordinate system is calculated according to the horizontal pixel offset, and the vertical angle between the lens coordinate system and the vehicle coordinate system is calculated according to the vertical pixel offset.

7. A vehicle-mounted camera extrinsic parameter calibration device, characterized in that: include, A lane line acquisition module is used to acquire an environmental image of the vehicle's environment using a forward-facing vehicle-mounted camera, and to sense the environmental image to acquire two or more lane line lines located on both sides of the vehicle; A forward reference point pixel position acquisition module is used to obtain a point on the environment image with the minimum distance to the two or more lane line straight lines as a forward reference point, and obtain the pixel position of the forward reference point; a pixel offset acquisition module, configured to calculate the pixel offset of the distortion center point relative to the forward reference point based on the pixel position of the forward reference point and the pixel position of the distortion center point on the environment image; as well as A coordinate system angle acquisition module is used to calculate the angle between the lens coordinate system of the forward-facing vehicle-mounted camera and the body coordinate system of the vehicle based on the pixel offset.

8. A driving system, characterized in that: Including, the driving system includes the vehicle-mounted camera extrinsic parameter calibration device as described in claim 7.

9. A computer-readable storage medium storing computer instructions, characterized in that: When the computer instructions are executed, the computer executes the vehicle-mounted camera extrinsic parameter calibration method according to any one of claims 1 to 6.

10. A computer device comprising a processor and a memory, wherein the memory stores computer instructions, When the computer instructions are executed by the processor, the vehicle-mounted camera extrinsic parameter calibration method according to any one of claims 1 to 6 is implemented.

Images