Online calibration method, device, vehicle and storage medium for vehicle-mounted camera

Abstract

The present application relates to the field of autonomous driving technology, and in particular to an online calibration method, device, vehicle, and storage medium for an on-board camera, wherein the method comprises: obtaining a lane line image of the current road captured by the on-board camera; identifying lane line features in the lane line image, calculating lane line intersections in the lane line image based on the lane line features, and / or projecting the lane line features onto a preset bird's-eye view to obtain lane line projections; calculating the optimal camera pose of the on-board camera based on the lane line intersections and / or lane line projections, matching an optimized value based on the pose difference between the optimal camera pose and the actual camera pose, using the optimized value to correct the yaw and pitch angles of the on-board camera, and determining the online calibration result of the on-board camera. This solves the problem in the related art that an inertial measurement unit needs to be additionally installed to calibrate the pitch angle of the on-board camera, which is costly, and the inertial measurement unit has low accuracy, which can lead to errors in the results of the online camera calibration.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to an online calibration method, device, vehicle, and storage medium for a vehicle-mounted camera. Background Art

[0002] Autonomous driving solutions with panoramic vision typically use six camera modules installed in a circle around the vehicle to perceive its surroundings in 360°. As a tool for collecting image information, the quality of camera calibration directly impacts the accuracy of visual perception results. Factory calibration is performed during pre-installed vehicle production. To account for camera position drift during vehicle operation or during bumpy rides, online calibration, which dynamically updates and corrects the camera's pose, is crucial.

[0003] In related technologies, the pitch angle of the vehicle-mounted camera is determined based on the direction of gravity, the angle between the vehicle-mounted camera and the inertial measurement unit, and the pitch angle direction determined by the inertial measurement unit. The pitch angle of the vehicle-mounted camera can be calibrated online in real time in a fully automatic manner in combination with the inertial measurement unit.

[0004] However, the related technology requires the additional installation of an inertial measurement unit to calibrate the pitch angle of the vehicle-mounted camera, which will cause additional cost. At the same time, due to the existence of errors, the inertial measurement unit has low accuracy. The inertial measurement unit may not accurately track the movement of the vehicle to obtain the first trajectory corresponding to a moving distance, which will cause corresponding errors in the results of the camera online calibration. Summary of the Invention

[0005] The present application provides an online calibration method, device, vehicle, and storage medium for a vehicle-mounted camera to solve the problems in the related art of requiring the additional installation of an inertial measurement unit for pitch angle calibration of the vehicle-mounted camera, which is costly and has low accuracy, resulting in errors in the results of the camera's online calibration.

[0006] In a first aspect, an embodiment of the present application provides an online calibration method for a vehicle-mounted camera, comprising the following steps: obtaining a lane line image around a current road captured by at least one vehicle-mounted camera; identifying at least one lane line feature in the lane line image, calculating a lane line intersection in the lane line image based on the at least one lane line feature, and / or projecting the at least one lane line feature onto a preset bird's-eye view to obtain a lane line projection; calculating the optimal camera pose of the at least one vehicle-mounted camera based on the lane line intersection and / or the lane line projection, matching an optimization value based on the pose difference between the optimal camera pose and the actual camera pose, using the optimization value to correct the yaw angle and pitch angle of the at least one vehicle-mounted camera, and determining the online calibration result of the at least one vehicle-mounted camera.

[0007] According to the above technical means, the embodiment of the present application can obtain the lane line intersection and lane line projection in the lane line image based on the lane line features around the road, and calculate the camera posture. By correcting and restoring the camera's yaw angle and pitch angle, the posture of the vehicle-mounted camera can be calibrated online, so that the vehicle-mounted camera can realize its own online calibration based on the image collected by the vehicle-mounted camera, without the need to use additional sensors, thereby reducing the cost of online calibration. At the same time, online calibration can be a calibration correction based on the prior knowledge of the factory calibration results, so the speed and accuracy of the online calibration results can be guaranteed.

[0008] Optionally, in one embodiment of the present application, when the at least one vehicle-mounted camera is a front-view camera or a rear-view camera, the optimal camera pose of the at least one vehicle-mounted camera is calculated based on the lane line intersection and / or the lane line projection, including: calculating the optimal camera pose of the front-view camera or the rear-view camera based on the lane line intersection.

[0009] According to the above technical means, the embodiment of the present application can calculate the optimal camera posture of the front-view camera or the rear-view camera based on the intersection of the lane lines, thereby restoring the yaw angle and pitch angle of the camera, so that the lane lines are in a flat state, achieving the effect of online calibration of the front-view and rear-view cameras.

[0010] Optionally, in one embodiment of the present application, when the at least one vehicle-mounted camera is a left-view camera or a right-view camera, the optimal camera pose of the at least one vehicle-mounted camera is calculated based on the lane line intersection and / or the lane line projection, including: identifying the same lane line captured by the left-view camera or the right-view camera and the front-view camera and / or the rear-view camera; obtaining the distance difference between the left-view camera or the right-view camera and the lane line projection corresponding to the same lane line in the front-view camera and / or the rear-view camera, and calculating the optimal camera pose of the left-view camera or the right-view camera based on the distance difference and the optimal camera pose of the front-view camera or the rear-view camera.

[0011] According to the above technical means, in an embodiment of the present application, when the lane line in the left-view or right-view camera may be the same as that appearing in the front view or rear view, the optimal camera posture of the left-view or right-view camera is determined based on the distance difference between the lane line projections corresponding to the same lane line, thereby realizing the function of online calibration of the left-view or right-view camera.

[0012] Optionally, in one embodiment of the present application, projecting the at least one lane line feature onto a preset bird's-eye view to obtain a lane line projection includes: projecting the lane lines captured by the calibrated front-view camera and / or the rear-view camera onto the preset bird's-eye view to obtain lane line projections corresponding to the lane lines captured by the front-view camera and / or the rear-view camera; projecting the lane lines captured by the left-view camera and / or the right-view camera onto the preset bird's-eye view to obtain lane line projections corresponding to the lane lines captured by the left-view camera and / or the right-view camera.

[0013] According to the above technical means, the embodiment of the present application can project the lane lines in the front and / or rear view cameras and the left and / or right view cameras with optimized posture onto the bird's-eye view map, further determine the optimal camera posture of the vehicle-mounted camera, and ensure the efficiency of calibration and the accuracy of the results.

[0014] A second aspect of the present application provides an online calibration device for a vehicle-mounted camera, comprising: an acquisition module for acquiring a lane line image around a current road captured by at least one vehicle-mounted camera; an identification module for identifying at least one lane line feature in the lane line image, calculating a lane line intersection in the lane line image based on the at least one lane line feature, and / or projecting the at least one lane line feature onto a preset bird's-eye view to obtain a lane line projection; a calibration module for calculating the optimal camera pose of the at least one vehicle-mounted camera based on the lane line intersection and / or the lane line projection, matching an optimization value based on the pose difference between the optimal camera pose and the actual camera pose, using the optimization value to correct the yaw angle and pitch angle of the at least one vehicle-mounted camera, and determining the online calibration result of the at least one vehicle-mounted camera.

[0015] Optionally, in one embodiment of the present application, the calibration module is further used to calculate the optimal camera pose of the front-view camera or the rear-view camera according to the lane line intersection when the at least one vehicle-mounted camera is a front-view camera or a rear-view camera.

[0016] Optionally, in one embodiment of the present application, the calibration module is further used to, when the at least one vehicle-mounted camera is a front-view camera or a rear-view camera, identify the same lane line captured by the left-view camera or the right-view camera and the front-view camera and / or the rear-view camera; obtain the distance difference between the lane line projections corresponding to the same lane line in the left-view camera or the right-view camera and the front-view camera and / or the rear-view camera, and calculate the optimal camera pose of the left-view camera or the right-view camera based on the distance difference and the optimal camera pose of the front-view camera or the rear-view camera.

[0017] Optionally, in one embodiment of the present application, the recognition module is further used to project the lane lines captured by the calibrated front-view camera and / or the rear-view camera onto the preset bird's-eye view to obtain lane line projections corresponding to the lane lines captured by the front-view camera and / or the rear-view camera; and project the lane lines captured by the left-view camera and / or the right-view camera onto the preset bird's-eye view to obtain lane line projections corresponding to the lane lines captured by the left-view camera and / or the right-view camera.

[0018] A third aspect of the present application provides a vehicle, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the online calibration method for the vehicle-mounted camera as described in the above embodiment.

[0019] A fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the online calibration method for a vehicle-mounted camera as described in the above embodiment.

[0020] Therefore, this application has at least the following beneficial effects:

[0021] 1. The embodiment of the present application can obtain the lane line intersection and lane line projection in the lane line image based on the lane line features around the road, and calculate the camera pose. By correcting and restoring the camera's yaw angle and pitch angle, the pose of the on-board camera can be calibrated online, so that the on-board camera can be calibrated online based on the image collected by the on-board camera without the need for additional sensors, thereby reducing the cost of online calibration. At the same time, the online calibration can be a calibration correction based on the prior knowledge of the factory calibration result, thereby ensuring the speed and accuracy of the online calibration result.

[0022] 2. The embodiment of the present application can calculate the optimal camera pose of the front-view camera or the rear-view camera based on the intersection of the lane lines, thereby restoring the yaw angle and pitch angle of the camera, making the lane lines in a flat state, and achieving the effect of online calibration of the front-view and rear-view cameras.

[0023] 3. In an embodiment of the present application, when the lane line in the left-view or right-view camera may be the same as that appearing in the front view or rear view, the optimal camera posture of the left-view or right-view camera is determined based on the distance difference between the lane line projections corresponding to the same lane line, thereby realizing the function of online calibration of all cameras.

[0024] 4. The embodiment of the present application can project the lane lines in the front-view and / or rear-view cameras and the left-view camera and / or right-view camera after posture optimization onto the bird's-eye view map, further determine the optimal camera posture of the vehicle-mounted camera, and ensure the efficiency of calibration and the accuracy of the results.

[0025] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0027] Figure 1 This is a flowchart of an online calibration method for a vehicle-mounted camera provided according to an embodiment of the present application;

[0028] Figure 2 A schematic diagram of an online calibration method for a vehicle-mounted camera provided according to an embodiment of the present application;

[0029] Figure 3 Schematic diagram of a block diagram of an online calibration device for a vehicle-mounted camera according to an embodiment of the present application;

[0030] Figure 4 Schematic diagram of the structure of a vehicle according to an embodiment of the present application.

[0031] Description of the accompanying drawings: acquisition module-100, identification module-200, calibration module-300, memory-401, processor-402, communication interface-403. DETAILED DESCRIPTION

[0032] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0033] The following describes the online calibration method, device, vehicle and storage medium of the vehicle-mounted camera of the embodiment of the present application with reference to the accompanying drawings. In response to the problems mentioned in the above background technology, the present application provides an online calibration method for a vehicle-mounted camera. In this method, the lane line intersection and lane line projection in the lane line image are obtained according to the lane line features around the road, and the camera posture is calculated. By correcting and restoring the yaw angle and pitch angle of the camera, the posture of the vehicle-mounted camera is calibrated online, so that the vehicle-mounted camera can realize its own online calibration based on the image collected by the vehicle-mounted camera without the use of additional sensors, reducing the cost of online calibration. At the same time, the online calibration can be a calibration correction based on the prior knowledge of the factory calibration result, so the speed and accuracy of the online calibration result can be guaranteed. Therefore, the problem that the related art needs to install an additional inertial measurement unit to calibrate the pitch angle of the vehicle-mounted camera, which is high in cost and low in accuracy of the inertial measurement unit, which will lead to errors in the results of the camera online calibration is solved.

[0034] Specifically, Figure 1 A flowchart of an online calibration method for a vehicle-mounted camera provided in an embodiment of the present application.

[0035] like Figure 1 As shown, the online calibration method of the vehicle-mounted camera includes the following steps:

[0036] In step S101 , a lane line image around a current road captured by at least one vehicle-mounted camera is obtained.

[0037] It is understood that the embodiments of the present application can identify lane features based on images captured by an onboard camera and perform online calibration based on the lane detection results, without the need for additional sensors, thereby ensuring calibration efficiency and accuracy. Therefore, before performing lane feature recognition, the embodiments of the present application can first obtain an image of the lane lines around the current road captured by the onboard camera.

[0038] In step S102, at least one lane line feature in the lane line image is identified, a lane line intersection in the lane line image is calculated based on the at least one lane line feature, and / or the at least one lane line feature is projected onto a preset bird's-eye view to obtain a lane line projection.

[0039] Specifically, it can be found from the lane line image around the current road captured by the on-board camera that the lanes in the image are not parallel, and the points of intersection in the image are called lane line intersections. The embodiment of the present application can identify the lane line image to obtain the lane line features in the lane line image, and calculate the lane line intersections in the lane line image, and further use the lane line intersections to calculate the posture of the on-board camera. The embodiment of the present application can also project the lane line features onto the bird's-eye view to obtain the lane line projection, and further determine the optimal camera posture of the on-board camera to ensure the efficiency of the calibration and the accuracy of the results. In the actual implementation process, the embodiment of the present application can use image recognition technology or target detection algorithm to identify lane line features. For this, those skilled in the art can make settings according to actual conditions without making specific limitations.

[0040] In step S103, the optimal camera pose of at least one vehicle-mounted camera is calculated based on the lane line intersection and / or lane line projection, an optimization value is matched based on the pose difference between the optimal camera pose and the actual camera pose, the yaw angle and pitch angle of the at least one vehicle-mounted camera are corrected using the optimization value, and the online calibration result of the at least one vehicle-mounted camera is determined.

[0041] It can be understood that the embodiment of the present application can calculate the optimal camera pose of the vehicle-mounted camera based on the lane line intersection and / or lane line projection, and correct the yaw angle and pitch angle of the current vehicle-mounted camera, so that the online calibration result of the vehicle-mounted camera is more accurate. At the same time, the results of lane line detection are utilized and corrections are made based on the factory calibration results to further ensure the efficiency of the online calibration results.

[0042] In one embodiment of the present application, when at least one vehicle-mounted camera is a front-view camera or a rear-view camera, the optimal camera pose of at least one vehicle-mounted camera is calculated based on the lane line intersection and / or lane line projection, including: calculating the optimal camera pose of the front-view camera or the rear-view camera based on the lane line intersection.

[0043] It is understandable that for images captured by the vehicle-mounted camera, which is a front-view camera or a rear-view camera, it can be found that the lanes in the image are not balanced, such as Figure 2 As shown, the embodiment of the present application can calculate the lane line intersection in the lane line image based on the lane line detection results, and use the lane line intersection to calculate the optimal position of the camera, thereby restoring the yaw angle and pitch angle of the camera, so that the lane line is in a flat state, achieving the effect of online calibration of the front and rear view cameras.

[0044] In one embodiment of the present application, when at least one vehicle-mounted camera is a left-view camera or a right-view camera, the optimal camera pose of at least one vehicle-mounted camera is calculated based on the lane line intersection and / or lane line projection, including: identifying the same lane line captured by the left-view camera or the right-view camera and the front-view camera and / or the rear-view camera; obtaining the distance difference between the lane line projection corresponding to the same lane line in the left-view camera or the right-view camera and the front-view camera and / or the rear-view camera, and calculating the optimal camera pose of the left-view camera or the right-view camera based on the distance difference and the optimal camera pose of the front-view camera or the rear-view camera.

[0045] It is understandable that considering that the lane line in the left or right view camera may be the same as that in the front or rear view, the optimal camera pose of the left or right view camera can be obtained based on this condition, thereby realizing the function of online calibration of all cameras.

[0046] Specifically, if Figure 2As shown, the embodiment of the present application can identify the same lane lines captured by the left-view camera or the right-view camera and the front-view camera and / or the rear-view camera, and project the lane lines in the front-view and rear-view cameras with optimized postures onto the bird's-eye view. For the left-view or right-view cameras located on both sides, the same lane lines are also projected onto the bird's-eye view. It can be found that there are some deviations in the postures. However, for the same lane line, after it is projected onto the bird's-eye view in the current and front-view or rear-view camera images, the distance between the two should be very close. The embodiment of the present application can obtain the optimal camera pose of the left-view camera or the right-view camera based on the distance difference between the lane line projections corresponding to the same lane line and the optimal camera pose of the front-view camera or the rear-view camera.

[0047] In one embodiment of the present application, at least one lane line feature is projected onto a preset bird's-eye view to obtain a lane line projection, including: projecting the lane lines captured by the calibrated front-view camera and / or rear-view camera onto the preset bird's-eye view to obtain lane line projections corresponding to the lane lines captured by the front-view camera and / or rear-view camera; projecting the lane lines captured by the left-view camera and / or right-view camera onto the preset bird's-eye view to obtain lane line projections corresponding to the lane lines captured by the left-view camera and / or right-view camera.

[0048] It can be understood that the embodiments of the present application can project the lane lines in the front-view and / or rear-view cameras after posture optimization onto the bird's-eye view, and project the lane lines collected by the left-view camera and / or right-view camera onto the bird's-eye view, so as to obtain the actual physical representation of the lane lines in the left-view or right-view camera screen, further determine the optimal camera posture of the vehicle-mounted camera, and ensure the efficiency of calibration and the accuracy of the results.

[0049] According to the online calibration method for vehicle-mounted cameras proposed in the embodiment of the present application, the lane line intersection and lane line projection in the lane line image are obtained based on the lane line features around the road, and the camera posture is calculated. By correcting and restoring the camera's yaw angle and pitch angle, the vehicle-mounted camera posture is calibrated online. This allows the vehicle-mounted camera to perform its own online calibration based on the image captured by the vehicle-mounted camera without the need for additional sensors, reducing the cost of online calibration. At the same time, online calibration can be a calibration correction based on the prior knowledge of the factory calibration result, thereby ensuring the speed and accuracy of the online calibration result. This solves the problem in the related art of requiring the additional installation of an inertial measurement unit for vehicle-mounted camera pitch angle calibration, which is costly, and the low accuracy of the inertial measurement unit, which can lead to errors in the results of the camera online calibration.

[0050] Next, an online calibration device for a vehicle-mounted camera proposed according to an embodiment of the present application will be described with reference to the accompanying drawings.

[0051] Figure 3 4 is a block diagram of an online calibration device for a vehicle-mounted camera according to an embodiment of the present application.

[0052] like Figure 3 As shown, the online calibration device 10 for a vehicle-mounted camera includes: an acquisition module 100 , a recognition module 200 and a calibration module 300 .

[0053] Among them, the acquisition module 100 is used to obtain a lane line image around the current road captured by at least one vehicle-mounted camera; the recognition module 200 is used to identify at least one lane line feature in the lane line image, calculate the lane line intersection in the lane line image based on the at least one lane line feature, and / or project the at least one lane line feature onto a preset bird's-eye view to obtain a lane line projection; the calibration module 300 is used to calculate the optimal camera pose of at least one vehicle-mounted camera based on the lane line intersection and / or the lane line projection, match the optimization value based on the pose difference between the optimal camera pose and the actual camera pose, use the optimization value to correct the yaw angle and pitch angle of at least one vehicle-mounted camera, and determine the online calibration result of at least one vehicle-mounted camera.

[0054] In one embodiment of the present application, the calibration module 300 is further configured to calculate the optimal camera pose of the front-view camera or the rear-view camera according to the lane line intersection when at least one vehicle-mounted camera is a front-view camera or a rear-view camera.

[0055] In one embodiment of the present application, the calibration module 300 is further used to, when at least one vehicle-mounted camera is a front-view camera or a rear-view camera, identify the same lane line captured by the left-view camera or the right-view camera and the front-view camera and / or the rear-view camera; obtain the distance difference between the lane line projections corresponding to the same lane line in the left-view camera or the right-view camera and the front-view camera and / or the rear-view camera, and calculate the optimal camera pose of the left-view camera or the right-view camera based on the distance difference and the optimal camera pose of the front-view camera or the rear-view camera.

[0056] In one embodiment of the present application, the recognition module 200 is further used to project the lane lines captured by the calibrated front-view camera and / or rear-view camera onto a preset bird's-eye view map to obtain lane line projections corresponding to the lane lines captured by the front-view camera and / or rear-view camera; and project the lane lines captured by the left-view camera and / or right-view camera onto a preset bird's-eye view map to obtain lane line projections corresponding to the lane lines captured by the left-view camera and / or right-view camera.

[0057] It should be noted that the above explanation of the embodiment of the online calibration method for a vehicle-mounted camera is also applicable to the online calibration device for a vehicle-mounted camera of this embodiment, and will not be repeated here.

[0058] According to the online calibration device for a vehicle-mounted camera proposed in an embodiment of the present application, the lane line intersection and lane line projection in the lane line image are obtained based on the lane line features around the road, and the camera posture is calculated. By correcting and restoring the camera's yaw and pitch angles, the vehicle-mounted camera posture is calibrated online. This allows the vehicle-mounted camera to perform its own online calibration based on the image captured by the vehicle-mounted camera without the need for additional sensors, thereby reducing the cost of online calibration. At the same time, online calibration can be a calibration correction based on the prior knowledge of the factory calibration result, thereby ensuring the speed and accuracy of the online calibration result. This solves the problem in the related art of requiring the additional installation of an inertial measurement unit for vehicle-mounted camera pitch angle calibration, which is costly, and the low accuracy of the inertial measurement unit, which can lead to errors in the results of the camera online calibration.

[0059] Figure 4 A schematic diagram of the structure of a vehicle provided in an embodiment of the present application. The vehicle may include:

[0060] Memory 401 , processor 402 , and computer programs stored in the memory 401 and executable on the processor 402 .

[0061] When the processor 402 executes the program, the online calibration method for the vehicle-mounted camera provided in the above embodiment is implemented.

[0062] Furthermore, the vehicle further comprises:

[0063] The communication interface 403 is used for communication between the memory 401 and the processor 402 .

[0064] The memory 401 is used to store computer programs that can be run on the processor 402 .

[0065] The memory 401 may include a high-speed RAM (Random Access Memory) memory, and may also include a non-volatile memory, such as at least one disk memory.

[0066] If the memory 401, the processor 402, and the communication interface 403 are implemented independently, the communication interface 403, the memory 401, and the processor 402 can be connected to each other via a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0067] Optionally, in a specific implementation, if the memory 401, the processor 402 and the communication interface 403 are integrated on a chip, the memory 401, the processor 402 and the communication interface 403 can communicate with each other through an internal interface.

[0068] The processor 402 may be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application.

[0069] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned online calibration method for a vehicle-mounted camera.

[0070] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0071] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this application, "N" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0072] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application belong.

[0073] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiment, the N steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array, a field programmable gate array, etc.

[0074] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0075] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. An online calibration method for a vehicle-mounted camera, characterized in that: The following steps are involved: Obtaining a lane line image around a current road captured by at least one vehicle-mounted camera; identifying at least one lane feature in the lane image, calculating a lane intersection in the lane image based on the at least one lane feature, and / or projecting the at least one lane feature onto a preset bird's-eye view image to obtain a lane projection; calculating an optimal camera pose of the at least one on-board camera based on the lane line intersection and / or the lane line projection, matching an optimization value based on a pose difference between the optimal camera pose and an actual camera pose, correcting a yaw angle and a pitch angle of the at least one on-board camera using the optimization value, and determining an online calibration result of the at least one on-board camera; When the at least one vehicle-mounted camera is a front-view camera or a rear-view camera, calculating the optimal camera pose of the at least one vehicle-mounted camera according to the lane line intersection and / or the lane line projection includes: Calculating an optimal camera pose of the front-view camera or the rear-view camera according to the lane line intersection; When the at least one vehicle-mounted camera is a left-view camera or a right-view camera, calculating the optimal camera pose of the at least one vehicle-mounted camera according to the lane line intersection and / or the lane line projection includes: Identifying the same lane line captured by the left-view camera or the right-view camera and the front-view camera and / or the rear-view camera; Obtain a distance difference between the lane line projection corresponding to the same lane line in the left-view camera or the right-view camera and the front-view camera and / or the rear-view camera, and calculate the optimal camera pose of the left-view camera or the right-view camera based on the distance difference and the optimal camera pose of the front-view camera or the rear-view camera.

2. The method according to claim 1, characterized in that The projecting the at least one lane line feature onto a preset bird's-eye view to obtain a lane line projection includes: Projecting the calibrated lane lines captured by the front-view camera and / or the rear-view camera onto the preset bird's-eye view map to obtain lane line projections corresponding to the lane lines captured by the front-view camera and / or the rear-view camera; The lane lines captured by the left-view camera and / or the right-view camera are projected onto the preset bird's-eye view map to obtain lane line projections corresponding to the lane lines captured by the left-view camera and / or the right-view camera.

3. An online calibration device for a vehicle-mounted camera, characterized in that: include: An acquisition module, configured to acquire a lane line image around a current road captured by at least one vehicle-mounted camera; a recognition module, configured to recognize at least one lane line feature in the lane line image, calculate a lane line intersection in the lane line image based on the at least one lane line feature, and / or project the at least one lane line feature onto a preset bird's-eye view image to obtain a lane line projection; a calibration module, configured to calculate an optimal camera pose of the at least one on-board camera based on the lane line intersection and / or the lane line projection, match an optimization value based on a pose difference between the optimal camera pose and an actual camera pose, correct a yaw angle and a pitch angle of the at least one on-board camera using the optimization value, and determine an online calibration result of the at least one on-board camera; The calibration module is further configured to: When the at least one vehicle-mounted camera is a front-view camera or a rear-view camera, calculating an optimal camera pose of the front-view camera or the rear-view camera according to the lane line intersection; The calibration module is further configured to: When the at least one vehicle-mounted camera is a front-view camera or a rear-view camera, identifying the same lane line captured by the left-view camera or the right-view camera and the front-view camera and / or the rear-view camera; Obtain a distance difference between the lane line projection corresponding to the same lane line in the left-view camera or the right-view camera and the front-view camera and / or the rear-view camera, and calculate the optimal camera pose of the left-view camera or the right-view camera based on the distance difference and the optimal camera pose of the front-view camera or the rear-view camera.

4. The device according to claim 3, characterized in that The identification module is further configured to: Projecting the calibrated lane lines captured by the front-view camera and / or the rear-view camera onto the preset bird's-eye view map to obtain lane line projections corresponding to the lane lines captured by the front-view camera and / or the rear-view camera; The lane lines captured by the left-view camera and / or the right-view camera are projected onto the preset bird's-eye view map to obtain lane line projections corresponding to the lane lines captured by the left-view camera and / or the right-view camera.

5. A vehicle, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the online calibration method for a vehicle-mounted camera according to any one of claims 1 to 2.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the online calibration method for a vehicle-mounted camera according to any one of claims 1 to 2.

Images