Calibration device and method for motor vehicle panoramic image system

Abstract

The embodiment of the application provides a kind of calibration device and method of motor vehicle panoramic image system, the device includes: image extraction module, from the road video image of acquisition extracts the original road image corresponding to lateral;Image correction module, each original road image is corrected to obtain corrected road image;Lane detection and judgment module, from each corrected road image, the original lane line is detected, when any adjacent lateral of two corrected road images exists original lane line belonging to the same actual lane line, output calibratable instruction;Lane projection module, the projection lane line generated by original lane line projection to the preset panoramic overhead plane is calculated, loss function is constructed based on approximation algorithm model, and optimal RT inverse projection matrix is obtained;Panoramic image generation module, each corrected road image is projected to panoramic overhead plane to generate target panoramic image.This embodiment can be conveniently calibrated by user and has high calibration efficiency.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of motor vehicle assisted driving technology, and in particular to a calibration device and method for a motor vehicle panoramic imaging system. Background Art

[0002] An Around View Monitor (AVM) system for vehicles projects four images from the front, back, left, and right sides of a vehicle onto a pre-defined overhead panoramic plane to create a target panoramic image of the vehicle. To effectively stitch these four images together, image calibration is required during the development of AVM systems.

[0003] The calibration of an existing motor vehicle panoramic imaging system is usually completed at a calibration site before the vehicle leaves the factory. However, when a user actually collides with the vehicle, causing the shooting angle and position of the on-board camera to change, or when the user replaces the on-board camera, the panoramic imaging system calibrated before leaving the factory can no longer be used. The user can only return the vehicle to the factory for recalibration, which takes a lot of time. The user cannot perform the calibration by himself, and the calibration efficiency is relatively low. Summary of the Invention

[0004] The technical problem to be solved by the embodiments of the present invention is to provide a calibration device for a panoramic imaging system of a motor vehicle, which can facilitate user self-calibration and has high calibration efficiency.

[0005] A further technical problem to be solved by the embodiments of the present invention is to provide a calibration method for a panoramic imaging system of a motor vehicle, which is convenient for users to calibrate themselves and has high calibration efficiency.

[0006] In order to solve the above technical problems, the embodiment of the present invention first provides the following technical solution: a calibration device for a motor vehicle panoramic imaging system, comprising:

[0007] An image extraction module is connected to the on-board cameras respectively arranged on each side of the motor vehicle, and is used to extract the original road image of the corresponding side from the road video images captured by each of the on-board cameras in response to the initial calibration instruction;

[0008] An image correction module, connected to the image extraction module, for performing image correction on each original road image using a pre-stored image correction model to obtain a corresponding corrected road image;

[0009] A lane line detection and judgment module, connected to the image correction module, is used to detect original lane lines from each corrected road image using a pre-stored lane line detection model, analyze and judge the original lane lines detected in each corrected road image, and output a calibrable instruction when the original lane lines belonging to the same actual lane line are present in any two adjacent corrected road images in both directions;

[0010] a lane line projection module, connected to the lane line detection and judgment module, configured to respond to the calibratable instruction, calculate projected lane lines generated by projecting the original lane lines in each corrected road image onto a preset panoramic bird's-eye view plane using a pre-stored initial RT inverse projection matrix, construct a loss function for the planar positional relationship between two projected lane lines belonging to the same actual lane line based on an approximation algorithm model, and solve the loss function to obtain an optimal RT inverse projection matrix corresponding to each corrected road image when the loss function is minimized; and

[0011] The panoramic image generation module is connected to the lane line projection module and is used to respectively use each of the optimal RT inverse projection matrices to project each corrected road image onto the panoramic bird's-eye view plane to generate a target panoramic image.

[0012] Furthermore, the device further comprises:

[0013] An iterative control module is connected to the lane line projection module and is used to control the lane line projection module to cycle a predetermined number of times. In each cycle, the optimal RT inverse projection matrix calculated in the previous cycle is used to update the initial RT inverse projection matrix, and the optimal RT inverse projection matrix calculated in the last cycle is output to the panoramic image generation module.

[0014] Furthermore, the device further comprises:

[0015] A loop detection module is connected to the panoramic image generation module and the iterative control module, respectively, and is used to determine whether the parallelism and relative distance of each projected lane line in the target panoramic image meet preset calibration conditions. When the parallelism and relative distance do not meet the preset calibration conditions, the iterative control module is controlled to restart.

[0016] Furthermore, the device further comprises:

[0017] A preprocessing module is connected to the image correction module and the lane line detection and judgment module respectively, and is used to preprocess the corrected road image and output it to the lane line detection and judgment module. The preprocessing includes at least: image enhancement, edge detection, contour processing, perspective transformation and pixel integration.

[0018] Furthermore, the lane line detection model is a least squares lane line detection fitting model.

[0019] On the other hand, in order to solve the above-mentioned further technical problems, the embodiment of the present invention further provides the following technical solution: a calibration method of a vehicle panoramic imaging system, comprising the following steps:

[0020] An image extraction step, in response to the initial calibration instruction, extracts an original road image of the corresponding side from the road video images captured by the on-board cameras respectively arranged on each side of the motor vehicle;

[0021] An image correction step, using a pre-stored image correction model to perform image correction on each original road image to obtain a corresponding corrected road image;

[0022] A lane line detection and judgment step uses a pre-stored lane line detection model to detect original lane lines from each corrected road image, analyzes and judges the original lane lines detected in each corrected road image, and outputs a calibratable instruction when the original lane lines belonging to the same actual lane line exist in any two adjacent corrected road images in both directions;

[0023] a lane line projection step, in response to the calibratable instruction, calculating projected lane lines generated by projecting the original lane lines in each of the corrected road images onto a preset panoramic bird's-eye view plane using a pre-stored initial RT inverse projection matrix, constructing a loss function for the planar positional relationship between two projected lane lines belonging to the same actual lane line based on an approximation algorithm model, and solving the loss function to obtain an optimal RT inverse projection matrix corresponding to each of the corrected road images when the loss function is minimized; and

[0024] The panoramic image generation step is to project each corrected road image onto the panoramic bird's-eye view plane using each of the optimal RT inverse projection matrices to generate a target panoramic image.

[0025] Furthermore, the method further comprises:

[0026] An iterative control step controls the lane line projection step to be looped for a predetermined number of times, wherein the optimal RT inverse projection matrix calculated in the previous cycle is used to update the initial RT inverse projection matrix in each cycle, and the optimal RT inverse projection matrix calculated in the last cycle is used to execute the panoramic image generation step.

[0027] Furthermore, the method further comprises:

[0028] The loop detection step determines whether the parallelism and relative distance of each projected lane line in the target panoramic image meet the preset calibration conditions, and re-executes the iterative control step when the parallelism and relative distance do not meet the preset calibration conditions.

[0029] Furthermore, the method further comprises:

[0030] A preprocessing step is performed on the corrected road image before executing the lane line detection and judgment step. The preprocessing step includes at least: image enhancement, edge detection, contour processing, perspective transformation and pixel integration.

[0031] Furthermore, the lane line detection model is a least squares lane line detection fitting model.

[0032] After adopting the above technical solution, the embodiment of the present invention has at least the following beneficial effects: after the embodiment of the present invention extracts the original road image from the road video image collected by each vehicle-mounted camera through the image extraction module, the image correction module is first used to perform image correction on the original road image to obtain a corrected road image, and then the lane line detection and judgment module detects the original lane line from each corrected road image. When the original lane line exists in at least two corrected road images and the original lane line belongs to the same actual lane line, the panoramic imaging system can be calibrated, thereby outputting a calibrable instruction, and further the lane line projection module calculates the projection of the original lane line onto the panoramic bird's-eye view plane by using the pre-stored initial RT inverse projection matrix. Projected lane lines. In theory, the distance between projected lane lines belonging to the same actual lane line should be zero, and they should be aligned and parallel with each other and have the same slope. Taking into account the actual error, the distance loss function is constructed to minimize the loss function by solving the loss function, thereby obtaining the optimal RT inverse projection matrix. Finally, the panoramic image generation module can project each corrected road image onto the panoramic bird's-eye view plane based on the optimal RT inverse projection matrix, and finally generate the target panoramic image. When the user needs to calibrate the panoramic imaging system of a motor vehicle, he only needs to drive the motor vehicle on a road with lane lines, and then calibrate by automatically or manually inputting the initial calibration command. The overall calibration process is simple and the user can perform the calibration by himself. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a functional module diagram of an optional embodiment of the calibration device of the motor vehicle panoramic imaging system of the present invention.

[0034] Figure 2 This is a top view of an optional embodiment of the calibration device for a panoramic imaging system of a motor vehicle after calibration.

[0035] Figure 3This is a functional module diagram of another optional embodiment of the calibration device of the motor vehicle panoramic imaging system of the present invention.

[0036] Figure 4 The figure is a flowchart of an optional embodiment of the calibration method of the vehicle panoramic imaging system of the present invention. DETAILED DESCRIPTION

[0037] The present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the following exemplary embodiments and descriptions are only used to explain the present invention and are not intended to limit the present invention. Moreover, the embodiments and features in the embodiments of the present application may be combined with each other unless there is a conflict.

[0038] like Figure 1 As shown, an optional embodiment of the present invention provides a calibration device 1 for a panoramic imaging system of a motor vehicle, comprising:

[0039] An image extraction module 10 is connected to the on-board cameras 3 respectively arranged on each side of the motor vehicle, and is used to extract the original road image of the corresponding side from the road video images captured by each of the on-board cameras 3 in response to the initial calibration instruction; an image correction module 11 is connected to the image extraction module 10, and is used to perform image correction on each of the original road images using a pre-stored image correction model to obtain a corresponding corrected road image;

[0040] A lane line detection and judgment module 12 is connected to the image correction module 11 and is used to detect original lane lines from each corrected road image using a pre-stored lane line detection model, analyze and judge the original lane lines detected in each corrected road image, and output a calibrable instruction when the original lane lines belonging to the same actual lane line are present in any two adjacent corrected road images in both directions;

[0041] a lane line projection module 13, connected to the lane line detection and judgment module 12, for responding to the calibratable instruction, calculating projected lane lines generated by projecting the original lane lines in each corrected road image onto a preset panoramic bird's-eye view plane using a pre-stored initial RT inverse projection matrix, constructing a loss function for the planar positional relationship between two projected lane lines belonging to the same actual lane line based on an approximation algorithm model, and solving the loss function to obtain an optimal RT inverse projection matrix corresponding to each corrected road image when the loss function is minimized; and

[0042] The panoramic image generation module 14 is connected to the lane line projection module 13 and is used to respectively use the optimal RT inverse projection matrix to project each corrected road image onto the panoramic bird's-eye view plane to generate a target panoramic image.

[0043] In the embodiment of the present invention, after the image extraction module 10 extracts the original road image from the road video image captured by each vehicle-mounted camera 3, the image correction module 11 is first used to perform image correction on the original road image to obtain a corrected road image. The lane line detection and judgment module 12 further detects the original lane line from each corrected road image. When the original lane line exists in at least two corrected road images and the original lane line belongs to the same actual lane line, the panoramic imaging system can be calibrated, thereby outputting a calibrable instruction. The lane line projection module 13 further calculates the projected lane line generated by projecting the original lane line onto the panoramic bird's-eye view plane by using a pre-stored initial RT inverse projection matrix. The theoretical state Under the condition that the distance between the projected lane lines belonging to the same actual lane line should be zero, they should be aligned and parallel with each other and have the same slope. In consideration of the actual error, the loss function is constructed to minimize the loss function by solving the loss function, thereby obtaining the optimal RT inverse projection matrix. Finally, the panoramic image generation module 14 can project each corrected road image to the panoramic bird's-eye view plane according to each optimal RT inverse projection matrix, and finally generate the target panoramic image. When the user needs to calibrate the panoramic imaging system of a motor vehicle, he only needs to drive the motor vehicle on a road with lane lines, and then automatically or manually input the initial calibration command to achieve calibration. The overall calibration process is simple, and the user can perform the calibration by himself. In specific implementation, such as Figure 1 As shown, a vehicle is usually equipped with a vehicle-mounted camera 3 at each of the front, rear, left, and right sides of the vehicle for capturing road video images at the corresponding side of the vehicle.

[0044] In a specific implementation, first, the internal parameter matrix of the vehicle-mounted camera 3 according to the embodiment of the present invention can be expressed as:

[0045]

[0046] Among them, a x =1 / d x , d x Indicates the discrete width of the image unit pixel, a y =1 / d y , d y Indicates the discrete height of the unit pixel of the image;

[0047] Furthermore, the rotation matrix R and translation vector T of the three-dimensional world coordinates to the camera coordinates can be expressed as:

[0048]

[0049]

[0050] From Formula 2 and Formula 3, the RT matrix can be expressed as:

[0051]

[0052] Furthermore, the angles of rotation of the three-dimensional world coordinates to the camera coordinates around the x-axis, y-axis, and z-axis are set to α (pitch angle), β (roll angle), and γ (yaw angle), respectively;

[0053] Let the rotation matrix R = Rz*Rx*Ry (Formula 5)

[0054] Then there is

[0055]

[0056] In addition, when establishing a three-dimensional world coordinate system, the ground plane is generally set as the zero plane, that is, the z plane in the three-dimensional world coordinate system w =0, according to this condition:

[0057]

[0058] Because z w =0, according to formula 7:

[0059]

[0060]

[0061] Let the H matrix be:

[0062]

[0063]

[0064] Among them, H -1 Represents the inverse projection matrix.

[0065] Where s is the scale factor, (u, v) represents the pixel coordinates in the corrected road image; (x w ,y w , 0) represents the pixel coordinates in the panoramic overhead plane. According to Formula 4 and Formula 7, the original lane lines in the corrected road image can be projected onto the panoramic overhead plane to generate projected lane lines;

[0066] Finally, when solving the loss function, the planar position relationship between the two projected lane lines belonging to the same actual lane line specifically includes: the slopes of the two projected lane lines are closest, the parallelism is maximized, the relative distance is minimized, and the left-right translation distance, the up-down translation distance, and the magnification ratio of the two projected lane lines are minimized. In the solution process, a one-to-one approximation method is adopted. The independent variables when the loss functions of the slope, parallelism, and relative distance are minimized correspond to α (pitch angle), β (roll angle), and γ (yaw angle), respectively. The independent variables when the loss functions of the left-right translation distance, up-down translation distance, and magnification ratio of the two projected lane lines are minimized correspond to t1, t2, and t3, respectively.

[0067] In addition, it can be understood that the embodiment of the present invention calculates and generates an optimal inverse projection matrix for each original lane line in each corrected road image, for example: Figure 2 As shown, during the specific calibration, the driver is usually required to park the motor vehicle between two lane lines. The motor vehicle usually has an on-board camera installed around the body to realize on-board monitoring, so four corrected road images are generated accordingly, that is, four optimal inverse projection matrices are obtained accordingly.

[0068] In an optional embodiment of the present invention, Figure 3 As shown, the device 1 further includes:

[0069] The iterative control module 15 is connected to the lane projection module 13 and is configured to control the lane projection module 13 to cycle a predetermined number of times. In each cycle, the optimal RT inverse projection matrix calculated in the previous cycle is used to update the initial RT inverse projection matrix. The optimal RT inverse projection matrix calculated in the last cycle is then output to the panoramic image generation module 14. In this embodiment, the iterative control module 15 is configured to control the lane projection module 13 to cycle a predetermined number of times. This allows the RT inverse projection matrix to be continuously optimized over multiple iterations, resulting in a more accurate final calibration.

[0070] In an optional embodiment of the present invention, Figure 3 As shown, the device 1 further includes:

[0071] The loop detection module 16 is connected to the panoramic image generation module 14 and the iterative control module 15, respectively, and is used to determine whether the parallelism and relative distance of each projected lane line in the target panoramic image meet the preset calibration conditions. When the parallelism and relative distance do not meet the preset calibration conditions, the iterative control module 15 is controlled to work again.

[0072] In this embodiment, after the target panoramic image is generated, the parallelism and relative distance of the projected lane lines in the target panoramic image are judged again. When the parallelism and relative distance of each projected lane line do not meet the preset calibration conditions, the iterative calculation is repeated again, and the RT inverse projection matrix is ​​calculated until the target panoramic image finally obtained meets the calibration requirements. In a specific implementation, it can be understood that the loop detection module 16 can be a display module that displays the target panoramic image and performs human-computer interaction. By displaying the target panoramic image to the user, the user can observe and determine whether the target panoramic image meets the preset calibration conditions, thereby subjectively determining whether to control the iterative control module 15 to work again. Of course, it can also be an automated detection and judgment; in addition, the preset calibration conditions can be that the parallelism of each projected lane line should ensure no obvious deviation, and the projected lane lines belonging to the same actual lane line should ensure collinear alignment, etc.

[0073] In an optional embodiment of the present invention, Figure 2 As shown, the device 1 further includes:

[0074] The preprocessing module 17 is connected to the image correction module 11 and the lane detection and determination module 12, and is configured to preprocess the corrected road image before outputting it to the lane detection and determination module 12. The preprocessing includes at least image enhancement, edge detection, contour processing, perspective transformation, and pixel integration. In this embodiment, after image correction, optimization processes such as image enhancement, edge detection, contour processing, perspective transformation, and pixel integration are performed on the image to improve the accuracy of subsequent line detection.

[0075] In an optional embodiment of the present invention, the lane line detection model is a least squares lane line detection fitting model. In this embodiment, the lane line detection model uses least squares straight line detection. The overall calculation process is relatively simple and can quickly detect lane lines from the image. In specific implementation, it is understandable that on actual roads, lane lines may have different styles at different positions. For example, at a curve, the lane line is also a curve. Therefore, for different line types, the lane line detection model can be curved lane line detection or straight lane line detection. Generally, in order to improve detection efficiency and simplify detection difficulty, straight lane line detection is usually used in the embodiments of the present invention.

[0076] On the other hand, Figure 3 As shown, an embodiment of the present invention further provides a calibration method for a motor vehicle panoramic imaging system, comprising the following steps:

[0077] S1: an image extraction step, in response to an initial calibration instruction, extracting an original road image of the corresponding side from a road video image captured by the vehicle-mounted camera 3 respectively arranged on each side of the vehicle;

[0078] S2: an image correction step, performing image correction on each original road image using a pre-stored image correction model to obtain a corresponding corrected road image;

[0079] S3: Lane line detection and judgment step, using a pre-stored lane line detection model to detect original lane lines from each corrected road image, and analyzing and judging the original lane lines detected in each corrected road image. When the original lane lines belonging to the same actual lane line exist in any two adjacent corrected road images in both directions, a calibrable instruction is output;

[0080] S4: a lane line projection step, in response to the calibratable instruction, calculating projected lane lines generated by projecting the original lane lines in each of the corrected road images onto a preset panoramic bird's-eye view plane using a pre-stored initial RT inverse projection matrix, constructing a loss function for the planar positional relationship between two projected lane lines belonging to the same actual lane line based on an approximation algorithm model, and solving the loss function to obtain an optimal RT inverse projection matrix corresponding to each of the corrected road images when the loss function is minimized; and

[0081] S5: A panoramic image generation step, wherein each of the optimal RT inverse projection matrices is used to project each corrected road image onto the panoramic bird's-eye view plane to generate a target panoramic image.

[0082] In the embodiment of the present invention, after extracting the original road image from the road video image captured by each vehicle-mounted camera 3 through the above method, the original road image is first corrected to obtain a corrected road image, and the original lane line is further detected from each corrected road image. When the original lane line exists in at least two corrected road images and the original lane line belongs to the same actual lane line, the panoramic imaging system can be calibrated, thereby outputting a calibrable instruction, and further calculating the projected lane line generated by projecting the original lane line onto the panoramic bird's-eye view plane by using the pre-stored initial RT inverse projection matrix. In theory, the original lane line belongs to the same actual lane line. The distance between the projected lane lines should be zero, they should be aligned and parallel to each other and have the same slope. Taking into account the actual error, the loss function is constructed to minimize the loss function by solving the loss function, thereby obtaining the optimal RT inverse projection matrix. Finally, based on the optimal RT inverse projection matrix, each corrected road image can be projected onto the panoramic bird's-eye view plane to generate a target panoramic image. When the user needs to calibrate the panoramic imaging system of a motor vehicle, he only needs to drive the motor vehicle on a road with lane lines, and then calibrate by automatically or manually inputting the initial calibration command. The overall calibration process is simple and the user can perform the calibration by himself.

[0083] In an optional embodiment of the present invention, the method further includes:

[0084] The iterative control step controls step S4 to be looped a predetermined number of times. In each loop, the optimal RT inverse projection matrix calculated in the previous loop is used to update the initial RT inverse projection matrix. The optimal RT inverse projection matrix calculated in the last loop is then used to execute step S5. In this embodiment, step S4 is looped a predetermined number of times, thereby continuously optimizing the RT inverse projection matrix during multiple iterations, thereby making the final calibration more accurate.

[0085] In an optional embodiment of the present invention, the method further includes:

[0086] The loop detection step determines whether the parallelism and relative distances of the projected lane lines in the target panoramic image meet preset calibration conditions. If the parallelism and relative distances do not meet the preset calibration conditions, step S4 is re-executed. In this embodiment, after the target panoramic image is generated, the parallelism and relative distances of the projected lane lines in the target panoramic image are again determined. If the parallelism and relative distances of the projected lane lines do not meet the preset calibration conditions, the iterative calculation and RT inverse projection matrix are repeated until the final target panoramic image meets the calibration requirements.

[0087] In an optional embodiment of the present invention, the method further includes:

[0088] The preprocessing step, in which step S3 is executed after preprocessing the corrected road image, includes at least image enhancement, edge detection, contour processing, perspective transformation, and pixel integration. In this embodiment, after the image is corrected, optimization processes such as image enhancement, edge detection, contour processing, perspective transformation, and pixel integration are performed on the image to improve the accuracy of subsequent line detection.

[0089] In an optional embodiment of the present invention, the lane detection model is a least squares lane detection fitting model. In this embodiment, the lane detection model uses least squares line detection, and the overall calculation process is relatively simple, which can quickly detect lane lines from the image.

[0090] If the functions described in the embodiments of the present invention are implemented in the form of software function modules or units and sold or used as independent products, they can be stored in a storage medium readable by a computing device. Based on this understanding, the part of the embodiment of the present invention that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computing device (which can be a personal computer, server, mobile computing device or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, etc. Various media that can store program codes. The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0091] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which fall within the scope of protection of the present invention.

Claims

1. A calibration device for a motor vehicle panoramic imaging system, characterized in that: The device comprises: An image extraction module is connected to the on-board cameras respectively arranged on each side of the motor vehicle, and is used to extract the original road image of the corresponding side from the road video images captured by each of the on-board cameras in response to the initial calibration instruction; An image correction module, connected to the image extraction module, for performing image correction on each original road image using a pre-stored image correction model to obtain a corresponding corrected road image; A lane line detection and judgment module, connected to the image correction module, is used to detect original lane lines from each corrected road image using a pre-stored lane line detection model, analyze and judge the original lane lines detected in each corrected road image, and output a calibrable instruction when the original lane lines belonging to the same actual lane line are present in any two adjacent corrected road images in both directions; a lane line projection module, connected to the lane line detection and judgment module, configured to respond to the calibratable instruction, calculate projected lane lines generated by projecting the original lane lines in each corrected road image onto a preset panoramic bird's-eye view plane using a pre-stored initial RT inverse projection matrix, construct a loss function for the planar positional relationship between two projected lane lines belonging to the same actual lane line based on an approximation algorithm model, and solve the loss function to obtain an optimal RT inverse projection matrix corresponding to each corrected road image when the loss function is minimized; and The panoramic image generation module is connected to the lane line projection module and is used to respectively use each of the optimal RT inverse projection matrices to project each corrected road image onto the panoramic bird's-eye view plane to generate a target panoramic image.

2. The calibration device for a vehicle panoramic imaging system according to claim 1, wherein: The device further comprises: An iterative control module is connected to the lane line projection module and is used to control the lane line projection module to cycle a predetermined number of times. In each cycle, the optimal RT inverse projection matrix calculated in the previous cycle is used to update the initial RT inverse projection matrix, and the optimal RT inverse projection matrix calculated in the last cycle is output to the panoramic image generation module.

3. The calibration device for a vehicle panoramic imaging system according to claim 2, wherein: The device further comprises: A loop detection module is connected to the panoramic image generation module and the iterative control module, respectively, and is used to determine whether the parallelism and relative distance of each projected lane line in the target panoramic image meet preset calibration conditions. When the parallelism and relative distance do not meet the preset calibration conditions, the iterative control module is controlled to restart.

4. The calibration device for a vehicle panoramic imaging system according to claim 1, wherein: The device further comprises: A preprocessing module is connected to the image correction module and the lane line detection and judgment module respectively, and is used to preprocess the corrected road image and output it to the lane line detection and judgment module. The preprocessing includes at least: image enhancement, edge detection, contour processing, perspective transformation and pixel integration.

5. The calibration device for a vehicle panoramic imaging system according to claim 1, wherein: The lane line detection model is a least squares lane line detection fitting model.

6. A calibration method for a motor vehicle panoramic imaging system, characterized in that: The method comprises the following steps: An image extraction step, in response to the initial calibration instruction, extracts an original road image of the corresponding side from the road video images captured by the on-board cameras respectively arranged on each side of the motor vehicle; An image correction step, using a pre-stored image correction model to perform image correction on each original road image to obtain a corresponding corrected road image; A lane line detection and judgment step uses a pre-stored lane line detection model to detect original lane lines from each corrected road image, analyzes and judges the original lane lines detected in each corrected road image, and outputs a calibratable instruction when the original lane lines belonging to the same actual lane line exist in any two adjacent corrected road images in both directions; a lane line projection step, in response to the calibratable instruction, calculating projected lane lines generated by projecting the original lane lines in each of the corrected road images onto a preset panoramic bird's-eye view plane using a pre-stored initial RT inverse projection matrix, constructing a loss function for the planar positional relationship between two projected lane lines belonging to the same actual lane line based on an approximation algorithm model, and solving the loss function to obtain an optimal RT inverse projection matrix corresponding to each of the corrected road images when the loss function is minimized; and The panoramic image generation step is to respectively use the optimal RT inverse projection matrix to project each corrected road image onto the panoramic bird's-eye view plane to generate a target panoramic image.

7. The calibration method of a vehicle panoramic imaging system according to claim 6, wherein: The method further comprises: An iterative control step controls the lane line projection step to be looped for a predetermined number of times, wherein the optimal RT inverse projection matrix calculated in the previous cycle is used to update the initial RT inverse projection matrix in each cycle, and the optimal RT inverse projection matrix calculated in the last cycle is used to execute the panoramic image generation step.

8. The calibration method of a vehicle panoramic imaging system according to claim 7, wherein: The method also includes: a loop detection step, determining whether the parallelism and relative distance of each projected lane line in the target panoramic image meet preset calibration conditions, and re-executing the iterative control step when the parallelism and relative distance do not meet the preset calibration conditions.

9. The calibration method of a vehicle panoramic imaging system according to claim 6, wherein: The method further comprises: A preprocessing step is performed on the corrected road image before executing the lane line detection and judgment step. The preprocessing step includes at least: image enhancement, edge detection, contour processing, perspective transformation and pixel integration.

10. The calibration method of a vehicle panoramic imaging system according to claim 6, wherein: The lane line detection model is a least squares lane line detection fitting model.

Images