A panoramic parking calibration method, device and storage medium

By calibrating the center point and the four corner points, the relationship lookup table between the fisheye source map and the panoramic image is solved, and the problem of complex and time-consuming panoramic parking calibration in the existing technology is achieved, and a fast and simple panoramic parking calibration process is realized.

CN111986270BActive Publication Date: 2024-11-12SHENZHEN APICAL TECH CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202010869752.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-08-26
Publication Date
2024-11-12
Estimated Expiration
2040-08-26

AI Technical Summary

Technical Problem

The existing panoramic parking calibration methods are complex and time-consuming, making it difficult to achieve a fast and simple calibration process.

Method used

Four fisheye cameras are used to collect the image of the chessboard calibration plate around the vehicle body, and the image relationship lookup table between the fisheye source map and the panoramic image is generated by calibrating the center point and the four predetermined corner point coordinates. The homography matrix and double lines are used. Sexual interpolation method generates panoramic images.

Benefits of technology

It realizes the simplicity, speed and high accuracy of panoramic parking calibration, reduces calibration time and manual intervention, and improves the ease of use of user operations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111986270B_ABST
    Figure CN111986270B_ABST
Patent Text Reader

Abstract

The panoramic parking calibration method, device and storage medium provided by the present invention include: controlling the camera to collect the fisheye source image of the corresponding area, the camera is four cameras installed on the front, back, left and right of the target vehicle, and the fisheye source image includes a chessboard calibration plate laid around the body of the target vehicle; calibrating the center point coordinates of each of the fisheye source images, and calibrating the four predetermined corner point coordinates of the chessboard calibration plate in each of the fisheye source images; generating an image relationship lookup table between the coordinate points of the fisheye source image and the coordinate points of the panoramic image according to the center point coordinates, the four predetermined corner point coordinates and the actual position parameters input in advance, and generating a panoramic image according to the image relationship lookup table. The present invention can generate a panoramic image by calibrating the center point and the four corner points, and obtaining the image relationship lookup table, which is simple, fast and time-saving.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of electronic technology, and in particular to a panoramic parking calibration method, device and storage medium. Background Art

[0002] For a long time, when drivers park their vehicles in parking spaces, they can only rely on rearview mirrors to roughly observe the environment around the vehicle body, or rely on a single rearview camera to observe obstacles behind the vehicle body, which results in a large visual blind spot. In order to solve the parking safety problem, an ultra-wide-angle fisheye camera is now installed on the front, rear, left and right sides of the vehicle body. The acquired images are used to synthesize a synthetic bird's-eye view looking down from the sky. The driver can see the real-time image around the vehicle body through the bird's-eye view on the display screen in the car, which can clearly show the relative position of obstacles and the vehicle body. In this way, how to synthesize a bird's-eye view with no blind spots and good splicing from four ultra-wide-angle fisheye cameras is very critical. This process is called calibration. The current calibration method for panoramic parking is to use the image coordinates of the corner points of the black and white chessboard calibration board to calculate the spatial three-dimensional coordinates of all corner points, thereby determining the position characteristics of the fisheye lens. The calibration process is complex and time-consuming.

[0003] Therefore, the prior art has defects and needs to be improved and developed. Summary of the invention

[0004] The technical problem to be solved by the present invention is that, in view of the above-mentioned defects of the prior art, a panoramic parking calibration method, device and storage medium are provided, aiming to solve the problem that the panoramic parking calibration method used in the prior art has a complex calibration process and is time-consuming.

[0005] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0006] A panoramic parking calibration method, comprising:

[0007] Controlling the camera to collect a fisheye source image of a corresponding area, wherein the camera is four cameras installed on the front, back, left and right sides of the target vehicle, and the fisheye source image includes a chessboard calibration plate laid around the body of the target vehicle;

[0008] Calibrate the coordinates of the center point of each of the fisheye source images, and calibrate the coordinates of four predetermined corner points of the chessboard calibration plate in each of the fisheye source images;

[0009] An image relationship lookup table between the coordinate points of the fisheye source image and the coordinate points of the panoramic image is generated according to the center point coordinates, the four predetermined corner point coordinates and the actual position parameters input in advance, and a panoramic image is generated according to the image relationship lookup table.

[0010] In a further implementation, the step of generating an image relationship lookup table between the coordinate points of the fisheye source image and the coordinate points of the panoramic image according to the center point coordinates, the coordinates of the four predetermined corner points and the actual position parameters input in advance specifically includes:

[0011] Performing distortion correction on the center point coordinates and the four predetermined corner point coordinates, and calculating the homography matrix of each fisheye source image according to the actual position parameters input in advance;

[0012] Generate a front image relationship lookup table between the coordinate points of the front fisheye source image and the coordinate points of the panoramic image according to the homography matrix of the fisheye source image captured by the front camera;

[0013] A rear image relationship lookup table between coordinate points of the rear fisheye source image and coordinate points of the panoramic image is generated according to the homography matrix of the fisheye source image captured by the rear camera.

[0014] In a further implementation, the step of calculating the homography matrix of each fisheye source image further includes:

[0015] Generate a left image relationship lookup table between left fisheye source image coordinate points and panoramic image coordinate points according to the homography matrix of the fisheye source image captured by the left camera;

[0016] A right image relationship lookup table between coordinate points of the right fisheye source image and coordinate points of the panoramic image is generated according to the homography matrix of the fisheye source image taken by the right camera.

[0017] In a further implementation, the step of generating a panoramic image according to the image relationship lookup table specifically includes:

[0018] The image components at each fisheye source image coordinate are copied to the panoramic image coordinate according to the image relationship lookup table to generate a panoramic image.

[0019] In a further implementation, the step of calculating the homography matrix of each fisheye source image specifically includes:

[0020] Calculate the projection radius according to the coordinates of the center point of each fisheye source image and the coordinates of four predetermined corner points;

[0021] Obtaining an incident angle corresponding to the projection radius according to a table of relationships between incident angles and projection radius of a camera;

[0022] Calculate the radius of the corresponding corrected image according to the focal length and the incident angle, and calculate the coordinates of four predetermined corner points on the corrected image;

[0023] The homography matrix is ​​calculated based on the relationship between the coordinates of the four predetermined corner points and the coordinate points on the rectified image.

[0024] In a further implementation, the actual position parameters include: a calibrated length, a calibrated width, and an offset position.

[0025] In a further implementation, the chessboard calibration plate is a black and white chessboard calibration plate, and the four predetermined corner point coordinates are the upper left corner coordinate, the upper right corner coordinate, the lower left corner coordinate and the lower right corner coordinate of the black and white chessboard calibration plate.

[0026] In a further implementation, the step of copying the image components at each fisheye source image coordinate to the panoramic image coordinate according to the image relationship lookup table, and generating the panoramic image further includes:

[0027] When the position coordinates of the fisheye source image and the panoramic image are not integer vectors, the bilinear interpolation method is used to obtain the image component information.

[0028] The present invention also provides a device, which includes a memory, a processor, and a panoramic parking calibration program stored in the memory and executable on the processor, wherein the panoramic parking calibration program implements the steps of the panoramic parking calibration method described above when executed by the processor.

[0029] The present invention also provides a storage medium, wherein the storage medium stores a computer program, and the computer program can be executed to implement the steps of the panoramic parking calibration method as described above.

[0030] The panoramic parking calibration method, device and storage medium provided by the present invention include: controlling the camera to collect the fisheye source image of the corresponding area, the camera is four cameras installed on the front, back, left and right of the target vehicle, and the fisheye source image includes a chessboard calibration plate laid around the body of the target vehicle; calibrating the center point coordinates of each of the fisheye source images, and calibrating the four predetermined corner point coordinates of the chessboard calibration plate in each of the fisheye source images; generating an image relationship lookup table between the coordinate points of the fisheye source image and the coordinate points of the panoramic image according to the center point coordinates, the four predetermined corner point coordinates and the actual position parameters input in advance, and generating a panoramic image according to the image relationship lookup table. The present invention can generate a panoramic image by calibrating the center point and the four corner points, and obtaining the image relationship lookup table, which is simple, fast and time-saving. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a flow chart of a preferred embodiment of the panoramic parking calibration method in the present invention.

[0032] Figure 2 It is a schematic diagram of the position of the chessboard calibration plate in the preferred embodiment of the panoramic parking calibration method of the present invention.

[0033] Figure 3It is a functional principle block diagram of a preferred embodiment of the device in the present invention. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0035] Currently, panoramic parking calibration uses the image coordinates of the corner points of a black and white chessboard calibration plate to calculate the spatial three-dimensional coordinates of all corner points, thereby determining the position characteristics (offset, radius) of the fisheye lens. The calibration process is complex and time-consuming, and professional training is required to complete the calibration.

[0036] In order to promote the use of panoramic parking on a large scale, the present invention provides a calibration method that is simple, fast, highly accurate and easy to operate.

[0037] See also Figure 1 , Figure 1 : is a flow chart of the panoramic parking calibration method in the present invention. Figure 1 As shown, the panoramic parking calibration method described in the embodiment of the present invention includes the following steps:

[0038] S100, controlling cameras to collect a fisheye source image of a corresponding area, wherein the cameras are four cameras installed on the target vehicle, the front, rear, left, and right, and the fisheye source image includes a chessboard calibration plate laid around the body of the target vehicle.

[0039] Specifically, the chessboard calibration plate is a black and white chessboard calibration plate. The present invention performs calibration through a vehicle-mounted device, and the vehicle-mounted device can be a four-channel vehicle-mounted video recorder. Before calibration, a black and white chessboard calibration plate is prepared and laid around the target vehicle body. The four cameras in the front, back, left and right are all fisheye cameras.

[0040] S200, calibrating the coordinates of the center point of each of the fisheye source images, and calibrating the coordinates of four predetermined corner points of the chessboard calibration plate in each of the fisheye source images.

[0041] Specifically, the four predetermined corner point coordinates can be the upper left corner coordinate, upper right corner coordinate, lower left corner coordinate and lower right corner coordinate of the black and white chessboard calibration plate. The center points of the four fisheye lenses are calibrated one by one through human-computer interaction, and the four corner point coordinates on the four fisheye source images are calibrated respectively.

[0042] S300, generating an image relationship lookup table between fisheye source image coordinate points and panoramic image coordinate points according to the center point coordinates, four predetermined corner point coordinates and pre-input actual position parameters, and generating a panoramic image according to the image relationship lookup table.

[0043] The actual position parameters include: calibrated length, calibrated width and offset position; see Figure 2 , the calibration length is the vehicle length L, the calibration width is the vehicle width D, and the offset position refers to the left and right offset distance D. Among them, when the black and white chessboard calibration plate is placed, the calibration plate in the front and rear directions is placed in the central position of the vehicle width, that is, A1=A2, B1=B2, and the positions of the calibration plates in the left and right directions are corresponding, that is, the offset position D1=D2. In other words, after identifying the coordinates of the center point and the four corner points, and then according to the actual position parameters such as the calibration length, calibration width and offset position, the software can make corrections based on this information, generate an image relationship lookup table between the coordinate points of the fisheye source image and the coordinate points of the panoramic image, and store it in the hard disk. In this way, the present invention only needs to calibrate the center point and the four corner points, and after obtaining the image relationship lookup table, a panoramic image can be generated without calculating all the corner points. It is simple, fast, and time-saving. This operation method is simple and fast. The user only needs to use the remote control and the predetermined calibration plate to quickly complete the calibration. The calibration time is short and easy to understand.

[0044] In one implementation, the step S300 of “generating an image relationship lookup table between the coordinate points of the fisheye source image and the coordinate points of the panoramic image according to the coordinates of the center point, the coordinates of the four predetermined corner points, and the actual position parameters input in advance” specifically includes:

[0045] S310, performing distortion correction on the center point coordinates and the four predetermined corner point coordinates, and calculating the homography matrix of each fisheye source image according to the actual position parameters input in advance;

[0046] S320, generating a front image relationship lookup table between the coordinate points of the front fisheye source image and the coordinate points of the panoramic image according to the homography matrix of the fisheye source image captured by the front camera;

[0047] S330, generating a rear image relationship lookup table between coordinate points of the rear fisheye source image and coordinate points of the panoramic image according to the homography matrix of the fisheye source image captured by the rear camera.

[0048] S340, generating a left image relationship lookup table between left fisheye source image coordinate points and panoramic image coordinate points according to the homography matrix of the fisheye source image captured by the left camera;

[0049] S350, generating a right image relationship lookup table between coordinate points of the right fisheye source image and coordinate points of the panoramic image according to the homography matrix of the fisheye source image taken by the right camera.

[0050] That is to say, the mapping relationship between the target image and the source image is established using the homography matrix. Since the homography matrix has 8 unknowns, 4 known numbers are required to correspond to the four corner points on the fisheye source image. Distortion correction is performed on the image calibration points in the front, rear, left, and right sides respectively, so as to calculate the homography matrix of each fisheye source image, and generate the front image relationship lookup table, rear image relationship lookup table, left image relationship lookup table, and right image relationship lookup table. Specifically, it can be a left and front 10*10 image lookup table, a right and front 10*10 image lookup table, a left and back 10*10 image lookup table, and a right and back 10*10 image lookup table.

[0051] In one implementation, the step of “generating a panoramic image according to the image relationship lookup table” in step S300 specifically includes:

[0052] S360 , copying the image components at each fisheye source image coordinate to the panoramic image coordinate according to the image relationship lookup table to generate a panoramic image.

[0053] Specifically, when the position coordinates of the fisheye source image and the panoramic image are not integer vectors, the bilinear interpolation method is used to obtain the image component information. Generating a panoramic image is a panoramic stitching process. First, the image relationship lookup table generated by the calibration process is loaded from the hard disk, and the corresponding fisheye source image coordinates are obtained by mapping the coordinates of the panoramic image. The image components at the fisheye source image coordinates are copied to the panoramic image coordinates. Since the position coordinates of the image before and after the transformation are generally not integer vectors, for decimal coordinates, the bilinear interpolation method must be used to obtain the image component information. Component copying means that the image is in YUV420 format. When copying, the Y component is copied first, and then the U and V components are copied.

[0054] In a further implementation, the step of "calculating the homography matrix of each fisheye source image" in step S310 specifically includes:

[0055] S311, calculating the projection radius according to the coordinates of the center point of each fisheye source image and the coordinates of four predetermined corner points;

[0056] S312, obtaining an incident angle corresponding to the projection radius according to a table of relationships between incident angles and projection radius of a camera;

[0057] S313, calculating the radius of the corresponding corrected image according to the focal length and the incident angle, and calculating the coordinates of four predetermined corner points on the corrected image;

[0058] S314, calculating a homography matrix according to the relationship between the coordinates of the four predetermined corner points and the coordinate points on the corrected image.

[0059] That is to say, the projection radius is calculated from the known coordinates of the four calibration corner points and the center coordinates. The relationship table between the incident angle and the projection radius can be provided by the camera manufacturer. The corresponding incident angle is obtained according to the relationship table. The radius of the corrected image corresponding to the calibration corner point coordinates is calculated according to the focal length and the incident angle based on the pinhole imaging principle. The coordinates of the four calibration points in the corrected image are calculated, and the homography matrix is ​​calculated using the above four pairs of coordinate points.

[0060] In this way, the homography matrix is ​​calculated through the four corner points to obtain the mapping transformation relationship and generate a lookup table method from the target image to the original image. The method of automatically extracting corner point information can be used instead of manually calibrating corner points, reducing human intervention.

[0061] The present invention also provides a device, see Figure 3 The device may be a four-channel vehicle-mounted video recorder, comprising a memory 20, a processor 10, and a panoramic parking calibration program stored in the memory 20 and executable on the processor 10, wherein the panoramic parking calibration program implements the steps of the panoramic parking calibration method as described above when executed by the processor 10; specifically as described above.

[0062] The present invention also provides a storage medium, wherein the storage medium stores a computer program, and the computer program can be executed to implement the steps of the panoramic parking calibration method as described above; specifically as described above.

[0063] In summary, the present invention discloses a panoramic parking calibration method, device and storage medium, including: controlling the camera to collect the fisheye source image of the corresponding area, the camera is the front, rear, left and right four cameras installed on the target vehicle, and the fisheye source image includes a chessboard calibration plate laid around the body of the target vehicle; calibrating the center point coordinates of each of the fisheye source images, and calibrating the four predetermined corner point coordinates of the chessboard calibration plate in each of the fisheye source images; generating an image relationship lookup table between the coordinate points of the fisheye source image and the coordinate points of the panoramic image according to the center point coordinates, the four predetermined corner point coordinates and the actual position parameters input in advance, and generating a panoramic image according to the image relationship lookup table. The present invention can generate a panoramic image by calibrating the center point and the four corner points and obtaining the image relationship lookup table, which is simple, fast and time-saving.

[0064] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A panoramic parking calibration method, characterized in that: include: Controlling the camera to collect the fisheye source image of the corresponding area, the camera is four cameras installed on the front, back, left and right of the target vehicle, and the fisheye source image includes a chessboard calibration plate laid around the body of the target vehicle; Calibrate the coordinates of the center point of each of the fisheye source images, and calibrate the coordinates of four predetermined corner points of the chessboard calibration plate in each of the fisheye source images; Generate an image relationship lookup table between the coordinate points of the fisheye source image and the coordinate points of the panoramic image according to the coordinates of the center point, the coordinates of the four predetermined corner points and the actual position parameters input in advance, and generate a panoramic image according to the image relationship lookup table; The step of generating a panoramic image according to the image relationship lookup table specifically includes: Copying the image components at each fisheye source image coordinate to the panoramic image coordinate according to the image relationship lookup table to generate a panoramic image; The steps for calculating the homography matrix of each fisheye source image specifically include: Calculate the projection radius according to the coordinates of the center point of each fisheye source image and the coordinates of four predetermined corner points; Obtaining an incident angle corresponding to the projection radius according to a table of relationships between incident angles and projection radius of a camera; Calculate the radius of the corresponding corrected image according to the focal length and the incident angle, and calculate the coordinates of four predetermined corner points on the corrected image; Calculate the homography matrix based on the relationship between the coordinates of the four predetermined corner points and the coordinate points on the corrected image; The step of copying the image components at each fisheye source image coordinate to the panoramic image coordinate according to the image relationship lookup table, and generating the panoramic image further comprises: When the position coordinates of the fisheye source image and the panoramic image are not integer vectors, the bilinear interpolation method is used to obtain the image component information; Among them, the radius of the corrected image corresponding to the coordinates of the calibration corner point is calculated according to the focal length and the incident angle based on the pinhole imaging principle; The step of generating an image relationship lookup table between the coordinate points of the fisheye source image and the coordinate points of the panoramic image according to the coordinates of the center point, the coordinates of the four predetermined corner points and the actual position parameters input in advance specifically comprises: Performing distortion correction on the center point coordinates and the four predetermined corner point coordinates, and calculating the homography matrix of each fisheye source image according to the actual position parameters input in advance; Generate a front image relationship lookup table between the coordinate points of the front fisheye source image and the coordinate points of the panoramic image according to the homography matrix of the fisheye source image captured by the front camera; Generate a rear image relationship lookup table between coordinate points of the rear fisheye source image and coordinate points of the panoramic image according to the homography matrix of the fisheye source image captured by the rear camera; The step of calculating the homography matrix of each fisheye source image further includes: Generate a left image relationship lookup table between left fisheye source image coordinate points and panoramic image coordinate points according to the homography matrix of the fisheye source image captured by the left camera; Generate a right image relationship lookup table between the coordinate points of the right fisheye source image and the coordinate points of the panoramic image according to the homography matrix of the fisheye source image taken by the right camera; The actual position parameters include: calibrated length, calibrated width and offset position; The chessboard calibration plate is a black and white chessboard calibration plate, and the four predetermined corner point coordinates are the upper left corner coordinate, the upper right corner coordinate, the lower left corner coordinate and the lower right corner coordinate of the black and white chessboard calibration plate; The image relationship lookup table generated by the calibration process is loaded from the hard disk. The coordinates of the panoramic image are mapped and transformed to obtain the corresponding fisheye source image coordinates. The image components at the fisheye source image coordinates are copied to the panoramic image coordinates. When copying components, the Y component of the panoramic image in YUV420 format is copied first, and then the U component and V component are copied.

2. A device, characterized in that: The invention comprises a memory, a processor and a panoramic parking calibration program stored in the memory and executable on the processor, wherein the panoramic parking calibration program implements the steps of the panoramic parking calibration method as claimed in claim 1 when executed by the processor.

3. A storage medium, characterized in that: The storage medium stores a computer program, which can be executed to implement the steps of the panoramic parking calibration method as claimed in claim 1.

Citation Information

Patent Citations

  • Calibration method, calibration system, correction method, correction system and vehicle

    CN109816732A

  • Lens calibration method of vehicle-mounted monocular panoramic system and related device

    CN110264395A

  • Panoramic parking calibration method and calibration system

    CN110766762A

  • Image calibrating, stitching and depth rebuilding method of panoramic fish-eye camera and system thereof

    JP2017112602A