Fish-eye camera external parameter calibration method and device, storage medium and electronic equipment

By calculating the adaptive adjustment of the ROI area of ​​the homography matrix, the problem that the ROI area in the external parameter calibration of the fisheye camera cannot cover the target, and the calibration success rate and efficiency are improved.

CN119027507APending Publication Date: 2024-11-26GUANGZHOU AUTOMOBILE GROUP CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202310598431.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-24
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

During the external parameter calibration process of the on-board fisheye camera, the fixed setting of the ROI area causes the installation position and angle deviation, and the target cannot be covered, resulting in the calibration failure, and the feature point search time is extended and environmental interference is introduced.

Method used

By calculating the homography matrix adaptively adjusting the ROI area to ensure that it covers the target, thereby improving the success rate of external parameter calibration of the fisheye camera.

Benefits of technology

The calibration success rate is improved at different installation positions and angles, reducing environmental interference and improving calibration efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027507A_ABST
    Figure CN119027507A_ABST
Patent Text Reader

Abstract

The invention relates to a fisheye camera external parameter calibration method and device and a storage medium, and the method comprises the steps: obtaining a current target image obtained by photographing a target point through a fisheye camera, and obtaining the pixel coordinates of a plurality of feature points of the current target image; acquiring pixel coordinates of the plurality of feature points in the original target image, and calculating a homography matrix according to the pixel coordinates of the plurality of feature points in the original target image and the pixel coordinates of the plurality of feature points in the current target image; determining a current ROI region in the current target image according to an original ROI region in the original target image and the homography matrix; and performing external parameter calibration of the fisheye camera according to the current ROI area. According to the invention, the ROI can be adaptively adjusted, and the external parameter calibration success rate of the fisheye camera is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of vehicle-mounted sensing technology, and in particular to a fisheye camera external parameter calibration method and its device, storage medium, and electronic device. Background Art

[0002] In autonomous driving, surround fisheye cameras are often used for panoramic imaging and parking applications. After the fisheye camera is installed, it is an essential step to calibrate its internal and external parameters. Usually, when calibrating the external parameters of the on-board fisheye camera, the vehicle is parked in a fixed calibration room or a specific open area. In this area, calibration targets are distributed on the ground around the vehicle. By collecting photos of the targets in this area, feature points are extracted and external parameter calibration is performed.

[0003] Due to differences in vehicles and assemblies, the camera installation position and angle of each vehicle cannot be guaranteed to be completely consistent. Therefore, the position of the ground target in the camera is not the same. Usually, before calibration, in order to speed up the target retrieval and eliminate unnecessary environmental interference, the calibration algorithm will search for target points in the pre-set ROI (Region of Interest) area. If the camera installation position deviation is too large, the ROI area may not cover the target, resulting in calibration failure. Among them, the ROI area is usually set to the default area and cannot be adjusted dynamically. Therefore, if the ROI area is set too small, the deviation of the camera installation angle will most likely make the ROI area unable to cover the target point. If the ROI area is increased in order to reduce the impact of the camera installation angle on the target point, the time for feature point search will be extended, and more environmental interference factors will be introduced. Summary of the invention

[0004] The purpose of this application is to propose a fisheye camera external parameter calibration method and device, and a computer-readable storage medium, which can adaptively adjust the ROI area and improve the calibration success rate.

[0005] To achieve the above-mentioned purpose, the present application provides a method for calibrating external parameters of a fisheye camera, comprising:

[0006] Obtaining a fisheye camera to photograph a target point to obtain a current target image, and obtaining pixel coordinates of a plurality of feature points of the current target image;

[0007] Obtaining pixel coordinates of the plurality of feature points in the original target image, and calculating a homography matrix according to the pixel coordinates of the plurality of feature points in the original target image and the pixel coordinates of the plurality of feature points in the current target image;

[0008] Determine a current ROI region in the current target image according to the original ROI region in the original target image and the homography matrix;

[0009] The external parameters of the fisheye camera are calibrated according to the current ROI area.

[0010] In some embodiments, the plurality of feature points are respectively located in four regions, namely, the upper left, upper right, lower left, and lower right, of the current target image, and any three of the plurality of feature points are not collinear.

[0011] In some embodiments, determining the current ROI region in the current target image according to the original ROI region in the original target image and the homography matrix includes:

[0012] Obtain the pixel coordinates of the four corner points of the original ROI area in the original target image, calculate the pixel coordinates of the four corner points in the current target image according to the homography matrix and the pixel coordinates of the four corner points in the original target image, and determine the current ROI area in the current target image according to the pixel coordinates of the four corner points in the current target image.

[0013] In some embodiments, determining the current ROI region in the current target image according to the original ROI region in the original target image and the homography matrix includes:

[0014] The pixel coordinates of the target point in the original target image are obtained, the pixel coordinates of the target point in the current target image are calculated according to the homography matrix and the pixel coordinates of the target point in the original target image, and the current ROI area in the current target image is determined according to the pixel coordinates of the target point in the current target image and a preset ROI area size.

[0015] The present application also provides a fisheye camera external parameter calibration device, including:

[0016] An image processing module is used to obtain a current target image by photographing a target point with a fisheye camera, and obtain pixel coordinates of multiple feature points of the current target image;

[0017] A homography matrix acquisition module, used to acquire pixel coordinates of the plurality of feature points in the original target image, and calculate a homography matrix according to the pixel coordinates of the plurality of feature points in the original target image and the pixel coordinates of the plurality of feature points in the current target image;

[0018] An ROI region determination module, configured to determine a current ROI region in the current target image according to an original ROI region in the original target image and the homography matrix;

[0019] The external parameter calibration module is used to perform external parameter calibration of the fisheye camera according to the current ROI area.

[0020] In some embodiments, the plurality of feature points are respectively located in four regions, namely, the upper left, upper right, lower left, and lower right, of the current target image, and any three of the plurality of feature points are not collinear.

[0021] In some embodiments, the ROI region determination module is specifically used to:

[0022] Obtain the pixel coordinates of the four corner points of the original ROI area in the original target image, calculate the pixel coordinates of the four corner points in the current target image according to the homography matrix and the pixel coordinates of the four corner points in the original target image, and determine the current ROI area in the current target image according to the pixel coordinates of the four corner points in the current target image.

[0023] In some embodiments, the ROI region determination module is specifically used to:

[0024] The pixel coordinates of the target point in the original target image are obtained, the pixel coordinates of the target point in the current target image are calculated according to the homography matrix and the pixel coordinates of the target point in the original target image, and the current ROI area in the current target image is determined according to the pixel coordinates of the target point in the current target image and a preset ROI area size.

[0025] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the fisheye camera extrinsic parameter calibration method as described above is implemented.

[0026] An embodiment of the present application also provides an electronic device, characterized in that it includes a processor, a memory, and a computer program stored in the memory and executable on the processor, and when the processor executes the program, the fisheye camera extrinsic parameter calibration method as described above is implemented.

[0027] The embodiments of the present application provide a fisheye camera external parameter calibration device and its device, computer-readable storage medium, and electronic device. When calibrating the fisheye camera, the ROI area is adaptively adjusted according to the homography matrix so that the ROI area can correctly cover the target point, thereby improving the calibration success rate of the fisheye camera external parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0029] Figure 1 This is a flow chart of a fisheye camera extrinsic parameter calibration device in one embodiment of the present application.

[0030] Figure 2 This is a schematic diagram of a fisheye camera extrinsic parameter calibration device in another embodiment of the present application. DETAILED DESCRIPTION

[0031] The detailed description of the drawings is intended as an illustration of some embodiments of the present application, and is not intended to represent the only form in which the present application can be implemented. It should be understood that the same or equivalent functions can be accomplished by different embodiments intended to be included in the spirit and scope of the present application.

[0032] An embodiment of the present application provides a method for calibrating external parameters of a fisheye camera. Figure 1 , the method of this embodiment includes the following steps:

[0033] Step S100, obtaining a current target image by photographing a target point with a fisheye camera, and obtaining pixel coordinates of a plurality of feature points of the current target image.

[0034] Specifically, when calibrating a vehicle-mounted fisheye camera, the vehicle to be calibrated enters the calibration area, and the vehicle is moved so that the target point is at a preset position in the field of view of the fisheye camera. When the fisheye camera is started to shoot, the fisheye camera can capture the target point, and the current target image includes the target point. Specifically, in order to improve the calibration quality and success rate, the target point is multiple points.

[0035] Step S200, obtaining pixel coordinates of the plurality of feature points in the original target image, and calculating a homography matrix according to the pixel coordinates of the plurality of feature points in the original target image and the pixel coordinates of the plurality of feature points in the current target image.

[0036] Specifically, the original target image can be obtained by shooting the target point by other vehicles. During calibration, an original ROI area is determined according to the original target image, and the original ROI area covers all the target points. In step S200, the original target image and the current target image are matched with feature points to find the pixel coordinates of the multiple feature points in the original target image. There are many algorithms for feature point matching, such as SIFT feature point matching algorithm, SURF feature point matching algorithm, ORB feature point matching algorithm, etc., and this embodiment is not limited to any one of them.

[0037] Furthermore, according to the pixel coordinates of the multiple feature points in the two images (the original target image and the current target image), the homography matrix between the two images can be solved, as follows:

[0038] Let the homography matrix be:

[0039]

[0040] For any feature point in the pixel coordinates (x1, y1) and (x2, y2) of the two images, there is the following relationship:

[0041]

[0042]

[0043] Therefore, the four feature points can construct more than four sets of relationship equations, and the parameters in the homography matrix can be obtained by solving the four sets of relationship equations.

[0044] Step S300 : determining a current ROI region in the current target image according to the original ROI region in the original target image and the homography matrix.

[0045] Specifically, the homography matrix describes the geometric transformation relationship between the original target image and the current target image. Therefore, the position and size of the original ROI area in the current target image can be determined according to the homography matrix to obtain the current ROI area.

[0046] Step S400, calibrating the external parameters of the fisheye camera according to the current ROI area.

[0047] Specifically, the current ROI area obtained in step S300 can correctly cover all target points, avoiding the situation where the ROI area cannot cover the target due to large deviation in the installation position of the fisheye camera; after obtaining the current ROI area, the current ROI area can be processed using a preset calibration algorithm to obtain a calibration result; the main principle of the calibration algorithm is to calculate the external parameters of the fisheye camera based on the pixel coordinates of the target point in the image and the coordinates of the target point in the world coordinate system / vehicle coordinate system.

[0048] In some embodiments, the plurality of feature points are respectively located in the upper left, upper right, lower left, and lower right regions of the current target image, and any three of the plurality of feature points are not collinear.

[0049] Specifically, the installation positions of fisheye cameras of different vehicles may deviate, but the deviation will not be particularly large. Therefore, a feature point can be taken from each of the upper left, upper right, lower left, and lower right areas of the current target image, and the taken feature points can also find matching points in the original target image.

[0050] In some embodiments, the step S300 specifically includes:

[0051] Obtain the pixel coordinates of the four corner points of the original ROI area in the original target image, calculate the pixel coordinates of the four corner points in the current target image according to the homography matrix and the pixel coordinates of the four corner points in the original target image, and determine the current ROI area in the current target image according to the pixel coordinates of the four corner points in the current target image.

[0052] Specifically, the size and position of the original ROI area can be determined according to the four corner points of the original ROI area. Therefore, when the four corner points of the original ROI area are transformed and calculated through the homography matrix to obtain the pixel coordinates of the four corner points in the current target image, the ROI area in the current target image can be determined. Since the original ROI area covers all target points, the current ROI area should also cover all target points.

[0053] In some embodiments, the step S300 specifically includes:

[0054] The pixel coordinates of the target point in the original target image are obtained, the pixel coordinates of the target point in the current target image are calculated according to the homography matrix and the pixel coordinates of the target point in the original target image, and the current ROI area in the current target image is determined according to the pixel coordinates of the target point in the current target image and a preset ROI area size.

[0055] Specifically, in the original target image, the target point is located in the middle of the original ROI area as much as possible, so that the fisheye cameras of other subsequent vehicles can also capture the target point. Therefore, in step S300, the pixel coordinates of at least one target point in the original target image and the homography matrix can be used for transformation to determine the pixel coordinates of the at least one target point in the current target image, and the relationship between the at least one target point and the original ROI area is determined, and the size of the ROI area is determined, that is, a current ROI area can be automatically generated, so that the current position of the at least one target point in the current ROI area is the same as the original position of the at least one target point in the original ROI area, so that the current ROI area can also cover all target points.

[0056] Corresponding to the method of the above embodiment, another embodiment of the present application further provides a fisheye camera external parameter calibration device, including a module that can be used to execute the steps of the method of the above embodiment, and a method for calibrating the external parameters of the fisheye camera. Figure 2 , the module includes:

[0057] Image processing module 1, used to obtain a current target image by photographing a target point with a fisheye camera, and obtain pixel coordinates of multiple feature points of the current target image;

[0058] A homography matrix acquisition module 2 is used to acquire the pixel coordinates of the multiple feature points in the original target image, and calculate the homography matrix according to the pixel coordinates of the multiple feature points in the original target image and the pixel coordinates of the multiple feature points in the current target image;

[0059] An ROI region determination module 3 is used to determine a current ROI region in the current target image according to the original ROI region in the original target image and the homography matrix;

[0060] The external parameter calibration module 4 is used to perform external parameter calibration of the fisheye camera according to the current ROI area.

[0061] In some embodiments, the plurality of feature points are respectively located in the upper left, upper right, lower left, and lower right regions of the current target image, and any three of the plurality of feature points are not collinear.

[0062] In some embodiments, the ROI region determination module 3 is specifically used to:

[0063] Obtain the pixel coordinates of the four corner points of the original ROI area in the original target image, calculate the pixel coordinates of the four corner points in the current target image according to the homography matrix and the pixel coordinates of the four corner points in the original target image, and determine the current ROI area in the current target image according to the pixel coordinates of the four corner points in the current target image.

[0064] In some embodiments, the ROI region determination module 3 is specifically used to:

[0065] The pixel coordinates of the target point in the original target image are obtained, the pixel coordinates of the target point in the current target image are calculated according to the homography matrix and the pixel coordinates of the target point in the original target image, and the current ROI area in the current target image is determined according to the pixel coordinates of the target point in the current target image and a preset ROI area size.

[0066] The fisheye camera external parameter calibration device of the above-described embodiment is merely illustrative, wherein the modules described as separate components may or may not be physically separated, and the components of the modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the fisheye camera external parameter calibration device of the embodiment.

[0067] It should be noted that the fisheye camera external parameter calibration device of the above-mentioned embodiment corresponds to the fisheye camera external parameter calibration method of the above-mentioned embodiment. Therefore, the part of the fisheye camera external parameter calibration device of the above-mentioned embodiment that is not described in detail can be obtained by referring to the content of the fisheye camera external parameter calibration method of the above-mentioned embodiment, that is, the specific steps recorded in the fisheye camera external parameter calibration method of the above-mentioned embodiment can be understood as the functions that can be achieved by the fisheye camera external parameter calibration device of the above-mentioned embodiment, and will not be repeated here.

[0068] Furthermore, if the fisheye camera external parameter calibration device of the above embodiment is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0069] Another embodiment of the present application further proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the fisheye camera extrinsic parameter calibration method as described in the above embodiment is implemented.

[0070] Specifically, the computer-readable storage medium may include: any entity or recording medium that can carry the computer program instructions, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0071] Another embodiment of the present application provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the fisheye camera extrinsic parameter calibration method described in the above embodiment when executing the program.

[0072] Wherein, the electronic device may also include a bus connecting different components (including memory and processor). The memory may include a computer-readable medium in the form of a volatile memory, such as a random access memory (RAM) and / or a cache memory. The memory may also include at least one program product, which has a set of (e.g., at least one) program modules, which are configured to perform the functions of the various embodiments of the present application. The electronic device may also communicate with one or more external devices (e.g., keyboards, pointing devices, displays, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device, and / or communicate with any device (e.g., a network card) that enables the electronic device to communicate with one or more other computing devices, such communication may be performed through an input / output (I / O) interface, and the electronic device may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs) and / or public networks, such as the Internet) through a network adapter.

[0073] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for calibrating external parameters of a fisheye camera, characterized in that: include: Obtaining a fisheye camera to photograph a target point to obtain a current target image, and obtaining pixel coordinates of a plurality of feature points of the current target image; Obtaining pixel coordinates of the plurality of feature points in the original target image, and calculating a homography matrix according to the pixel coordinates of the plurality of feature points in the original target image and the pixel coordinates of the plurality of feature points in the current target image; Determine a current ROI region in the current target image according to the original ROI region in the original target image and the homography matrix; The external parameters of the fisheye camera are calibrated according to the current ROI area.

2. The method according to claim 1, characterized in that The plurality of feature points are respectively located in the upper left, upper right, lower left and lower right regions of the current target image, and any three of the plurality of feature points are not collinear.

3. The method according to claim 1, characterized in that: The determining the current ROI area in the current target image according to the original ROI area in the original target image and the homography matrix includes: Obtain the pixel coordinates of the four corner points of the original ROI area in the original target image, calculate the pixel coordinates of the four corner points in the current target image according to the homography matrix and the pixel coordinates of the four corner points in the original target image, and determine the current ROI area in the current target image according to the pixel coordinates of the four corner points in the current target image.

4. The method according to claim 1, characterized in that: The determining the current ROI area in the current target image according to the original ROI area in the original target image and the homography matrix includes: The pixel coordinates of the target point in the original target image are obtained, the pixel coordinates of the target point in the current target image are calculated according to the homography matrix and the pixel coordinates of the target point in the original target image, and the current ROI area in the current target image is determined according to the pixel coordinates of the target point in the current target image and a preset ROI area size.

5. A fisheye camera external parameter calibration device, characterized in that: include: An image processing module is used to obtain a current target image by photographing a target point with a fisheye camera, and obtain pixel coordinates of multiple feature points of the current target image; A homography matrix acquisition module, used to acquire pixel coordinates of the plurality of feature points in the original target image, and calculate a homography matrix according to the pixel coordinates of the plurality of feature points in the original target image and the pixel coordinates of the plurality of feature points in the current target image; An ROI region determination module, configured to determine a current ROI region in the current target image according to an original ROI region in the original target image and the homography matrix; The external parameter calibration module is used to perform external parameter calibration of the fisheye camera according to the current ROI area.

6. The device according to claim 5, characterized in that The plurality of feature points are respectively located in four regions, namely, the upper left, upper right, lower left, and lower right of the current target image, and any three of the plurality of feature points are not collinear.

7. The device according to claim 5, characterized in that The ROI region determination module is specifically used for: Obtain the pixel coordinates of the four corner points of the original ROI area in the original target image, calculate the pixel coordinates of the four corner points in the current target image according to the homography matrix and the pixel coordinates of the four corner points in the original target image, and determine the current ROI area in the current target image according to the pixel coordinates of the four corner points in the current target image.

8. The device according to claim 5, characterized in that The ROI region determination module is specifically used for: The pixel coordinates of the target point in the original target image are obtained, the pixel coordinates of the target point in the current target image are calculated according to the homography matrix and the pixel coordinates of the target point in the original target image, and the current ROI area in the current target image is determined according to the pixel coordinates of the target point in the current target image and a preset ROI area size.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the fisheye camera extrinsic parameter calibration method according to any one of claims 1 to 4 is implemented.

10. An electronic device, characterized in that: The invention comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the fisheye camera extrinsic parameter calibration method according to any one of claims 1 to 4 is implemented.

Citation Information

Cited By

  • Camera external parameter calibration method, device and equipment and storage medium

    CN120259446A