Vehicle-mounted around-view camera calibration method and device and storage medium
By using multiple targets to automatically verify calibration errors in fisheye camera calibration, the problem of calibration error verification in the prior art has been solved, the accuracy and efficiency of calibration results have been improved, and the surrounding view of autonomous driving vehicles has been improved Splicing and parking space recognition capabilities.
Patent Information
- Application Number
- CN202310513863.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-08
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2043-05-08
AI Technical Summary
The existing fisheye camera calibration method cannot automatically verify the calibration error, resulting in the calibration results that do not meet the requirements and require manual inspection, which takes a long time, affecting the surround view splicing and parking space recognition accuracy of autonomous driving vehicles.
Using a combination of multiple targets (calibration target and calibration target) methods, the feature points in the image taken by the fisheye camera are extracted and calculated external parameters are automatically verified to determine whether the camera has passed the calibration and avoid manual inspection.
Automatic calibration error verification is realized, which reduces the time-consuming process, improves the accuracy and efficiency of calibration results, and improves the vehicle's surround view splicing and parking space recognition capabilities.
Smart Images

Figure CN118918193A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of vehicle perception technology, and in particular to a vehicle-mounted surround view camera calibration method and device, and storage medium. Background Art
[0002] In autonomous driving, cameras are indispensable sensing components. Among them, fisheye cameras are usually used in panoramic imaging and parking applications because of their large FOV field of view. When applying fisheye cameras, the calibration of internal and external parameters is an inevitable and extremely important step. If accurate internal and external parameter calibration results are obtained, the surround stitching and parking space recognition of the vehicle will be greatly improved. In the field of external parameter calibration, the current calibration method is to use a single target style, aim the fisheye camera at the target to shoot a fisheye image, and call the existing opencv open source algorithm to extract feature points and calculate external parameters of the fisheye image. However, the current calibration method mainly focuses on whether the calibration can be successful. It is impossible to perform automatic calibration error verification. Calibration results that do not meet the requirements cannot be automatically eliminated, and manual inspection is required, which is time-consuming. Summary of the invention
[0003] The purpose of this application is to propose a vehicle-mounted surround view camera calibration method and device, and a computer-readable storage medium to perform automatic calibration error verification.
[0004] To achieve the above-mentioned purpose, an embodiment of the present application provides a method for calibrating a vehicle-mounted surround view camera, wherein the vehicle-mounted surround view camera includes a plurality of fisheye cameras;
[0005] The method comprises:
[0006] Acquire a fisheye image captured by the fisheye camera; wherein a calibration target is arranged at a preset position in the field of view of the fisheye camera, and correction targets are distributed around the calibration target;
[0007] Identify a calibration target in the fisheye image, extract feature points of the calibration target to obtain calibration target feature points, and obtain pixel coordinates of the calibration target feature points;
[0008] Acquire the world coordinates of the calibration target feature points, and calculate the external parameters of the fisheye camera according to the pixel coordinates and world coordinates of the calibration target feature points;
[0009] Identify a calibration target in the fisheye image, extract calibration target feature points on the calibration target, and obtain first pixel coordinates of the calibration target feature points;
[0010] Acquire the world coordinates of the calibration target feature point, and calculate the second pixel coordinates of the calibration target feature point according to the world coordinates of the calibration target feature point and the external parameter;
[0011] Determine whether the fisheye camera passes calibration according to the difference between the first pixel coordinate and the second pixel coordinate.
[0012] According to the above method, in some embodiments, the calculating the second pixel coordinate of the calibration target feature point according to the world coordinate of the calibration target feature point and the external parameter includes:
[0013] Calculating the coordinates of the calibration target feature point in the camera coordinate system according to the world coordinates of the calibration target feature point and the external parameter;
[0014] The internal parameters of the fisheye camera are obtained, and the coordinates of the calibration target feature points in the camera coordinate system are converted into corresponding pixel coordinates according to the internal parameters, and the pixel coordinates are dedistorted according to the internal parameters to obtain second pixel coordinates.
[0015] According to the above method, in some embodiments, the calibration target includes a grid target and a circular target, and the calibration target feature point includes a plurality of feature points, and the plurality of feature points include a center point of the grid target and a center point of the circular target;
[0016] The calibration target includes a chessboard target, and the calibration target feature points include a plurality of corner points of the chessboard target.
[0017] According to the above method, in some implementations, determining whether the fisheye camera passes calibration according to the difference between the first pixel coordinate and the second pixel coordinate includes:
[0018] The difference between the first pixel coordinate and the second pixel coordinate of each corner point is calculated respectively to obtain a plurality of difference values; and an average value of the plurality of difference values is calculated, and whether the fisheye camera passes the calibration is determined according to a comparison result between the average value and a preset threshold value.
[0019] According to the above method, in some embodiments, it also includes:
[0020] When the multiple fisheye cameras are all calibrated, the fisheye images taken by the multiple fisheye cameras are subjected to dedistortion processing to obtain multiple dedistorted fisheye images, and the multiple fisheye images are stitched together to obtain a panoramic image.
[0021] The embodiment of the present application also provides a vehicle-mounted surround view camera calibration device, wherein the vehicle-mounted surround view camera includes a plurality of fisheye cameras;
[0022] The device comprises:
[0023] An image acquisition module, used to acquire the fisheye image taken by the fisheye camera; wherein a calibration target is arranged at a preset position in the field of view of the fisheye camera, and correction targets are distributed around the calibration target;
[0024] A first feature point extraction module is used to identify a calibration target in the fisheye image, extract feature points of the calibration target to obtain calibration target feature points, and obtain pixel coordinates of the calibration target feature points;
[0025] An external parameter calculation module, used to obtain the world coordinates of the calibration target feature points, and calculate the external parameters of the fisheye camera according to the pixel coordinates and world coordinates of the calibration target feature points;
[0026] A second feature point extraction module is used to identify a calibration target in the fisheye image, extract feature points of the calibration target, and obtain first pixel coordinates of the feature points of the calibration target;
[0027] A pixel coordinate calculation module, used to obtain the world coordinates of the calibration target feature point, and calculate the second pixel coordinates of the calibration target feature point according to the world coordinates of the calibration target feature point and the external parameter;
[0028] The calibration result output module is used to determine whether the fisheye camera passes the calibration according to the difference between the first pixel coordinate and the second pixel coordinate.
[0029] According to the above device, in some implementations, the pixel coordinate calculation module is used to:
[0030] Calculating the coordinates of the calibration target feature point in the camera coordinate system according to the world coordinates of the calibration target feature point and the external parameter;
[0031] The internal parameters of the fisheye camera are obtained, and the coordinates of the calibration target feature points in the camera coordinate system are converted into corresponding pixel coordinates according to the internal parameters, and the pixel coordinates are dedistorted according to the internal parameters to obtain second pixel coordinates.
[0032] According to the above device, in some embodiments, the calibration target includes a grid target and a circular target, and the calibration target feature point includes a plurality of feature points, and the plurality of feature points include a center point of the grid target and a center point of the circular target;
[0033] The calibration target includes a chessboard target, and the calibration target feature points include a plurality of corner points of the chessboard target.
[0034] According to the above device, in some implementations, the calibration result output module is specifically used to:
[0035] The difference between the first pixel coordinate and the second pixel coordinate of each corner point is calculated respectively to obtain a plurality of difference values; and an average value of the plurality of difference values is calculated, and whether the fisheye camera passes the calibration is determined according to a comparison result between the average value and a preset threshold value.
[0036] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the vehicle-mounted surround view camera calibration method as described above is implemented.
[0037] The embodiments of the present application provide a method and device for calibrating a vehicle-mounted surround view camera, and a computer-readable storage medium. A calibration target and a correction target are set. During calibration, the fisheye camera captures a fisheye image containing the calibration target and the correction target. The feature points of the calibration target in the fisheye image can be used to calibrate the external parameters of the fisheye camera, and the feature points of the correction target in the fisheye image can be used to automatically check the calibration error. Calibration results that do not meet the requirements can be automatically discarded without manual inspection, thus avoiding a long time. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0039] Figure 1 The present invention is a flowchart of a method for calibrating a vehicle-mounted surround view camera in one embodiment of the present application.
[0040] Figure 2 It is a schematic diagram of a calibration target and a correction target in one embodiment of the present application.
[0041] Figure 3 A schematic diagram of a vehicle-mounted surround view camera calibration device in one embodiment of the present application. DETAILED DESCRIPTION
[0042] The detailed description of the drawings is intended as an illustration of some embodiments of the present application, and is not intended to represent the only form in which the present application can be implemented. It should be understood that the same or equivalent functions can be accomplished by different embodiments intended to be included in the spirit and scope of the present application.
[0043] An embodiment of the present application provides a method for calibrating a vehicle-mounted surround view camera, wherein the vehicle-mounted surround view camera includes a plurality of fisheye cameras;
[0044] The method of this embodiment includes calibrating each fisheye camera separately, see Figure 1 , ,calibration of each fisheye camera includes the following steps:
[0045] Step S100, obtaining a fisheye image captured by the fisheye camera, wherein a calibration target is arranged at a preset position in the field of view of the fisheye camera, and correction targets are distributed around the calibration target.
[0046] Specifically, when calibrating the on-board surround view camera, the vehicle to be calibrated enters the calibration area. By moving the vehicle and the calibration target and correction target, the calibration target and correction target are placed in a preset position in the field of view of the fisheye camera. When the fisheye camera is started to shoot, the fisheye camera can capture the calibration target and correction target.
[0047] Before starting the fisheye camera to shoot, you need to use a surveying tool to measure the coordinate positions of the feature points of the calibration target and the correction target in the world coordinate system. Each position is recorded as w(x, y, z), for example. Here, it is assumed that all target positions are on the ground, so z=0, that is, the coordinates are w(x, y, 0). Create a configuration file and store the coordinate positions of the feature points of the calibration target and the correction target in the world coordinate system (that is, the world coordinates described in the subsequent steps) in the configuration file. The configuration file is used for calculations in subsequent steps.
[0048] Step S200 , identifying a calibration target in the fisheye image, extracting feature points of the calibration target to obtain calibration target feature points, and obtaining pixel coordinates of the calibration target feature points.
[0049] Specifically, when identifying the calibration target in the fisheye image, some preprocessing may be performed to improve the clarity of the image, such as performing image enhancement on the fisheye image using an image enhancement algorithm.
[0050] For example, for the preprocessed fisheye image, the calibration target feature points can be extracted in the following way: the calibration target area is segmented from the fisheye image, the calibration target area is binarized, and then the feature points are extracted using a preset feature point extraction algorithm to obtain the calibration target feature points; the calibration target feature points can be one feature point or multiple feature points; it should be noted that the feature point extraction algorithm in step S200 is not limited to a certain one, such as SIFT algorithm, SURF algorithm, FAST algorithm, ORB algorithm, AKAZE algorithm, etc., and can be specifically considered based on the algorithm time consumption, effect, etc. and actual needs.
[0051] After obtaining the calibration target feature points, the sub-pixel point finding algorithm can be used to perform sub-pixel extraction to obtain the pixel coordinates of the calibration target feature points; specifically, for algorithms based on image features, the sub-pixel point finding algorithm can improve the accuracy of matching and reduce the possibility of algorithm errors.
[0052] Step S300, obtaining the world coordinates of the calibration target feature points, and calculating the extrinsic parameters of the fisheye camera according to the pixel coordinates and the world coordinates of the calibration target feature points.
[0053] Specifically, the world coordinates of the calibration target feature points are obtained from the configuration file, and the preset calibration algorithm is called. Based on the calibration algorithm, the external parameters of the fisheye camera can be calculated according to the pixel coordinates and world coordinates of the calibration target feature points; the external parameters refer to the position and posture of the camera in space, that is, the rotation and translation of the camera coordinate system (also called the camera coordinate system) relative to the world coordinate system, so the external parameters specifically include the rotation matrix R and the translation matrix T; the calibration algorithm is, for example, the solvePnP() algorithm, which is an OpenCV function that uses 3D-2D point pairs to solve the camera posture (rotation, translation) and the 3D coordinates of the target point in the camera coordinate system.
[0054] Step S400 , identifying a calibration target in the fisheye image, extracting calibration target feature points from the calibration target, and acquiring first pixel coordinates of the calibration target feature points.
[0055] For example, for the preprocessed fisheye image, the correction target feature points can be extracted in the following way: the correction target area is segmented from the fisheye image, the correction target area is binarized, and then the feature points are extracted using a preset feature point extraction algorithm to obtain the correction target feature points; the correction target feature points can be one feature point or multiple feature points; it should be noted that the feature point extraction algorithm in step S400 is not limited to a certain one, such as SIFT algorithm, SURF algorithm, FAST algorithm, ORB algorithm, AKAZE algorithm, etc., and can be specifically considered based on the algorithm time consumption, effect, etc. and actual needs.
[0056] Step S500 , obtaining the world coordinates of the calibration target feature points, and calculating the second pixel coordinates of the calibration target feature points according to the world coordinates of the calibration target feature points and the external parameters.
[0057] Specifically, the corresponding coordinates of the correction target feature point in the camera coordinate system can be calculated based on the world coordinates of the correction target feature point and the external parameters. Further, the second pixel coordinates can be obtained based on the corresponding coordinates and the known internal parameters of the fisheye camera.
[0058] Step S600, determining whether the fisheye camera passes calibration according to the difference between the first pixel coordinate and the second pixel coordinate.
[0059] Specifically, the calibration error may be automatically checked according to the difference between the first pixel coordinate and the second pixel coordinate to determine whether the calibration is passed.
[0060] In summary, the method of this embodiment sets a calibration target and a correction target. During calibration, the fisheye image captured by the fisheye camera contains the calibration target and the correction target. The feature points of the calibration target in the fisheye image can be used to calibrate the external parameters of the fisheye camera, and the feature points of the correction target in the fisheye image can be used to automatically check the calibration error. The calibration results that do not meet the requirements can be automatically eliminated without manual inspection, thus avoiding time-consuming. The method of this embodiment adopts a combination of calibration targets and verification targets, which can realize an integrated process of calibration and calibration result verification, which is of great help to improve the calibration quality, while reducing the workload and time consumption of calibration result verification.
[0061] In some embodiments, the step S500 specifically includes:
[0062] Step S501, calculating the coordinates of the calibration target feature point in the camera coordinate system according to the world coordinates of the calibration target feature point and the external parameter;
[0063] For example, let the world coordinates of the calibration target feature point be w(X w ,Y w ,Z w ), where Z w is 0; let the coordinate corresponding to w in the camera coordinate system be c(ui,vi,0);
[0064] Then we have:
[0065]
[0066] Among them, R is the rotation matrix of the external parameter, and T is the translation matrix of the external parameter;
[0067] Step S502, obtaining the internal parameters of the fisheye camera, and converting the coordinates of the calibration target feature points in the camera coordinate system into corresponding pixel coordinates according to the internal parameters, and performing dedistortion processing on the pixel coordinates according to the internal parameters to obtain second pixel coordinates.
[0068] Specifically, the internal parameters are known, including the focal length f, the pixel coordinates of the center point of the image (u0, v0), and the distortion parameters; according to the internal parameters and c(ui, vi, 0), the corresponding coordinates of c in the pixel coordinate system can be calculated, as follows:
[0069]
[0070] in, is the focal length of the camera in the x direction; is the focal length of the camera in the y direction.
[0071] In some embodiments, the calibration target includes a grid target and a circular target, and the calibration target feature point includes a plurality of feature points, and the plurality of feature points include a center point of the grid target and a center point of the circular target.
[0072] The calibration target includes a chessboard target, and the calibration target feature points include a plurality of corner points of the chessboard target; specifically, the chessboard target includes a plurality of chessboard squares, and the corner points refer to four corner points of each chessboard square.
[0073] For example, Figure 2 FIG. 1 is a design example of a calibration target and a correction target of this embodiment. Figure 2 The target comprises a plurality of square targets and circular targets, and chessboard targets are arranged between the square targets and circular targets.
[0074] It should be noted that the current calibration method has algorithm defects due to the single calibration target feature point, resulting in low calibration accuracy. Therefore, this embodiment proposes combining a grid target and a circular target as calibration targets to obtain different types of calibration target feature points, which is conducive to improving the calibration accuracy.
[0075] In some embodiments, determining whether the fisheye camera passes calibration according to the difference between the first pixel coordinate and the second pixel coordinate includes:
[0076] The difference between the first pixel coordinate and the second pixel coordinate of each corner point is calculated respectively to obtain a plurality of difference values; and an average value of the plurality of difference values is calculated, and whether the fisheye camera passes the calibration is determined according to a comparison result between the average value and a preset threshold value.
[0077] Specifically, when extracting feature points, since there are multiple corner points on the chessboard target, each corner point has a corresponding first pixel coordinate and a second pixel coordinate. Therefore, it is necessary to consider the pixel coordinate deviation of all corner points. All corner points are comprehensively considered by taking the average value of the area, which can be expressed as the following formula:
[0078]
[0079] Among them, Err is the average value of multiple differences, i represents the i-th corner point, n represents the total number of corner points, and u pi is the horizontal coordinate of the first pixel coordinate of the i-th corner point, v pi is the ordinate of the first pixel coordinate of the i-th corner point, u ji is the horizontal coordinate of the second pixel coordinate of the i-th corner point, v ji is the ordinate of the second pixel coordinate of the i-th corner point;
[0080] In this embodiment, a preset threshold is set to control the calibration quality. If Err is less than the preset threshold, the calibration is determined to be passed, and the qualified calibration result is saved for subsequent surround stitching. If Err is greater than the preset threshold, the calibration is determined to be failed.
[0081] In some embodiments, it also includes:
[0082] Step S600, when the multiple fisheye cameras have passed the calibration, the fisheye images taken by the multiple fisheye cameras are subjected to dedistortion processing to obtain multiple dedistorted fisheye images, and the multiple fisheye images are stitched together to obtain a panoramic image.
[0083] Specifically, generally speaking, the vehicle-mounted surround-view fisheye camera includes four fisheye cameras in the front, back, left and right, which are used to capture environmental images in different directions of the vehicle. When the vehicle's fisheye camera fails to pass the calibration, it needs to be recalibrated; when all the fisheye cameras of the vehicle pass the calibration, the fisheye images taken by all the fisheye cameras are dedistorted according to the distortion parameters in the external parameters, and the surround-view stitching is performed to obtain a panoramic image.
[0084] Another embodiment of the present application further provides a vehicle-mounted surround view camera calibration device, which includes a module that can be used to execute the method steps of the above embodiment, and the vehicle-mounted surround view camera includes multiple fisheye cameras.
[0085] The device comprises:
[0086] The image acquisition module 1 is used to acquire the fisheye image taken by the fisheye camera, wherein a calibration target is arranged at a preset position in the field of view of the fisheye camera, and correction targets are distributed around the calibration target.
[0087] The first feature point extraction module 2 is used to identify the calibration target in the fisheye image, extract feature points of the calibration target to obtain calibration target feature points, and obtain pixel coordinates of the calibration target feature points.
[0088] The external parameter calculation module 3 is used to obtain the world coordinates of the calibration target feature points, and calculate the external parameters of the fisheye camera according to the pixel coordinates and world coordinates of the calibration target feature points.
[0089] The second feature point extraction module 4 is used to identify the calibration target in the fisheye image, extract the calibration target feature points, and obtain the first pixel coordinates of the calibration target feature points.
[0090] The pixel coordinate calculation module 5 is used to obtain the world coordinates of the calibration target feature point, and calculate the second pixel coordinates of the calibration target feature point according to the world coordinates of the calibration target feature point and the external parameter.
[0091] The calibration result output module 6 is used to determine whether the fisheye camera passes the calibration according to the difference between the first pixel coordinate and the second pixel coordinate.
[0092] In some embodiments, the pixel coordinate calculation module 5 is specifically used for:
[0093] Calculating the coordinates of the calibration target feature point in the camera coordinate system according to the world coordinates of the calibration target feature point and the external parameter;
[0094] The internal parameters of the fisheye camera are obtained, and the coordinates of the calibration target feature points in the camera coordinate system are converted into corresponding pixel coordinates according to the internal parameters, and the pixel coordinates are dedistorted to obtain second pixel coordinates.
[0095] In some embodiments, the calibration target includes a grid target and a circular target, and the calibration target feature point includes a plurality of feature points, and the plurality of feature points include a center point of the grid target and a center point of the circular target.
[0096] The calibration target includes a chessboard target, and the calibration target feature points include a plurality of corner points of the chessboard target.
[0097] In some embodiments, the calibration result output module 6 is specifically used to:
[0098] The difference between the first pixel coordinate and the second pixel coordinate of each corner point is calculated respectively to obtain a plurality of difference values, and the average value of the plurality of difference values is calculated, and whether the fisheye camera passes the calibration is determined according to the comparison result between the average value and a preset threshold value.
[0099] The vehicle-mounted surround view camera calibration device of the above-described embodiment is merely illustrative, wherein the modules described as separate components may or may not be physically separated, and the components of the modules may or may not be physical modules, that is, they may be located in one place, or may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the vehicle-mounted surround view camera calibration device of the embodiment.
[0100] It should be noted that the vehicle-mounted surround-view camera calibration device of the above-mentioned embodiment corresponds to the vehicle-mounted surround-view camera calibration method of the above-mentioned embodiment. Therefore, the part of the vehicle-mounted surround-view camera calibration device of the above-mentioned embodiment that is not described in detail can be obtained by referring to the content of the vehicle-mounted surround-view camera calibration method of the above-mentioned embodiment, that is, the specific steps recorded in the vehicle-mounted surround-view camera calibration method of the above-mentioned embodiment can be understood as the functions that can be achieved by the vehicle-mounted surround-view camera calibration device of the above-mentioned embodiment, and will not be repeated here.
[0101] Furthermore, if the vehicle-mounted surround view camera calibration device of the above embodiment is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium.
[0102] Another embodiment of the present application further proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the vehicle-mounted surround view camera calibration method as described in the above embodiment is implemented.
[0103] Specifically, the computer-readable storage medium may include: any entity or recording medium that can carry the computer program instructions, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.
[0104] Another embodiment of the present application provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the vehicle-mounted surround view camera calibration method described in the above embodiment when executing the program.
[0105] Wherein, the electronic device may also include a bus connecting different components (including memory and processor). The memory may include a computer-readable medium in the form of a volatile memory, such as a random access memory (RAM) and / or a cache memory. The memory may also include at least one program product, which has a set of (e.g., at least one) program modules, which are configured to perform the functions of the various embodiments of the present application. The electronic device may also communicate with one or more external devices (e.g., keyboards, pointing devices, displays, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device, and / or communicate with any device (e.g., a network card) that enables the electronic device to communicate with one or more other computing devices, such communication may be performed through an input / output (I / O) interface, and the electronic device may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs) and / or public networks, such as the Internet) through a network adapter.
[0106] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.
Claims
1. A method for calibrating a vehicle-mounted surround view camera, characterized in that: The vehicle-mounted surround view camera includes a plurality of fisheye cameras; The method comprises: Acquire a fisheye image captured by the fisheye camera; wherein a calibration target is arranged at a preset position in the field of view of the fisheye camera, and correction targets are distributed around the calibration target; Identify a calibration target in the fisheye image, extract feature points of the calibration target to obtain calibration target feature points, and obtain pixel coordinates of the calibration target feature points; Acquire the world coordinates of the calibration target feature points, and calculate the external parameters of the fisheye camera according to the pixel coordinates and world coordinates of the calibration target feature points; Identify a calibration target in the fisheye image, extract calibration target feature points on the calibration target, and obtain first pixel coordinates of the calibration target feature points; Acquire the world coordinates of the calibration target feature point, and calculate the second pixel coordinates of the calibration target feature point according to the world coordinates of the calibration target feature point and the external parameter; Determine whether the fisheye camera passes calibration according to the difference between the first pixel coordinate and the second pixel coordinate.
2. The vehicle-mounted surround view camera calibration method according to claim 1, characterized in that: The calculating the second pixel coordinate of the calibration target feature point according to the world coordinate of the calibration target feature point and the external parameter comprises: Calculating the coordinates of the calibration target feature point in the camera coordinate system according to the world coordinates of the calibration target feature point and the external parameter; The internal parameters of the fisheye camera are obtained, and the coordinates of the calibration target feature points in the camera coordinate system are converted into corresponding pixel coordinates according to the internal parameters, and the pixel coordinates are dedistorted according to the internal parameters to obtain second pixel coordinates.
3. The vehicle-mounted surround view camera calibration method according to claim 1, characterized in that: The calibration target includes a square target and a circular target, and the calibration target feature point includes a plurality of feature points, and the plurality of feature points include a center point of the square target and a center point of the circular target; The calibration target includes a chessboard target, and the calibration target feature points include a plurality of corner points of the chessboard target.
4. The vehicle-mounted surround view camera calibration method according to claim 3, characterized in that: The determining whether the fisheye camera passes calibration according to the difference between the first pixel coordinate and the second pixel coordinate includes: The difference between the first pixel coordinate and the second pixel coordinate of each corner point is calculated respectively to obtain a plurality of difference values; and an average value of the plurality of difference values is calculated, and whether the fisheye camera passes the calibration is determined according to a comparison result between the average value and a preset threshold value.
5. The vehicle-mounted surround view camera calibration method according to any one of claims 1 to 4, characterized in that: The method comprises: When the multiple fisheye cameras are all calibrated, the fisheye images taken by the multiple fisheye cameras are subjected to dedistortion processing to obtain multiple dedistorted fisheye images, and the multiple fisheye images are stitched together to obtain a panoramic image.
6. A vehicle-mounted surround view camera calibration device, characterized in that: The vehicle-mounted surround view camera includes a plurality of fisheye cameras; The device comprises: An image acquisition module, used to acquire the fisheye image taken by the fisheye camera; wherein a calibration target is arranged at a preset position in the field of view of the fisheye camera, and correction targets are distributed around the calibration target; A first feature point extraction module is used to identify a calibration target in the fisheye image, extract feature points of the calibration target to obtain calibration target feature points, and obtain pixel coordinates of the calibration target feature points; An external parameter calculation module, used to obtain the world coordinates of the calibration target feature points, and calculate the external parameters of the fisheye camera according to the pixel coordinates and world coordinates of the calibration target feature points; A second feature point extraction module is used to identify a calibration target in the fisheye image, extract feature points of the calibration target, and obtain first pixel coordinates of the feature points of the calibration target; A pixel coordinate calculation module, used to obtain the world coordinates of the calibration target feature point, and calculate the second pixel coordinates of the calibration target feature point according to the world coordinates of the calibration target feature point and the external parameter; The calibration result output module is used to determine whether the fisheye camera passes the calibration according to the difference between the first pixel coordinate and the second pixel coordinate.
7. The vehicle-mounted surround view camera calibration device according to claim 6, characterized in that: The pixel coordinate calculation module is specifically used for: Calculating the coordinates of the calibration target feature point in the camera coordinate system according to the world coordinates of the calibration target feature point and the external parameter; The internal parameters of the fisheye camera are obtained, and the coordinates of the calibration target feature points in the camera coordinate system are converted into corresponding pixel coordinates according to the internal parameters, and the pixel coordinates are dedistorted according to the internal parameters to obtain second pixel coordinates.
8. The vehicle-mounted surround view camera calibration device according to claim 6, characterized in that: The calibration target includes a square target and a circular target, and the calibration target feature point includes a plurality of feature points, and the plurality of feature points include a center point of the square target and a center point of the circular target; The calibration target includes a chessboard target, and the calibration target feature points include a plurality of corner points of the chessboard target.
9. The vehicle-mounted surround view camera calibration device according to claim 8, characterized in that: The calibration result output module is specifically used for: The difference between the first pixel coordinate and the second pixel coordinate of each corner point is calculated respectively to obtain a plurality of difference values; and an average value of the plurality of difference values is calculated, and whether the fisheye camera passes the calibration is determined according to a comparison result between the average value and a preset threshold value.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the vehicle-mounted surround view camera calibration method according to any one of claims 1 to 5 is implemented.
Citation Information
Patent Citations
Automatic re-calibration method and device for vehicle body look-around image
CN114972521A
Camera calibration method, object feature positioning method, device, equipment and medium
CN115294205A
Method, device and equipment for automatically correcting installation height of fisheye camera
CN115797468A
