Camera calibration method and device, computer equipment, computer readable storage medium and computer program product
By using L-shaped and T-shaped luminescent calibrators for camera calibration, the angle and distance limitations of traditional checkerboard calibration methods are solved, and a wider camera calibration applicability and higher accuracy are achieved.
Patent Information
- Application Number
- CN202411742514.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-29
- Publication Date
- 2025-03-14
AI Technical Summary
Traditional checkerboard calibration methods have limitations in terms of angle and distance, making it difficult to meet the wider camera calibration needs.
The camera calibration is performed using L-shaped and T-shaped luminescent calibrators. By acquiring images from multiple perspectives, identifying feature points, calibrating the initial camera parameters, and calibrating the difference between the physical distance and the three-dimensional distance, the target camera parameters are obtained.
It improves the applicability and flexibility of camera calibration, removes angle and distance limitations, enhances spatial angle identification information, and improves the accuracy of camera calibration.
Smart Images
Figure CN119625081A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of camera calibration, and in particular to a camera calibration method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art
[0002] The commonly used camera calibration method is the checkerboard calibration method. The checkerboard calibration method sets up multiple cameras to collect images of the same checkerboard calibration object at different viewing angles, and then calibrates the internal and external parameters of each camera based on the positions of the checkerboard corner points identified from the image. However, since the checkerboard corner points in the checkerboard calibration object are all on the same plane and are limited by the size of the checkerboard print frame, the checkerboard calibration method is prone to angle and distance limitations during use. With the further development of camera calibration technology, a camera calibration method with a wider range of applications is urgently needed. Summary of the invention
[0003] Based on this, it is necessary to provide a camera calibration method, device, computer equipment, computer-readable storage medium and computer program product with better applicability to the above technical problems.
[0004] In a first aspect, the present application provides a camera calibration method, comprising:
[0005] Acquire a first calibration image collected at multiple viewing angles for an L-shaped luminous calibration object, and a second calibration image collected at the multiple viewing angles for a T-shaped luminous calibration object;
[0006] Identify a first feature point where each first luminous mark of the L-shaped luminous calibration object is located from the first calibration image;
[0007] Obtaining initial camera parameters according to the first feature point calibration under the multiple viewing angles;
[0008] Identify the second feature point where each second luminous mark of the T-shaped luminous calibration object is located from the second calibration image;
[0009] Determine the three-dimensional distance between each of the second luminous marks using the initial camera parameters and the second feature point positions at the same viewing angle;
[0010] The initial camera parameters are calibrated according to the difference between the physical distance between each of the second luminous marks and the three-dimensional distance to obtain target camera parameters.
[0011] In one embodiment, the step of calibrating the initial camera parameters according to the difference between the physical distance between each of the second luminous markers and the three-dimensional distance to obtain the target camera parameters includes:
[0012] Calibrate the initial camera parameters using the difference between the physical distance and the three-dimensional distance to obtain calibrated camera parameters;
[0013] Determine the three-dimensional distance between each of the second luminous marks using the calibrated camera parameters and the second feature points at the same viewing angle;
[0014] When the difference between the physical distance and the three-dimensional distance corresponding to the calibrated camera parameters is within a preset range, the calibrated camera parameters are used as the target camera parameters.
[0015] In one of the embodiments, the initial camera parameters include initial camera extrinsics;
[0016] The obtaining of initial camera parameters according to the first feature point calibration under the multiple viewing angles includes:
[0017] Read the pre-stored initial camera internal parameters;
[0018] Constructing a three-dimensional model of the L-shaped luminous calibration object according to the shape characteristics of the L-shaped luminous calibration object;
[0019] The initial camera extrinsic parameters are determined using the three-dimensional model, the first feature point and the initial camera intrinsic parameters.
[0020] In one embodiment, the determining the three-dimensional distance between each of the second luminous marks using the initial camera parameters and the second feature points under the same viewing angle includes:
[0021] Projecting the second feature point under the same viewing angle into three-dimensional space using the initial camera extrinsic parameter to obtain a projected feature point corresponding to the second feature point;
[0022] The distance between the projection feature points in the three-dimensional space is determined, and the distance between the projection feature points is used as the three-dimensional distance between the corresponding second luminous marks.
[0023] In one embodiment, determining the initial camera extrinsic parameters using the three-dimensional model, the first feature point and the initial camera intrinsic parameters includes:
[0024] Constructing a basic matrix using the three-dimensional model, the first feature point and the initial camera intrinsic parameters;
[0025] Using the basic matrix and the first feature point at the first viewing angle, draw an epipolar line corresponding to the first feature point at the second viewing angle;
[0026] When it is determined that the first feature point under the second viewing angle matches the epipolar line, the basic matrix is decomposed to obtain a corresponding initial rotation vector and translation vector;
[0027] The initial rotation vector and the translation vector are used as the initial camera extrinsic parameters.
[0028] In one embodiment, the step of identifying, from the first calibration image, a first feature point where each first luminous mark of the L-shaped luminous calibration object is located includes:
[0029] Performing grayscale conversion and Gaussian blur processing on the first calibration image to obtain a processed grayscale calibration image;
[0030] Binarizing the grayscale calibration image to obtain a corresponding binary image;
[0031] Performing corrosion and dilation processing on the binary image to obtain a processed target binary image;
[0032] Performing contour detection on the target binary image to obtain a corresponding contour detection result;
[0033] Selecting a target contour whose contour area meets a preset condition from the contour detection results;
[0034] The first feature point position is determined according to the centroid position of the target contour.
[0035] In a second aspect, the present application also provides a camera calibration device, comprising:
[0036] An image acquisition module, used to acquire first calibration images collected at multiple viewing angles for the L-shaped luminous calibration object, and second calibration images collected at the multiple viewing angles for the T-shaped luminous calibration object;
[0037] An initial calibration module, configured to identify, from the first calibration image, a first feature point where each first luminous mark of the L-shaped luminous calibration object is located; and obtain initial camera parameters according to the first feature point calibration under the multiple viewing angles;
[0038] A calibration module is used to identify the second feature point position of each second luminous mark of the T-shaped luminous calibration object from the second calibration image; determine the three-dimensional distance between each second luminous mark using the initial camera parameters and the second feature point position under the same viewing angle; calibrate the initial camera parameters according to the difference between the physical distance between each second luminous mark and the three-dimensional distance to obtain the target camera parameters.
[0039] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the camera calibration method described in any one of the embodiments of the first aspect is implemented.
[0040] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the camera calibration method described in any one of the embodiments of the first aspect is implemented.
[0041] In a fifth aspect, the present application further provides a computer program product, wherein the computer program product includes a computer program, and when the computer program is executed by a processor, the camera calibration method described in any one of the embodiments of the first aspect is implemented.
[0042] The camera calibration method, apparatus, computer device, computer-readable storage medium and computer program product obtain a first calibration image collected at multiple viewing angles for an L-shaped luminous calibration object and a second calibration image collected at multiple viewing angles for a T-shaped luminous calibration object, identify a first feature point at which each first luminous mark of the L-shaped luminous calibration object is located from the first calibration image, calibrate the first feature point at multiple viewing angles to obtain initial camera parameters, identify a second feature point at which each second luminous mark of the T-shaped luminous calibration object is located from the second calibration image, determine the three-dimensional distance between each second luminous mark using the initial camera parameters and the second feature point at the same viewing angle, calibrate the initial camera parameters according to the difference between the physical distance and the three-dimensional distance between each second luminous mark, and obtain the target camera parameters. The structural characteristics of the L-shaped luminous calibration object and the T-shaped luminous calibration object can be used to provide more spatial angle recognition information, thereby solving the angle limitation and distance limitation of the traditional checkerboard calibration method, thereby improving the applicability and flexibility of the camera calibration method. In addition, the above-mentioned camera calibration method can improve the camera calibration accuracy by first using an L-shaped luminous calibration object for initial calibration, and then calibrating the calibration parameters based on the difference between the physical distance of the T-shaped luminous calibration object and the three-dimensional distance calculated using the initial camera parameters. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0044] Figure 1A diagram showing an application environment of a camera calibration method in one embodiment;
[0045] Figure 2 is a schematic flow chart of a camera calibration method in one embodiment;
[0046] Figure 3 A schematic diagram of a process of determining target camera parameters in one embodiment;
[0047] Figure 4 A schematic diagram of a process of determining initial camera parameters in one embodiment;
[0048] Figure 5 is a schematic flow chart of a step of determining initial camera extrinsic parameters in one embodiment;
[0049] Figure 6 It is a schematic flow chart of a first feature point recognition step in one embodiment;
[0050] Figure 7 is a schematic flow chart of a camera calibration method in another embodiment;
[0051] Figure 8 is a structural block diagram of a camera calibration device 800 in one embodiment;
[0052] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0053] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0054] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.
[0055] The camera calibration method provided in the embodiment of the present application can be applied to Figure 1 The application environment shown includes an L-shaped luminous calibration object 120 , a T-shaped luminous calibration object 140 , a camera 160 and a server 180 .
[0056] The L-shaped luminous marker 120 may be provided with a plurality of first luminous marks 122. Those skilled in the art will appreciate that Figure 1The figure only shows a deployment method of setting the first luminous mark 122 at four special points of the L-shaped luminous calibration object 120, which does not constitute a limitation on the L-shaped luminous object 120 in this case. Based on the same inventive concept as this case, those skilled in the art can also flexibly adjust the number of the first luminous mark 122.
[0057] A plurality of second luminous marks 142 may also be provided in the T-shaped luminous calibration object 140 . Figure 1 The figure only shows a deployment mode of setting a second luminous mark 142 at both ends of the T-shaped luminous mark 140, which does not constitute a limitation on the T-shaped luminous mark 140 in this case. Based on the same inventive concept as this case, those skilled in the art can also flexibly adjust the number of the second luminous marks 142.
[0058] Optionally, in some embodiments, the first luminous marker 122 and the second luminous marker 142 may be spherical infrared electronic display (LED) devices.
[0059] The plurality of cameras 160 may be used to capture images of the L-shaped luminous calibration object 120 and the T-shaped luminous calibration object 140 from different viewing angles.
[0060] The server 180 can be connected to each camera 160 through a bus or a network to receive and process the image data collected by each camera 160. The server 180 can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.
[0061] Exemplarily, the server 180 may obtain first calibration images at multiple viewing angles collected by different cameras 160 for the L-shaped luminous calibration object 120, and second calibration images at multiple viewing angles collected for the T-shaped luminous calibration object 140. The server 180 may identify the first feature point position where each first luminous mark 122 of the L-shaped luminous calibration object 120 is located from the first calibration image. The initial camera parameters of each camera 160 relative to the L-shaped luminous calibration object 120 are obtained by calibration based on the first feature point positions at multiple viewing angles. The server 180 may identify the second feature point position where each second luminous mark 142 of the T-shaped luminous calibration object 140 is located from the second calibration image. The server 180 may determine the three-dimensional distance between each second luminous mark 142 using the initial camera parameters and the second feature point position at the same viewing angle. The server 180 may calibrate the initial camera parameters according to the difference between the physical distance and the three-dimensional distance between each second luminous mark 142, and obtain the target camera parameters of each camera 160.
[0062] In this embodiment, since the design of the L-shaped luminous calibration object and the T-shaped luminous calibration object is easier to assemble and disassemble, the L-shaped luminous calibration object and the T-shaped luminous calibration object are used for camera calibration. Compared with the traditional checkerboard calibration method, it is more suitable for application scenarios that require frequent changes in test sites, simplifies the operational difficulty of camera calibration, and allows non-professionals to easily use the L-shaped luminous calibration object and the T-shaped luminous calibration object for camera calibration. At the same time, since the L-shaped luminous calibration object has a right-angle and asymmetric structure, it can provide more direct feedback on camera positioning and spatial angle recognition, and the T-shaped luminous calibration object can provide more angle recognition points and directional information, which is necessary to ensure accurate capture of long-distance cameras, and helps to improve the spatial recognition ability of the camera calibration method, thereby achieving the technical effect of improving the accuracy of camera calibration.
[0063] In an exemplary embodiment, Figure 2 As shown, a camera calibration method is provided, which is applied to Figure 1 The server 180 in the example is used as an example to illustrate the method, which includes the following steps S202 to S212. Among them:
[0064] Step S202 : acquiring a first calibration image collected at multiple viewing angles for an L-shaped luminous calibration object and a second calibration image collected at multiple viewing angles for a T-shaped luminous calibration object.
[0065] The L-shaped luminous marker can be used to represent an L-shaped rod with a self-luminous spherical first luminous mark. The T-shaped luminous marker can be used to represent a T-shaped rod with a self-luminous spherical second luminous mark.
[0066] Exemplarily, the server may obtain a first calibration image captured by multiple cameras at different viewing angles for an L-shaped luminous calibration object and a second calibration image captured for a T-shaped luminous calibration object. Optionally, in some embodiments, the first calibration image may be acquired when the L-shaped luminous calibration object is placed in the exact center of the camera motion capture area. At this time, the L-shaped luminous calibration object may be used as the origin of the entire spatial coordinate system, thereby facilitating a more accurate definition and calibration of spatial relationships. The second calibration image may be acquired when the T-shaped luminous calibration object is waved within the camera motion capture area.
[0067] Step S204: identifying the first characteristic point position where each first luminous mark of the L-shaped luminous calibration object is located from the first calibration image.
[0068] The first feature point may be used to indicate a pixel position corresponding to the first luminous mark in the first calibration image.
[0069] Optionally, in some embodiments, the server may store a brightness threshold that matches the first luminous mark. The server may perform brightness detection on the first calibration image, and use the pixel point whose detected pixel brightness matches the brightness threshold as the first feature point corresponding to the first luminous mark.
[0070] Alternatively, in some other embodiments, the server may also store contour features corresponding to the first luminous markers. The server may perform contour detection on the first calibration image to determine pixel points in the first calibration image that match the contour features. The first feature point position corresponding to each first luminous marker is determined using the pixel points that match the contour features.
[0071] Step S206, obtaining initial camera parameters according to the first feature point calibration under multiple viewing angles.
[0072] The initial camera parameters may be used to represent camera extrinsic parameters obtained through preliminary calibration, such as any one or more parameters such as a rotation vector and a translation vector.
[0073] Exemplarily, the server may store the camera's internal parameter matrix, distortion coefficients and other initial camera internal parameters. The server may use the camera's initial camera internal parameters, the physical size of the L-shaped luminous calibration object and the first feature point of the L-shaped luminous calibration object collected from multiple viewing angles to perform calculations and processing to obtain the camera's rotation vector, translation vector and other initial camera parameters relative to the L-shaped luminous calibration object at each viewing angle.
[0074] Optionally, in some implementations, the initial camera intrinsic parameters may be calculated by the server based on calibration images captured by the camera at multiple viewing angles with respect to a checkerboard calibration object.
[0075] Step S208 : identifying the second characteristic point position where each second luminous mark of the T-shaped luminous calibration object is located from the second calibration image.
[0076] The second feature point may be used to indicate a pixel position corresponding to the second luminous mark in the second calibration image.
[0077] Exemplarily, the server can refer to the method for determining the first feature point in the above step S204, and according to the brightness threshold or contour features of the second luminous mark in the T-shaped luminous calibration object, identify from the second calibration image the pixel coordinates corresponding to each second luminous mark in the T-shaped luminous calibration object, that is, the second feature point corresponding to each second luminous mark in the second mark image.
[0078] Step S210: Determine the three-dimensional distance between each second luminous mark using the initial camera parameters and the second feature points at the same viewing angle.
[0079] For example, the server can use the initial camera parameters to perform calculations on the second feature points under the same viewing angle to obtain the three-dimensional spatial position corresponding to each second feature point under the same viewing angle. The three-dimensional spatial position corresponding to the second feature point is used to calculate the three-dimensional distance between the second luminous marks corresponding to the second feature point.
[0080] Step S212: calibrate the initial camera parameters according to the difference between the physical distance between each second luminous marker and the three-dimensional distance to obtain the target camera parameters.
[0081] The physical distance may be used to represent the actual distance between each second luminous mark in the T-shaped luminous calibration object.
[0082] Exemplarily, the server may store the actual physical distance between each second luminous marker in the T-shaped luminous marker. Determine the difference between the actual physical distance between the second luminous markers and the three-dimensional distance between the second luminous markers calculated using the initial calibration parameters. Calibrate the initial camera parameters according to the difference. For example, when the physical distance is greater than the three-dimensional distance, the initial camera parameters may be increased, otherwise the initial camera parameters may be decreased, and the calibrated initial camera parameters are used as the target camera parameters.
[0083] In the above-mentioned camera calibration method, by acquiring a first calibration image collected at multiple viewing angles for an L-shaped luminous calibration object and a second calibration image collected at a T-shaped luminous calibration object, a first feature point at which each first luminous mark of the L-shaped luminous calibration object is located is identified from the first calibration image, and initial camera parameters are obtained by calibration according to the first feature points at multiple viewing angles, and a second feature point at which each second luminous mark of the T-shaped luminous calibration object is located is identified from the second calibration image, and the three-dimensional distance between each second luminous mark is determined using the initial camera parameters and the second feature points at the same viewing angle, and the initial camera parameters are calibrated according to the difference between the physical distance and the three-dimensional distance between each second luminous mark to obtain the target camera parameters. The structural characteristics of the L-shaped luminous calibration object and the T-shaped luminous calibration object can be used to provide more spatial angle recognition information, thereby solving the angle limitation and distance limitation of the traditional checkerboard calibration method, thereby improving the applicability and flexibility of the camera calibration method. In addition, the above-mentioned camera calibration method can improve the camera calibration accuracy by first using an L-shaped luminous calibration object for initial calibration, and then calibrating the calibration parameters based on the difference between the physical distance of the T-shaped luminous calibration object and the three-dimensional distance calculated using the initial camera parameters.
[0084] In an exemplary embodiment, Figure 3 As shown, the above step S212 may also include the following steps S302 to S306. Among them:
[0085] Step S302: calibrate the initial camera parameters using the difference between the physical distance and the three-dimensional distance to obtain calibrated camera parameters.
[0086] For example, the server may use the least square method to perform calculations on the physical distance between the second luminous markers and the three-dimensional distance calculated using the initial camera parameters to obtain a fitting coefficient between the physical distance and the three-dimensional distance. The fitting coefficient is used to calibrate the initial camera parameters to obtain calibrated camera parameters.
[0087] Step S304: Determine the three-dimensional distance between each second luminous mark using the calibrated camera parameters and the second feature points at the same viewing angle.
[0088] Exemplarily, the server can refer to the three-dimensional distance calculation method in the above step S210, use the calibrated camera parameters and the second feature points under the same viewing angle to perform calculations, obtain new three-dimensional space coordinates of each second luminous mark that matches the calibrated camera parameters, and determine the three-dimensional distance between the new three-dimensional space coordinates.
[0089] Step S306: When the difference between the physical distance and the three-dimensional distance corresponding to the calibrated camera parameters is within a preset range, the calibrated camera parameters are used as target camera parameters.
[0090] Exemplarily, the server may perform calculations on the actual physical distance and the new three-dimensional distance calculated in step S304. When it is determined that the difference between the physical distance and the three-dimensional distance corresponding to the calibrated camera parameters is within a preset range, the calibrated camera parameters at this time may be used as the target camera parameters.
[0091] Optionally, in other embodiments, when the difference between the physical distance and the three-dimensional distance corresponding to the calibrated camera parameters is still greater than a preset range, the operations of steps S302 to S304 can be repeated to iteratively optimize the camera parameters until the difference between the three-dimensional distance corresponding to the calibrated camera parameters and the actual physical distance is within a preset range.
[0092] In this embodiment, the calibration accuracy of the target camera parameters can be improved by iteratively optimizing and calibrating the initial camera parameters based on the fitting coefficient between the actual physical distance between each second luminous mark in the T-shaped luminous calibration object and the three-dimensional distance of the second luminous mark calculated by the initial camera parameters.
[0093] In an exemplary embodiment, the initial camera parameters include initial camera external parameters. Figure 4 As shown, the above step S206 may also include the following steps S402 to S406. Among them:
[0094] Step S402, reading the pre-stored initial camera intrinsic parameters.
[0095] Step S404: construct a three-dimensional model of the L-shaped luminous calibration object according to the shape characteristics of the L-shaped luminous calibration object.
[0096] Exemplarily, the server may read the pre-stored initial camera internal parameters from a local database or a data communication network. Figure 1 It can be seen from the shape of the L-shaped luminous calibration object 120 shown in that three first luminous marks 122 are arranged on the longer side of the L-shaped luminous calibration object 120, and two first luminous marks 122 are arranged on the shorter side. Therefore, the server can use the side with more first luminous marks as the first coordinate axis (such as the X-axis or the Y-axis) according to the shape characteristics of the L-shaped luminous calibration object (one side is longer and the other side is shorter), and use the side with fewer first luminous marks as the second coordinate axis (another coordinate axis different from the first coordinate axis), and use the intersection of the first coordinate axis and the second coordinate axis as the origin of the coordinate system to construct a third coordinate axis perpendicular to the first coordinate axis and the second coordinate axis. Use the first coordinate axis, the second coordinate axis and the third coordinate axis to construct a three-dimensional coordinate system where the L-shaped luminous calibration object is located. Construct a three-dimensional model of the L-shaped luminous calibration object in the three-dimensional coordinate system.
[0097] Step S406: determine initial camera extrinsic parameters using the three-dimensional model, the first feature point and the initial camera intrinsic parameters.
[0098] Exemplarily, the server can use an n-point perspective algorithm (also known as a PnP algorithm) to perform calculations on the three-dimensional spatial coordinates of the first luminous mark corresponding to the first feature point in the three-dimensional model, the first feature point, and the initial camera intrinsic parameters to estimate the relative poses between cameras at different viewing angles, thereby calculating the initial camera extrinsic parameters between each camera.
[0099] In this embodiment, a three-dimensional model of the L-shaped luminous calibration object is established by utilizing the shape features of the L-shaped luminous calibration object, and the initial camera extrinsic parameters are obtained by calibrating the initial camera intrinsic parameters, the three-dimensional spatial coordinates corresponding to the first luminous mark in the three-dimensional model, and the two-dimensional first feature point corresponding to the first luminous mark. The spatial angle information provided by the L-shaped luminous calibration object can be fully utilized, the difficulty of calibrating the initial camera parameters is simplified, and the efficiency of calibrating the initial camera parameters is greatly improved.
[0100] In an exemplary embodiment, the above step S210 may also be implemented in the following manner:
[0101] The second feature point under the same viewing angle is projected into three-dimensional space using the initial camera extrinsic parameters to obtain a projection feature point corresponding to the second feature point; the distance between the projection feature points in three-dimensional space is determined, and the distance between the projection feature points is used as the three-dimensional distance between the corresponding second luminous marks.
[0102] Exemplarily, the server can use the initial camera external parameters determined in step S406 to perform calculations on each second feature point under the same viewing angle, project the second feature point corresponding to the second luminous mark from the two-dimensional space to the three-dimensional space, and obtain the three-dimensional projection feature point corresponding to each second feature point. The three-dimensional space coordinates of the projection feature point are used to determine the distance of the projection feature point in the three-dimensional space. The distance between the projection feature points is used as the three-dimensional distance between the second luminous marks corresponding to the initial camera external parameters.
[0103] In this embodiment, by calculating the three-dimensional distance of the second luminous mark using the initial camera extrinsic parameters obtained by calibrating the L-shaped luminous calibration object and the two-dimensional second feature points of the T-shaped luminous calibration object captured by the camera, the three-dimensional distance determination process can be simplified and the efficiency of three-dimensional distance determination can be greatly improved.
[0104] In an exemplary embodiment, Figure 5 As shown, the above step S406 may also include the following steps S502 to S508. Among them:
[0105] Step S502: construct a basic matrix using the three-dimensional model, the first feature point and the initial camera intrinsic parameters.
[0106] Exemplarily, the server may use the three-dimensional space coordinates corresponding to the first luminous marker in the three-dimensional model and the first feature point corresponding to the first luminous marker to construct a feature point pair with a corresponding relationship. A basic matrix of the camera is obtained by performing calculations using a number of feature point pairs (e.g., seven, eight, or ten, etc.) and initial camera internal parameters such as an internal parameter matrix and a distortion coefficient.
[0107] Step S504: using the basic matrix and the first feature point at the first viewing angle, draw an epipolar line corresponding to the first feature point at the second viewing angle.
[0108] The first perspective and the second perspective are any two different perspectives among the multiple perspectives.
[0109] Exemplarily, the server may use the basic matrix and the first feature point at the first viewing angle to perform calculation processing to obtain the projection point of the first feature point at the second viewing angle. The projection point of the first feature point at the first viewing angle at the second viewing angle is connected to draw an epipolar line corresponding to the first feature point at the first viewing angle at the second viewing angle.
[0110] Step S506: When it is determined that the first feature point under the second viewing angle matches the epipolar line, the basic matrix is decomposed to obtain the corresponding initial rotation vector and translation vector.
[0111] Step S508: using the initial rotation vector and translation vector as initial camera extrinsic parameters.
[0112] Exemplarily, since the basic matrix is calculated accurately, the first feature point at the second perspective will fall on the epipolar line drawn based on the basic matrix. Therefore, the server can match the first feature point at the second perspective identified in the first calibration image with the epipolar line drawn in step S504. When the first feature point at the second perspective is on the epipolar line, it is determined that the first feature point at the second perspective matches the epipolar line. The basic matrix is decomposed using the essential matrix decomposition algorithm to obtain the initial rotation vector and translation vector corresponding to the basic matrix. The initial rotation vector and translation vector are used as the initial camera extrinsic parameters of the camera at the first perspective relative to the camera at the second perspective.
[0113] Optionally, in some embodiments, when the first feature point at the second viewing angle does not match the epipolar line, an epipolar constraint method may be used to optimize the basic matrix using the distance from the first feature point at the second viewing angle to the epipolar line, until the epipolar line at the second viewing angle calculated using the optimized basic matrix matches the first feature point at the second viewing angle.
[0114] In this embodiment, the camera extrinsic parameters are calibrated by adopting the epipolar geometry method, so that the calibration accuracy of the initial camera extrinsic parameters can be improved.
[0115] In an exemplary embodiment, Figure 6 As shown, step S204 may also include steps S602 to S612. Among them:
[0116] Step S602 , grayscale and Gaussian blur processing are performed on the first calibration image to obtain a processed grayscale calibration image.
[0117] Exemplarily, the server may grayscale the first calibration image, converting the first calibration image from a color image (RGB image) to a grayscale image, and perform Gaussian blur processing on the grayscale image using a preset Gaussian kernel (e.g., a kernel of size 9x9) to remove noise in the grayscale image, thereby obtaining a processed grayscale calibration image.
[0118] Step S604, binarizing the grayscale calibration image to obtain a corresponding binarized image.
[0119] Exemplarily, a preset threshold may be stored in the server. The preset threshold is used to perform binarization on the grayscale calibration image, the grayscale of pixels whose grayscale threshold is lower than the preset threshold is set as a first threshold, and the grayscale of pixels whose grayscale threshold is higher than the preset threshold is set as a second threshold, thereby obtaining a corresponding binarized image.
[0120] Step S606, performing erosion and dilation processing on the binary image to obtain a processed target binary image.
[0121] Step S608: Perform contour detection on the target binary image to obtain a corresponding contour detection result.
[0122] Exemplarily, the server may perform morphological processing of first corroding and then dilating the binary image to remove smaller noise points in the binary image, thereby obtaining a processed target binary image. Contour detection is performed on the target binary image, and contour detection results such as contour shape, contour number, and contour position in the target binary image are obtained.
[0123] Step S610: Select a target contour whose contour area meets a preset condition from the contour detection result.
[0124] Step S612: determining the first feature point according to the centroid position of the target contour.
[0125] Exemplarily, the server may sort the contour detection results according to the contour area. The first m contours (m is determined based on the number of first luminous marks set in the L-shaped luminous calibration object) with larger contour areas are selected from the contour detection results as the target contours that meet the preset conditions. The centroid position of the target contour is located and used as the first feature point corresponding to the first luminous mark.
[0126] In this embodiment, by graying, Gaussian blurring, binarizing, and corroding and dilating the first calibration image, noise in the first calibration image can be removed, which helps to improve image quality. By determining the target contour that matches the first luminous mark based on the contour area and preset conditions, and locating the first feature point using the center of mass position of the target contour, the accuracy of determining the first feature point can be improved.
[0127] In an exemplary embodiment, Figure 7 As shown, a camera calibration method is also provided, including the following steps S702 to S712. Among them:
[0128] Step S702: Acquire a first calibration image collected at multiple viewing angles for the L-shaped luminous calibration object.
[0129] Step S704: De-noising the first calibration image, and determining a first feature point corresponding to a first luminous mark of the L-shaped luminous calibration object according to a contour detection result of the target binary image obtained after the processing.
[0130] Exemplarily, the server can obtain the first calibration image collected by the multi-camera system for the L-shaped luminous calibration object at multiple viewing angles. The first calibration image is subjected to denoising processing such as grayscale, Gaussian blur, binarization, corrosion and expansion to obtain a processed target binary image. Contour detection is performed on the target binary image, and the contour detection results are sorted from large to small according to the contour size. The contour detection results with the contour size in the first 4 digits are used as the target contour. The centroid position of the target contour is determined by using the image moment as the first two-dimensional feature point corresponding to the first luminous mark.
[0131] Step S706 , determining initial camera extrinsic parameters using the initial camera intrinsic parameters, the three-dimensional model of the L-shaped luminous calibration object, and the first feature point positions at different viewing angles.
[0132] Exemplarily, the server may store the camera's intrinsic parameter matrix, distortion coefficients and other initial camera intrinsic parameters. The server may use the first luminous marker to establish a correspondence between the three-dimensional space coordinates in the three-dimensional model preset by the L-shaped luminous calibration object and the first feature point under different viewing angles. Using the intrinsic parameter matrix, distortion coefficients, the three-dimensional space coordinates with corresponding relationships and the first feature point under the first viewing angle, an epipolar line is drawn under the second viewing angle. The initial rotation vector and translation vector of the first viewing angle relative to the second viewing angle are determined by epipolar calibration, and the initial rotation vector and translation vector are used as the initial camera extrinsic parameters.
[0133] Step S708 , obtaining a second calibration image collected at multiple viewing angles for the T-shaped luminous calibration object, and determining a second feature point position corresponding to the second luminous mark of the T-shaped luminous calibration object in the second calibration image.
[0134] For example, the operator can wave a T-shaped luminous calibration object in the motion capture area of the multi-camera system. The server can obtain a second calibration image captured by the multi-camera system for the T-shaped luminous calibration object at multiple viewing angles. Referring to the determination method of the first feature point provided in step S704 above, the second calibration image is processed in the same manner, so as to identify the second feature point corresponding to the second luminous mark in the T-shaped luminous calibration object in the second calibration image.
[0135] Step S710, projecting the second feature point under the same viewing angle into the three-dimensional space using the initial camera extrinsic parameters, and determining the three-dimensional distance corresponding to the second feature point in the three-dimensional space.
[0136] Exemplarily, the server can use the second feature points under the same viewing angle outside the initial camera to perform calculations, project the two-dimensional second feature points into three-dimensional space, and obtain the three-dimensional distance between the second luminous marks corresponding to the initial camera parameters.
[0137] Step S712, calibrating the initial camera extrinsic parameters using the error between the actual physical distance of the second luminous calibration object and the three-dimensional distance determined by the initial camera extrinsic parameters to obtain the target camera extrinsic parameters.
[0138] Exemplarily, the server may store the actual physical distance between the second luminous calibration object. When the difference between the actual physical distance and the calculated three-dimensional distance is not within a preset range, the following operations are repeatedly performed to iteratively optimize the initial camera extrinsic parameters: the physical distance and the three-dimensional distance are processed by the least squares method to obtain corresponding fitting parameters. The initial camera extrinsic parameters are calibrated using the fitting parameters.
[0139] In this embodiment, by using L-shaped luminous calibration objects and T-shaped luminous calibration objects for camera calibration, compared with the traditional checkerboard calibration method, the difficulty of camera calibration operation can be simplified, more spatial angle information can be obtained, and angle limitations and distance limitations can be solved, thereby achieving the technical effect of improving the flexibility of camera calibration. By using L-shaped luminous calibration objects for preliminary calibration, and then combining the actual physical distance of the T-shaped luminous calibration objects to iteratively optimize the camera extrinsic parameters of the initial calibration, the accuracy of camera calibration can be improved.
[0140] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0141] Based on the same inventive concept, the embodiment of the present application also provides a camera calibration device for implementing the camera calibration method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more camera calibration device embodiments provided below can refer to the limitations of the camera calibration method above, and will not be repeated here.
[0142] In an exemplary embodiment, Figure 8 As shown, a camera calibration device 800 is provided, comprising: an image acquisition module 802, an initial calibration module 804 and a calibration module 806, wherein:
[0143] The image acquisition module 802 is used to acquire a first calibration image collected at multiple viewing angles for the L-shaped luminous calibration object and a second calibration image collected at multiple viewing angles for the T-shaped luminous calibration object.
[0144] The initial calibration module 804 is used to identify the first feature point where each first luminous mark of the L-shaped luminous calibration object is located from the first calibration image, and obtain initial camera parameters according to the first feature point calibration under multiple viewing angles.
[0145] The calibration module 806 is used to identify the second feature point of each second luminous mark of the T-shaped luminous calibration object from the second calibration image. The three-dimensional distance between each second luminous mark is determined using the initial camera parameters and the second feature point under the same viewing angle. The initial camera parameters are calibrated according to the difference between the physical distance and the three-dimensional distance between each second luminous mark to obtain the target camera parameters.
[0146] In an exemplary embodiment, the calibration module 806 is further used to calibrate the initial camera parameters using the difference between the physical distance and the three-dimensional distance to obtain the calibrated camera parameters. The three-dimensional distance between each second luminous mark is determined using the calibrated camera parameters and the second feature point under the same viewing angle. When the difference between the physical distance and the three-dimensional distance corresponding to the calibrated camera parameters is within a preset range, the calibrated camera parameters are used as the target camera parameters.
[0147] In an exemplary embodiment, the initial camera parameters include initial camera extrinsics. The initial calibration module 804 is further configured to read pre-stored initial camera intrinsic parameters. A three-dimensional model of the L-shaped luminous calibration object is constructed according to the shape features of the L-shaped luminous calibration object. The initial camera extrinsics are determined using the three-dimensional model, the first feature point, and the initial camera intrinsic parameters.
[0148] In an exemplary embodiment, the calibration module 806 is further used to project the second feature point under the same viewing angle into the three-dimensional space using the initial camera external parameters to obtain the projected feature point corresponding to the second feature point, determine the distance between the projected feature points in the three-dimensional space, and use the distance between the projected feature points as the three-dimensional distance between the corresponding second luminous marks.
[0149] In an exemplary embodiment, the initial calibration module 804 is further used to construct a basic matrix using the three-dimensional model, the first feature point and the initial camera intrinsic parameters. Using the basic matrix and the first feature point at the first viewing angle, an epipolar line corresponding to the first feature point is drawn at the second viewing angle. When it is determined that the first feature point at the second viewing angle matches the epipolar line, the basic matrix is decomposed to obtain the corresponding initial rotation vector and translation vector. The initial rotation vector and translation vector are used as the initial camera extrinsic parameters.
[0150] In an exemplary embodiment, the initial calibration module 804 is further used to grayscale and Gaussian blur the first calibration image to obtain a processed grayscale calibration image. Binarize the grayscale calibration image to obtain a corresponding binary image. Erosion and expansion are performed on the binary image to obtain a processed target binary image. Contour detection is performed on the target binary image to obtain a corresponding contour detection result. A target contour whose contour area meets a preset condition is selected from the contour detection result. The first feature point is determined according to the centroid position of the target contour.
[0151] Each module in the camera calibration device 800 can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.
[0152] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig. 9 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as a first calibration image, a second calibration image, a first feature point, a second feature point, an initial camera parameter, a three-dimensional distance, a physical distance, etc. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a camera calibration method is implemented.
[0153] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0154] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.
[0155] In an exemplary embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.
[0156] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.
[0157] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.
[0158] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0159] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A camera calibration method, characterized in that: The method comprises: Acquire a first calibration image collected at multiple viewing angles for an L-shaped luminous calibration object, and a second calibration image collected at the multiple viewing angles for a T-shaped luminous calibration object; Identify a first feature point where each first luminous mark of the L-shaped luminous calibration object is located from the first calibration image; Obtaining initial camera parameters according to the first feature point calibration under the multiple viewing angles; Identify the second feature point where each second luminous mark of the T-shaped luminous calibration object is located from the second calibration image; Determine the three-dimensional distance between each of the second luminous marks using the initial camera parameters and the second feature point positions at the same viewing angle; The initial camera parameters are calibrated according to the difference between the physical distance between each of the second luminous marks and the three-dimensional distance to obtain target camera parameters.
2. The method according to claim 1, characterized in that The step of calibrating the initial camera parameters according to the difference between the physical distance between each of the second luminous markers and the three-dimensional distance to obtain target camera parameters includes: Calibrate the initial camera parameters using the difference between the physical distance and the three-dimensional distance to obtain calibrated camera parameters; Determine the three-dimensional distance between each of the second luminous marks using the calibrated camera parameters and the second feature points at the same viewing angle; When the difference between the physical distance and the three-dimensional distance corresponding to the calibrated camera parameters is within a preset range, the calibrated camera parameters are used as the target camera parameters.
3. The method according to claim 1, characterized in that The initial camera parameters include initial camera extrinsics; The obtaining of initial camera parameters according to the first feature point calibration under the multiple viewing angles includes: Read the pre-stored initial camera internal parameters; Constructing a three-dimensional model of the L-shaped luminous calibration object according to the shape characteristics of the L-shaped luminous calibration object; The initial camera extrinsic parameters are determined using the three-dimensional model, the first feature point and the initial camera intrinsic parameters.
4. The method according to claim 3, characterized in that The determining the three-dimensional distance between each of the second luminous marks by using the initial camera parameters and the second feature points under the same viewing angle includes: Projecting the second feature point under the same viewing angle into three-dimensional space using the initial camera extrinsic parameter to obtain a projected feature point corresponding to the second feature point; The distance between the projection feature points in the three-dimensional space is determined, and the distance between the projection feature points is used as the three-dimensional distance between the corresponding second luminous marks.
5. The method according to claim 3, characterized in that: The determining the initial camera extrinsic parameters by using the three-dimensional model, the first feature point and the initial camera intrinsic parameters includes: Constructing a basic matrix using the three-dimensional model, the first feature point and the initial camera intrinsic parameters; Using the basic matrix and the first feature point at the first viewing angle, draw an epipolar line corresponding to the first feature point at the second viewing angle; When it is determined that the first feature point under the second viewing angle matches the epipolar line, the basic matrix is decomposed to obtain a corresponding initial rotation vector and translation vector; The initial rotation vector and the translation vector are used as the initial camera extrinsic parameters.
6. The method according to claim 1, characterized in that The step of identifying, from the first calibration image, a first feature point where each first luminous mark of the L-shaped luminous calibration object is located comprises: Performing grayscale conversion and Gaussian blur processing on the first calibration image to obtain a processed grayscale calibration image; Binarizing the grayscale calibration image to obtain a corresponding binary image; Performing corrosion and dilation processing on the binary image to obtain a processed target binary image; Performing contour detection on the target binary image to obtain a corresponding contour detection result; Selecting a target contour whose contour area meets a preset condition from the contour detection results; The first feature point position is determined according to the centroid position of the target contour.
7. A camera calibration device, characterized in that: The device comprises: An image acquisition module, used to acquire first calibration images collected at multiple viewing angles for the L-shaped luminous calibration object, and second calibration images collected at the multiple viewing angles for the T-shaped luminous calibration object; An initial calibration module, configured to identify, from the first calibration image, a first feature point where each first luminous mark of the L-shaped luminous calibration object is located; and obtain initial camera parameters according to the first feature point calibration under the multiple viewing angles; A calibration module is used to identify the second feature point position of each second luminous mark of the T-shaped luminous calibration object from the second calibration image; determine the three-dimensional distance between each second luminous mark using the initial camera parameters and the second feature point position under the same viewing angle; calibrate the initial camera parameters according to the difference between the physical distance between each second luminous mark and the three-dimensional distance to obtain the target camera parameters.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.