Infrared binocular camera calibration method, device, equipment and medium

Through infrared binocular camera calibration method, concentric circle identification and elliptical fitting technology, the binocular camera calibration method is easily affected by the quality and layout of the calibration plate, and the calibration accuracy and robustness are improved, achieving more efficiency calibration process.

CN118941646AActive Publication Date: 2024-11-12GUANGZHOU AIMUYI TECH CO LTD

Patent Information

Application Number
CN202410996333.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-24
Publication Date
2024-11-12
Estimated Expiration
2044-07-24

AI Technical Summary

Technical Problem

The existing binocular camera calibration methods are susceptible to the quality and layout of the calibration plate, resulting in a decrease in the accuracy of calibration results.

Method used

Using infrared binocular camera calibration method, the marking points under the internal coordinate system are obtained by obtaining the images of the marking plate at different object distances, and linear fitting and sorting of concentric circle markings, elliptical contour fitting, combined with averaging processing. The coordinates are matched with the calibration board's real reference system through the camera's internal coordinate system to optimize the calibration results.

Benefits of technology

Improve the redundancy and stability of calibration, enhance the generalization ability of the system, and can more accurately determine the center position under camera distortion, calibration plate tilt or light changes, improving the accuracy and freedom of marking point detection It is good, significantly improving the accuracy and efficiency of the calibration process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941646A_ABST
    Figure CN118941646A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of infrared binocular camera calibration, and discloses an infrared binocular camera calibration method and device, equipment and a medium, and the method comprises the steps: obtaining a preset number of shot images of a marking plate under different object distances; carrying out straight line fitting and sorting on concentric circle identifiers on the shot image; performing ellipse contour fitting on the concentric circle identification in the shot image to obtain an ellipse center; carrying out averaging processing on the ellipse center to obtain a center final coordinate of the concentric circle identifier; sorting based on the final coordinates of the circle centers to obtain coordinates of mark points in an internal coordinate system of the camera; and based on the coordinates of the mark points, matching an internal coordinate system of the camera with a real reference system of the calibration plate to obtain a calibration result. According to the method, the detection influence caused by the shape change of the mark point due to factors such as camera distortion, calibration plate inclination or illumination change can be reduced, and the accuracy and robustness of mark point detection are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of infrared binocular camera calibration, and in particular to an infrared binocular camera calibration method, device, equipment and medium. Background Art

[0002] Binocular cameras are widely used in many applications, such as stereo vision, 3D reconstruction, and visual measurement. Binocular cameras can simulate human binocular vision and calculate the depth information of objects in the scene through image data obtained from two perspectives, thereby realizing 3D visual perception. It helps to accurately superimpose virtual objects into the real world, and improves the realism and accuracy of augmented reality (AR) and virtual reality (VR) applications. At present, calibration methods are mainly divided into two categories. One is the method based on geometry: such as using specific patterns such as chessboards for calibration, calculating the camera's intrinsic parameters (such as focal length, principal point position) and extrinsic parameters (such as rotation and translation). The other is the method based on feature points: matching and calculating the transformation relationship between cameras by detecting feature points in the image (such as corner points, SIFT feature points). Modern binocular camera calibration methods usually need to achieve high accuracy and stability to meet the needs of complex applications, such as medical imaging, precision manufacturing, etc. For example, the traditional Zhang Zhengyou calibration method (Zhang's method), as a classic binocular camera calibration method, uses the camera's internal and external parameters and distortion parameters to describe the camera's geometric characteristics. Although this method is widely used and has many successful applications, it also has some disadvantages and limitations. For example, the quality and layout of the calibration plate have a great impact on the calibration results. If the calibration plate is of poor quality or is not placed correctly, the accuracy of the calibration results may decrease.

[0003] Therefore, the existing binocular camera calibration method is greatly affected by the quality and layout of the calibration plate, which easily leads to a decrease in accuracy and needs to be solved urgently. Summary of the invention

[0004] The main purpose of this application is to provide an infrared binocular camera calibration method, which aims to solve the technical problem that the existing binocular camera calibration method is greatly affected by the quality and layout of the calibration board, which easily leads to a decrease in accuracy.

[0005] The first aspect of the present application provides an infrared binocular camera calibration method, comprising:

[0006] Acquire a preset number of photographed images of the marking plate at different object distances;

[0007] Perform straight line fitting and sorting on the concentric circle marks on the captured image;

[0008] Fit the concentric circle marks in the captured image to an ellipse contour to obtain the center of the ellipse;

[0009] The ellipse centers are averaged to obtain the final coordinates of the centers of the concentric circles;

[0010] Sorting based on the final coordinates of the circle center obtains the coordinates of the marker points in the camera's internal coordinate system;

[0011] Based on the coordinates of the marking points, the internal coordinate system of the camera is matched with the real reference system of the calibration board to obtain the calibration result.

[0012] Furthermore, the concentric circle marks are composed of a preset number of concentric circles; and the step of performing straight line fitting and sorting on the concentric circle marks on the captured image comprises:

[0013] Use circle detection algorithm to detect and extract the center coordinate information of concentric circles;

[0014] Calculate the center coordinate slope and distance between concentric circles based on the center coordinate information to perform straight line fitting;

[0015] According to the straight line fitting results, the distance from each concentric circle to the fitting line is calculated and the concentric circles are sorted.

[0016] Furthermore, before the step of fitting the concentric circle marks in the captured image with an ellipse contour to obtain the center of the ellipse, the step includes: performing image preprocessing on the sorted captured image, and the image preprocessing step includes:

[0017] Gray value and filter denoising of the image;

[0018] Convert the image into a binary image and perform morphological processing on the image.

[0019] Furthermore, the step of fitting the concentric circle marks in the captured image to an ellipse contour to obtain the center of the ellipse includes:

[0020] Extract edge information from images based on edge detection algorithms;

[0021] According to the number of concentric circles, the contours that meet the corresponding conditions are selected in turn;

[0022] Perform ellipse fitting on the selected contour based on the ellipse fitting algorithm to obtain ellipse parameters;

[0023] The coordinates of the center of the ellipse are extracted from the ellipse parameters, that is, the center of the ellipse.

[0024] Furthermore, the step of averaging the ellipse centers to obtain the final coordinates of the centers of the concentric circle marks includes:

[0025] Collect the coordinates of the center of each ellipse contained in the concentric circle identifier to form a coordinate set;

[0026] The center coordinates of the ellipses in the coordinate set are averaged to obtain the final center coordinates of the concentric circle markers.

[0027] Furthermore, the step of obtaining the coordinates of the marking points in the camera internal coordinate system by sorting based on the final coordinates of the circle center includes:

[0028] Determine the coordinate system inside the camera and transform the final coordinates of the center of the circle into the coordinate system inside the camera;

[0029] According to the final coordinates of the circle center that have been converted to the camera internal coordinate system, calculate the camera internal coordinate value of each final coordinate of the circle center;

[0030] Sort the coordinate values ​​of the marker points in the camera's internal coordinate system to obtain the coordinates of the marker points in the camera's internal coordinate system.

[0031] Furthermore, the step of obtaining a calibration result by matching the internal coordinate system of the camera with the real reference system of the calibration board based on the coordinates of the marking points includes:

[0032] Use the space transformation matrix to transform the coordinates of the marker points in the camera's internal coordinate system to the calibration board's real reference system;

[0033] Compare the coordinates of the marking points in the real reference system of the calibration plate with the coordinates of the real marking points in the real reference system of the corresponding calibration plate to obtain a comparison result;

[0034] Based on the comparison results, the error between the coordinates of the marked points and the actual coordinates of the marked points is calculated, and the error is minimized based on the preset algorithm to obtain the optimization result;

[0035] Data extraction is performed based on the optimization result to obtain the calibration result.

[0036] The second aspect of the present application provides an infrared binocular camera calibration device, comprising:

[0037] An image acquisition module, used for acquiring a preset number of photographed images of the marking plate at different object distances;

[0038] A fitting and sorting module is used to perform straight line fitting and sorting on the concentric circle marks on the captured image;

[0039] The ellipse fitting module is used to fit the concentric circle marks in the captured image to the ellipse contour and obtain the center of the ellipse;

[0040] An averaging processing module is used to average the centers of the ellipses to obtain the final coordinates of the centers of the concentric circles;

[0041] A sorting module is used to sort based on the final coordinates of the circle center to obtain the coordinates of the marking points in the camera's internal coordinate system;

[0042] The matching module is used to match the coordinates of the marking points through the internal coordinate system of the camera and the real reference system of the calibration board to obtain a calibration result.

[0043] The third aspect of the present application provides an infrared binocular camera calibration device, comprising:

[0044] processor;

[0045] Memory for storing computer programs;

[0046] Wherein, when the processor executes the computer program, the infrared binocular camera calibration method described in any one of the above is implemented.

[0047] A fourth aspect of the present application provides a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the infrared binocular camera calibration method described in any one of the above is implemented.

[0048] Beneficial effects:

[0049] This application increases the number and complexity of calibration points by using concentric circle identification plates instead of traditional solid circle marking plates, thereby improving the redundancy and stability of calibration. Image acquisition at different object distances ensures the applicability of the calibration results at various working distances and enhances the generalization ability of the system. Through ellipse contour fitting and circle center averaging processing, the center position of the circle can be determined more accurately even in the case of camera distortion and calibration plate tilt, and it is more resistant to changes in the shape of the marking points caused by factors such as camera distortion, calibration plate tilt or illumination changes, thereby improving the accuracy and robustness of marking point detection. The use of multiple concentric circles and ellipse fitting for each circle enable the system to maintain good performance in the presence of noise and partial occlusion. The straight line fitting and sorting steps ensure that each marking point can be correctly identified and processed even in the case of image deformation. In summary, this solution can significantly improve the accuracy, robustness and efficiency of the calibration process. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is a schematic diagram of the steps of a method for calibrating an infrared binocular camera according to an embodiment of the present application;

[0051] Figure 2 It is a schematic block diagram of the structure of an infrared binocular camera calibration device according to an embodiment of the present application;

[0052] Figure 3 is a schematic block diagram of the structure of a computer device according to an embodiment of the present application;

[0053] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] It will be appreciated by those skilled in the art that, unless expressly stated, the singular forms "one", "above", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of features, integers, steps, operations, elements, modules, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, modules, components, and / or groups thereof. It should be understood that when an element is said to be "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may also be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any module and all combinations of one or more associated listed items.

[0056] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0057] Reference Figure 1 The present application embodiment provides an infrared binocular camera calibration method, comprising the following steps S1-S6:

[0058] S1, obtaining a preset number of photographed images of the marking plate at different object distances;

[0059] S2, performing straight line fitting and sorting on the concentric circle marks on the captured image;

[0060] S3, fitting the concentric circle marks in the captured image to an ellipse contour to obtain the center of the ellipse;

[0061] S4, averaging the ellipse centers to obtain the final coordinates of the centers of the concentric circles;

[0062] S5. Sorting based on the final coordinates of the circle center to obtain the coordinates of the marking points in the camera's internal coordinate system;

[0063] S6. Based on the coordinates of the marking points, the internal coordinate system of the camera is matched with the real reference system of the calibration board to obtain a calibration result.

[0064] In step S1, a designed marker plate is used, which has a preset number of concentric circle marks, such as 50; each concentric circle mark is composed of a preset number of concentric circles, for example, 3 concentric circles of different sizes. The marker plate should be placed in a flat and evenly lit environment, avoiding shadows and direct strong light to ensure image quality. According to the calibration requirements, different object distances are set, such as 1 meter, 1.3 meters, 1.8 meters and 2 meters. The selection of these distances should take into account the working range of the camera and the expected calibration accuracy. Using the infrared binocular camera to be calibrated, a certain number of images, such as 35, are taken at each set object distance. Diversified image data helps to estimate camera parameters more accurately, especially when dealing with non-ideal conditions (such as camera distortion or calibration plate tilt); when shooting, the camera should be stably fixed to avoid movement or vibration to ensure image clarity and consistency. In order to improve the robustness and accuracy of the calibration, the marker plate should be photographed from different angles and positions to capture the performance of the marker plate in different postures. This helps to estimate the internal and external parameters of the camera more accurately in subsequent steps. Step S1 is the starting point of the entire calibration process and can provide raw data for subsequent image processing and analysis. The captured image will be used to detect and locate the marking points on the marking plate, and then calculate the camera parameters.

[0065] In step S2, the captured image can be preprocessed, including grayscale, Gaussian filter denoising, binarization and morphological processing, etc., to enhance image contrast, reduce noise, and highlight concentric circle marks; use a circle detection algorithm (such as Hough transform or edge detection combined with a mathematical model of a circle) to detect the concentric circle marks in the image. The detected central coordinate information will be the basis for subsequent straight line fitting. The slope between the central coordinates of adjacent marking points is calculated, and these slopes and coordinate information are used to perform straight line fitting on the concentric circle marks. Straight line fitting can be achieved by the least squares method or other statistical methods, with the aim of finding a straight line that best represents the arrangement trend of the marking points. Straight line fitting can help locate the position of the marking points more accurately, especially when the shape of the marking points in the image may be distorted due to camera distortion. Then, the concentric circle marks are sorted according to the straight line fitting results. The sorting can be based on the distance between the concentric circle marks and the fitting straight line, or the central coordinate information of the concentric circle marks, to ensure that their order is correct, which is convenient for subsequent ellipse fitting and coordinate calculation. Sorting ensures that the processing of concentric circle marks is consistent in subsequent steps. Correct sorting ensures the smooth progress of subsequent steps (such as ellipse fitting and coordinate calculation), ensuring the consistency and effectiveness of the entire calibration process. This step is the first step to extract useful information from image data, laying the foundation for subsequent coordinate calculation and calibration parameter optimization. Through straight line fitting and sorting, the correct correspondence of each mark point can be ensured, which is crucial for accurate calculation of mark point coordinates and subsequent calibration.

[0066] In step S3, edge information can be extracted from the preprocessed image using Canny edge detection, Sobel operator or other edge detection algorithms, which helps to identify the boundaries of the concentric circles. Based on the edge information, the contour detection function in the computer vision library such as OpenCV (OpenSource Computer Vision Library) is used to identify and filter the contours corresponding to the concentric circle marks. Note that these contours may be elliptical due to camera distortion. For each detected contour, an ellipse fitting algorithm (such as least squares or RANSAC (random sampling consensus algorithm)) is applied to fit an optimal ellipse to match the actual shape of the contour. The goal of ellipse fitting is to find an ellipse equation that can minimize the sum of squares of the distances between the ellipse and the edge points of the contour. From the ellipse parameters obtained by fitting, the center coordinates of the ellipse are extracted. Assuming that each concentric circle mark is actually composed of three concentric circles, three ellipse center coordinates will be obtained for each concentric circle mark. Compared with directly using the circle detection result, ellipse fitting can more accurately reflect the true shape of the mark point, especially when the camera distortion is large, which can significantly improve the accuracy of the center coordinates. Ellipse fitting is more adaptable to changes in the shape of concentric circle markers, and can obtain reliable coordinates of the circle center even when the markers are slightly deformed or partially occluded. By improving the accuracy of marker point detection in the early stages, the error accumulation in subsequent steps can be reduced, thereby improving the overall accuracy of the entire calibration process.

[0067] In step S4, the coordinates of the centers of the N ellipses of the same concentric circle mark are averaged to obtain a more accurate and stable coordinate of the center. For each concentric circle mark, the x-coordinates and y-coordinates of the centers of the three ellipses are averaged, that is, the average values ​​of the N x-coordinates and the N y-coordinates, so as to obtain a new x-coordinate and y-coordinate, which will be used as the final coordinates of the center of the concentric circle mark. This step is processed by averaging, which reduces the coordinate error caused by camera distortion, calibration plate tilt or other factors, thereby providing more accurate basic data for subsequent coordinate sorting and calculation of the space transformation matrix. Averaging can reduce coordinate errors, and even when a single ellipse fitting is affected by noise or distortion, the averaged coordinates can be closer to the true center position. That is, in the case where some ellipse fitting results are not ideal, by fusing the information of multiple ellipse centers, relatively accurate center coordinates can be provided, thereby improving the robustness of coordinate calculation.

[0068] In step S5, these final center coordinates are sorted, and the purpose of sorting is to ensure the uniqueness and correctness of each mark point in subsequent processing. Sorting can be based on the geometric position of the mark point or the known mark plate layout, for example, in a left-to-right, top-to-bottom order. Create a list of mark point coordinates, which contains all the converted and sorted mark point coordinates. These coordinates will provide data support for the next step of spatial transformation matrix calculation in a unified format and order. This step converts the results of the previous image processing and feature point detection into coordinate data that can be directly used for parameter estimation, ensuring the accuracy and consistency of subsequent calibration calculations. Convert to the camera's internal coordinate system. This conversion usually involves the use of the camera's internal parameter matrix (including parameters such as focal length and principal point position) and possible distortion correction. A standardized conversion from image feature points to camera internal coordinate system mark point coordinates is achieved, providing a high-quality data basis for the final calibration result.

[0069] In step S6, the output of all previous steps, i.e., the coordinates of the marker points precisely ordered in the camera internal coordinate system, are matched with the real reference coordinate system of the calibration plate to determine the final calibration result. The coordinates of the marker points in the camera internal coordinate system obtained in step S5 are paired with the known real world coordinates on the calibration plate (the real reference coordinate system of the calibration plate). Using these corresponding point pairs, the internal and external parameters of the camera can be estimated by the least squares method or other optimization algorithms. These parameters include internal parameters such as the focal length, principal point position, pixel size, radial distortion coefficient of the camera, and external parameters such as rotation and translation vectors. Parameter estimation is usually a nonlinear optimization problem because the distortion equation of the camera model is nonlinear. Therefore, it is necessary to use iterative optimization techniques such as the Levenberg-Marquardt algorithm to solve it in order to minimize the reprojection error. In addition, after completing the parameter estimation, the accuracy of the calibration can be further checked. Specifically, it can be verified by reprojecting the known world coordinates (the real reference coordinate system of the calibration plate) to the image plane and then comparing the distance between these reprojected points and the actually detected marker points. This step effectively solves the calibration problem by matching the coordinates of the marker points in the camera's internal coordinate system with the real reference coordinates of the calibration plate.

[0070] In one embodiment, the concentric circle marks are composed of a preset number of concentric circles; the step of performing straight line fitting and sorting on the concentric circle marks on the captured image includes:

[0071] S10, using a circle detection algorithm to detect and extract the center coordinate information of the concentric circles;

[0072] S11, calculating the center coordinate slope and distance between concentric circles based on the center coordinate information to perform straight line fitting;

[0073] S12. According to the straight line fitting result, the distance between each concentric circle and the fitting straight line is calculated, and the concentric circles are sorted.

[0074] In this embodiment, the concentric circle marks are processed to ensure that these concentric circles can be accurately and efficiently identified and used during the camera calibration process. First, a circle detection algorithm is used to detect and extract the center coordinate information of the concentric circles. Because the center coordinates of the concentric circles are the key to determining the relative positional relationship between them. The circle detection algorithm finds edge points in the image and fits these points to a circle model to obtain the center coordinates and radius of the circle. This method is very effective for the recognition of concentric circle marks because it can accurately locate the center of each concentric circle. Next, based on the center coordinate information of the concentric circles, the center coordinate slope and distance between them are calculated to perform straight line fitting. Since the concentric circles are arranged according to certain rules, their center coordinates will approximate a straight line. By calculating the slope and distance between the center coordinates, the parameters of this line can be obtained, and the least squares method and other methods are used for fitting. The purpose of this step is to determine the relative positional relationship between the concentric circles, providing a basis for subsequent sorting and matching. Finally, according to the result of the straight line fitting, the distance from each concentric circle to the fitted straight line is calculated, and the concentric circles are sorted accordingly. The purpose of this step is to arrange the detected concentric circles in the order in which they appear on the calibration plate so that they can be subsequently matched with the real reference coordinate system of the calibration plate. Since the relative position relationship between the concentric circles has been determined by straight line fitting, they can be sorted according to their distance to the fitting line. This method is not only simple and effective, but also ensures the accuracy of the sorting.

[0075] In summary, steps S10 to S12 provide an accurate data basis for the subsequent calibration process by accurately identifying and processing the concentric circle marks. These steps not only improve the accuracy of the calibration, but also enhance the robustness of the system, making the entire calibration process more reliable and efficient.

[0076] In one embodiment, before the step of fitting the concentric circle marks in the captured image to obtain the center of the ellipse, the process includes: performing image preprocessing on the sorted captured image, and the image preprocessing step includes:

[0077] S20, graying and filtering the image to remove noise;

[0078] S21, converting the image into a binary image and performing morphological processing on the image.

[0079] In this embodiment, image preprocessing for the captured images after sorting is an important step to ensure the accuracy of subsequent ellipse contour fitting and circle center calculation. Image preprocessing can not only simplify image data and highlight target information, but also effectively eliminate or reduce noise and interference, providing a high-quality data source for subsequent analysis and processing. First, in step S20, the image is grayed. This is because the original image is usually in color and contains a large amount of color information, which is not necessary for contour recognition and will increase the complexity and amount of calculation of the processing. Graying is the process of converting a color image into a grayscale image. By calculating the average or weighted average of the three color components of red, green and blue for each pixel, a grayscale value is obtained to form a grayscale image. The grayed image only contains brightness information, which is convenient for subsequent processing and analysis. Next, in step S20, the grayed image is filtered and denoised. Due to the influence of the shooting environment and equipment, there may be noise and interference in the image, which will affect the recognition of the concentric circle logo and the calculation of the center of the circle. Filtering and denoising is the process of removing or reducing noise and interference by processing the image through a certain algorithm or filter. In this embodiment, Gaussian filtering, median filtering and other algorithms can be used for filtering and denoising to improve the clarity and quality of the image. Here, Gaussian filtering is taken as an example. The convolution operation based on the Gaussian function can effectively blur the image, making the image smoother and more continuous.

[0080] Gaussian function: (Wherein, (G(x,y)) is the probability distribution of the point (x,y) in the two-dimensional space, which is used to define a filter to blur the noise in the image; σ is the standard deviation of the Gaussian function, which determines the distribution range and peak value of the function); Gaussian filter operator: The Gaussian filter operator convolves the image with the Gaussian function, and its discrete form is Among them, (H(i,j)) is the operation of convolving the Gaussian function with the image, the purpose of which is to smooth the image; i and j are the coordinates of the convolution kernel, which are usually odd numbers, such as 5×5, to maintain central symmetry. Then, in step S21, the filtered and denoised image is converted into a binary image. Use a global threshold or an adaptive threshold method to convert the grayscale image into a binary image. The global threshold sets a fixed grayscale threshold, and pixels above this threshold are set to white (foreground), and pixels below this threshold are set to black (background). The adaptive threshold takes into account the brightness differences in local areas and is more suitable for processing images with uneven lighting. In this embodiment, a suitable threshold can be set according to the grayscale characteristics of the concentric circle logo, and the concentric circle logo can be separated from the background to form a binary image. Finally, in step S21, the binary image is morphologically processed. Morphological processing is an image processing technology based on mathematical morphology, which can be used for image segmentation, extraction, smoothing and other operations. In this embodiment, morphological operations such as expansion and erosion can be used to process the binary image to eliminate the holes and noise inside the concentric circle logo and smooth the edges of the concentric circle logo, so as to provide high-quality image data for the subsequent ellipse contour fitting and circle center calculation. For example, for three layers of concentric circles, since the focus is on three concentric circles, the focus is on finding three contours of decreasing size; once the three contours are identified, different thresholds or grayscale levels can be set for each circle layer according to their position and size. The outermost layer (the largest contour) can be set as a bright area (white), the middle layer (the medium-sized contour) can be set as a dark area (black), and the innermost layer (the smallest contour) can be set as a bright area (white) again. For each identified circle layer, the corresponding grayscale or binary threshold is applied to obtain a pattern of light-dark-light from the outside to the inside. After applying the above steps, morphological operations (opening and closing operations) are required to eliminate noise or disconnected boundaries to ensure the integrity and clarity of each circle layer.

[0081] In summary, this embodiment can effectively handle the situation of uneven illumination through gray value conversion, filtering denoising, binarization and morphological processing, and by combining adaptive threshold and image segmentation technology, and ensure that the light and dark levels in the concentric circle logo are accurately distinguished, thereby providing high-quality image input for subsequent feature detection and calibration. It provides an accurate and reliable data basis for the subsequent ellipse contour fitting and circle center calculation. This step not only improves the accuracy of calibration, but also enhances the robustness of the system, making the entire calibration process more reliable and efficient.

[0082] In one embodiment, the step of fitting the concentric circle marks in the captured image to an ellipse contour to obtain the center of the ellipse includes:

[0083] S30, extracting edge information in the image based on an edge detection algorithm;

[0084] S31, according to the number of concentric circles, select contours that meet corresponding conditions in sequence;

[0085] S32, performing ellipse fitting on the selected contour based on an ellipse fitting algorithm to obtain ellipse parameters;

[0086] S33. Extracting the coordinates of the center of the ellipse, that is, the center of the ellipse, from the ellipse parameters.

[0087] In this embodiment, edge detection is an important step in image processing for identifying areas with significant brightness changes in an image. In this step, the Canny edge detection algorithm is used, which detects edges by calculating the gradient intensity and direction of each pixel in the image. The advantage of the Canny algorithm is that it can simultaneously meet the requirements of low error rate, high positioning accuracy and single edge response, thereby ensuring that the edge information of the concentric circle logo can be accurately extracted from the complex image background. After extracting the edge information, it is necessary to screen out the contour corresponding to the concentric circle logo from many contours. Through the previous morphological processing and binarization steps, three contours of decreasing size have been obtained, corresponding to the three layers of concentric circles. In this step, corresponding thresholds or conditions, such as the size and shape of the contour, are set to screen out these three specific contours. Once the correct contour is screened out, the ellipse fitting algorithm is used to calculate the ellipse parameters corresponding to each contour. Here, the least squares method is selected for ellipse fitting because it can minimize the fitting error, thereby obtaining more accurate ellipse parameters. Through ellipse fitting, parameters such as the center position, major and minor axis lengths, and rotation angles of each concentric circle can be obtained. The last step is to extract the center coordinates of the ellipse from the ellipse parameters. By extracting the center coordinates of the circle, the center position of the concentric circle logo can be accurately calculated, providing an accurate and reliable data basis for subsequent feature detection and calibration.

[0088] In summary, steps S30 to S33 can accurately extract the center coordinates of the concentric circle markers through precise edge detection, contour screening and ellipse fitting algorithms, providing high-quality image input for subsequent feature detection and calibration, thereby improving the accuracy of calibration and the robustness of the system.

[0089] In one embodiment, the step of averaging the ellipse centers to obtain the final coordinates of the centers of the concentric circle marks includes:

[0090] S40, collecting the coordinates of the center of each ellipse contained in the concentric circle identifiers to form a coordinate set;

[0091] S41, averaging the center coordinates of the ellipses in the coordinate set to obtain final center coordinates of the concentric circle markers.

[0092] In this embodiment, it is ensured that the center position of the circle can be obtained with high precision even when the image is distorted or the shooting angle is not correct. After performing ellipse fitting and obtaining the center coordinates of each ellipse, these coordinates are collected to form a coordinate set. This is achieved by traversing all identified ellipses, reading and storing the center coordinates (x, y) of each ellipse. These coordinates may deviate due to changes in shooting angle, lens distortion or lighting conditions, but they all point to the same physical center. The primary function of collecting the coordinate set is to ensure that there is information about the centers of all ellipses, so that all data points can be used in the subsequent averaging process instead of relying on the measurement results of a single ellipse. This improves the stability and reliability of the final calculation result, because the center of a single ellipse may have a large deviation due to poor local image quality or algorithm errors. After obtaining the coordinate set of all ellipse centers, they are averaged to calculate a final center coordinate that integrates the information of all ellipse centers. The averaging process is achieved by calculating the arithmetic mean of the x and y coordinates in all coordinate sets. Specifically, all x-coordinates are averaged, and all y-coordinates are averaged, and the results are the final center coordinates. The main function of averaging is to reduce measurement errors and improve calibration accuracy. Since the center coordinates of each ellipse are calculated independently, they may be affected by various factors and offset. By calculating the average value, this random error can be offset and an estimate closer to the true center position can be obtained. In addition, this method is also robust. Even if the fitting results of some ellipses are greatly affected by noise, the final average coordinates can still maintain a high accuracy.

[0093] In one embodiment, the step of obtaining the coordinates of the marking points in the camera internal coordinate system by sorting based on the final coordinates of the circle center includes:

[0094] S50, determining the coordinate system inside the camera, and converting the final coordinates of the center of the circle into the coordinate system inside the camera;

[0095] S51, calculating the camera internal coordinate value of each circle center final coordinate according to the circle center final coordinate converted to the camera internal coordinate system;

[0096] S52, sorting according to the coordinate values ​​of the marking points in the camera's internal coordinate system to obtain the coordinates of the marking points in the camera's internal coordinate system.

[0097] In this embodiment, first, the internal coordinate system of the camera needs to be clarified. The internal coordinate system of the camera is usually established with the optical axis of the camera as the z-axis, the horizontal direction of the image plane as the x-axis, and the vertical direction as the y-axis. The origin of the coordinates is located at the center of the image, but in actual applications, the origin of the coordinate system may also be the upper left corner of the image, depending on the camera model and coordinate system definition used. Once the coordinate system is determined, the final coordinates of the center of the circle are then converted to the internal coordinate system of the camera. Converting the final coordinates of the center of the circle to the internal coordinate system of the camera can ensure that all subsequent calculations and analyses are performed under a unified coordinate framework, avoiding the accumulation of errors caused by inconsistent coordinate systems. After the coordinate conversion is completed, the value of the final coordinates of each center of the circle in the internal coordinate system of the camera is calculated. This process usually involves simple linear algebra operations, that is, substituting the converted coordinate values ​​into the mathematical expression of the camera model to obtain the internal coordinate values ​​of each center of the circle. Finally, the center of the circle coordinate values ​​calculated in the internal coordinate system of the camera are sorted. The basis for sorting can be the size of the coordinate value, the position relationship or other logical rules, the purpose is to facilitate subsequent processing and analysis. This embodiment converts the center coordinates into the camera internal coordinate system and sorts them, so that the camera parameters can be estimated and calibrated more effectively, thereby improving the performance and reliability of the overall system.

[0098] In one embodiment, based on the coordinates of the marking points, the step of obtaining the calibration result by matching the internal coordinate system of the camera with the real reference system of the calibration plate includes:

[0099] S60, using a space transformation matrix, transforming the coordinates of the marker points in the camera internal coordinate system into the real reference system of the calibration plate;

[0100] S61, comparing the coordinates of the marking points in the real reference system of the calibration plate with the coordinates of the real marking points in the real reference system of the corresponding calibration plate to obtain a comparison result;

[0101] S62, based on the comparison result, calculating the error between the coordinates of the marking point and the real coordinates of the marking point, and minimizing the error based on a preset algorithm to obtain an optimization result;

[0102] S63. Perform data extraction based on the optimization result to obtain the calibration result.

[0103] In this embodiment, a spatial transformation matrix, i.e., a rigid body transformation matrix, is used to transform the coordinates of the marker points in the camera's internal coordinate system into the calibration board's real reference system. This transformation matrix contains rotation and translation parameters, and can accurately describe the relative position and direction between the camera coordinate system and the world coordinate system (calibration board's real reference system). The transformation process involves multiplying each marker point coordinate by the transformation matrix to map it from the camera coordinate system to the calibration board coordinate system.

[0104] Perform corner detection on each image, assuming that the camera's internal parameters (such as focal length f x 、f y and principal point offset c x 、c y ), as well as external parameters (rotation matrix R and translation vector t) and distortion parameters (radial distortion coefficients k1, k2, tangential distortion coefficients p1, p2). Nonlinear optimization methods (such as least squares) are used to optimize the camera's internal and external parameters, as well as distortion parameters, by minimizing the reprojection error (the error between the actual corner points and the calculated corner points).

[0105] The camera projects a point (X, Y, Z) in three-dimensional space to a point (u, v) on the image plane. The projection process can be described by the following formula:

[0106]

[0107] It can be seen that the camera projection matrix P is composed of internal parameters (focal length fx, fy, principal point offset cx, cy), external parameters (rotation matrix R and translation vector t) and distortion parameters. Through this matrix, we can project the point (X, Y, Z) in the three-dimensional space to (u, v) on the image plane, that is, convert the coordinates of the marker points in the camera's internal coordinate system to the real reference system of the calibration plate. The purpose of this conversion process is to establish a connection between the camera coordinate system and the real-world coordinate system, so that we can compare the positions of the marker points observed by the camera with their known positions in the real world. This lays the foundation for subsequent error calculation and parameter optimization.

[0108] By comparing the coordinates of the marker points after the camera coordinate system is transformed with the coordinates of the actual position of the marker points on the calibration board, we can quantify the deviation between the camera observation and the actual value by comparing these two sets of coordinates. The actual coordinates of the marker points are pre-measured and known, and they constitute the "ground truth" in the calibration process. In the projection formula, we can see that the camera projects the point (X, Y, Z) in the three-dimensional space to (u, v) on the image plane. By comparing the projected (u, v) with the actual marker coordinates, we can get the projection error, which is the difference between the camera observation and the actual value. The comparison results reveal the accuracy of the camera calibration parameters, that is, whether the camera internal parameters (such as focal length, principal point position) and external parameters (such as rotation and translation matrices) are correct. This comparison is the basis for evaluating the calibration effect and optimizing parameters. Based on the comparison results obtained in S61, we calculate the error between each marker coordinate and its actual position. These errors are usually expressed as distances (such as Euclidean distances) and reflect the deviation between the camera observation and the actual position. Subsequently, nonlinear optimization methods (such as the Levenberg-Marquardt algorithm) are used to adjust the camera parameters to minimize the total error until a preset convergence criterion is reached.

[0109] By minimizing the projection error, we can update these parameters to make them closer to the true values, thereby obtaining optimized camera parameters. After obtaining the optimization results, we extract the final camera parameters from these results, including internal parameters (such as focal length, principal point offset, radial distortion coefficients k1, k2, tangential distortion coefficients p1, p2) and external parameters (rotation and translation matrices). These parameters constitute the final calibration results, which can be used to correct image distortion, enhance the accuracy of 3D reconstruction, and support other advanced visual processing tasks.

[0110] refer to Figure 2 , an infrared binocular camera calibration device, comprising:

[0111] An image acquisition module 100 is used to acquire a preset number of photographed images of the marking plate at different object distances;

[0112] A fitting and sorting module 200 is used to perform straight line fitting and sorting on the concentric circle marks on the captured image;

[0113] The ellipse fitting module 300 is used to fit the concentric circle marks in the captured image to the ellipse contour and obtain the center of the ellipse;

[0114] An averaging processing module 400 is used to average the ellipse centers to obtain the final coordinates of the centers of the concentric circle marks;

[0115] A sorting module 500, used for sorting based on the final coordinates of the circle center to obtain the coordinates of the marking point in the camera internal coordinate system;

[0116] The matching module 600 is used to match the camera's internal coordinate system with the calibration board's real reference system based on the marker point coordinates to obtain a calibration result.

[0117] Furthermore, the above-mentioned fitting sorting module 200 includes:

[0118] A detection unit, used to detect and extract the center coordinate information of the concentric circles using a circle detection algorithm;

[0119] A first fitting unit, used for calculating the center coordinate slope and distance between concentric circles based on the center coordinate information to perform straight line fitting;

[0120] The first sorting subunit is used to calculate the distance between each concentric circle and the fitting straight line according to the straight line fitting result, and sort the concentric circles.

[0121] Furthermore, the above device also includes a preprocessing module, including:

[0122] De-noising unit, used for gray-value conversion and filtering denoising of the image;

[0123] The processing unit is used for converting the image into a binary image and performing morphological processing on the image.

[0124] Furthermore, the ellipse fitting module 300 further includes:

[0125] An edge detection unit, used for extracting edge information in an image based on an edge detection algorithm;

[0126] A screening unit is used to screen out contours that meet corresponding conditions in sequence according to the number of concentric circles;

[0127] A first ellipse fitting subunit is used to perform ellipse fitting on the selected contour based on an ellipse fitting algorithm to obtain ellipse parameters;

[0128] The center coordinate extraction unit is used to extract the center coordinates of the ellipse, that is, the center of the ellipse, from the ellipse parameters.

[0129] Furthermore, the averaging processing module 400 includes:

[0130] A set forming unit, used for collecting the coordinates of the center of each ellipse contained in the concentric circle identification to form a coordinate set;

[0131] The final center coordinate calculation unit is used to average the center coordinates of the ellipses in the coordinate set to obtain the final center coordinates of the concentric circle marks.

[0132] Furthermore, the above-mentioned sorting module 500 includes:

[0133] A first conversion unit, used to determine the coordinate system inside the camera and convert the final coordinates of the center of the circle into the coordinate system inside the camera;

[0134] A first calculation unit, used for calculating the camera internal coordinate value of each final coordinate of the circle center according to the final coordinate of the circle center which has been converted into the camera internal coordinate system;

[0135] The internal coordinate acquisition unit is used to sort the coordinate values ​​of the marking points in the internal coordinate system of the camera to obtain the coordinates of the marking points in the internal coordinate system of the camera.

[0136] Furthermore, the matching module 600 includes:

[0137] A conversion unit, used to convert the coordinates of the marker points in the camera's internal coordinate system to the calibration board's real reference system using a space transformation matrix;

[0138] A comparison unit is used to compare the coordinates of the marking points in the real reference system of the calibration plate with the coordinates of the real marking points in the real reference system of the corresponding calibration plate to obtain a comparison result;

[0139] An optimization unit, used for calculating the error between the coordinates of the marking point and the coordinates of the real marking point based on the comparison result, and minimizing the error based on a preset algorithm to obtain an optimization result;

[0140] An extraction unit is used to extract data based on the optimization result to obtain the calibration result.

[0141] Reference Figure 3 In the embodiment of the present application, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a network interface and a database connected by a bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores operations, computer programs and databases. The internal memory provides an environment for the operation of the non-volatile storage medium and the computer program. The database of the computer device is used to store data such as an infrared binocular camera calibration method. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for calibrating an infrared binocular camera is implemented, including the steps of: obtaining a preset number of photographed images of a marking plate at different object distances; performing linear fitting and sorting on the concentric circle marks on the photographed images; performing elliptical contour fitting on the concentric circle marks in the photographed images to obtain the ellipse center; averaging the ellipse center to obtain the final coordinates of the center of the concentric circle mark; sorting based on the final coordinates of the center to obtain the coordinates of the marking point in the camera internal coordinate system; based on the coordinates of the marking point, matching the camera internal coordinate system with the real reference system of the calibration plate to obtain the calibration result.

[0142] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, an infrared binocular camera calibration method is implemented, including the following steps: obtaining a preset number of captured images of a marking plate at different object distances; performing straight line fitting and sorting on the concentric circle marks on the captured images; performing elliptical contour fitting on the concentric circle marks in the captured images to obtain the center of the ellipse; averaging the center of the ellipse to obtain the final coordinates of the center of the concentric circle marks; sorting based on the final coordinates of the center to obtain the coordinates of the marking points in the camera's internal coordinate system; and obtaining the calibration results by matching the camera's internal coordinate system with the calibration plate's real reference system based on the marking point coordinates.

[0143] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided in the present application and used in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0144] The above description is only a preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for calibrating an infrared binocular camera, characterized in that: include: Acquire a preset number of photographed images of the marking plate at different object distances; Perform straight line fitting and sorting on the concentric circle marks on the captured image; Fit the concentric circle marks in the captured image to an ellipse contour to obtain the center of the ellipse; The ellipse centers are averaged to obtain the final coordinates of the centers of the concentric circles; Sorting based on the final coordinates of the circle center obtains the coordinates of the marker points in the camera's internal coordinate system; Based on the coordinates of the marking points, the calibration result is obtained by matching the internal coordinate system of the camera with the real reference system of the calibration board.

2. The infrared binocular camera calibration method according to claim 1, characterized in that: The concentric circle marks are composed of a preset number of concentric circles; the step of performing straight line fitting and sorting on the concentric circle marks on the captured image comprises: Use circle detection algorithm to detect and extract the center coordinate information of concentric circles; Calculate the center coordinate slope and distance between concentric circles based on the center coordinate information to perform straight line fitting; According to the straight line fitting results, the distance from each concentric circle to the fitting line is calculated and the concentric circles are sorted.

3. The infrared binocular camera calibration method according to claim 1, characterized in that: Before the step of fitting the concentric circle marks in the captured image to obtain the center of the ellipse, the process includes: performing image preprocessing on the captured image after sorting, and the image preprocessing step includes: Gray value and filter denoising of the image; Convert the image into a binary image and perform morphological processing on the image.

4. The infrared binocular camera calibration method according to claim 1, characterized in that: The step of fitting the concentric circle marks in the captured image to an ellipse contour to obtain the center of the ellipse comprises: Extract edge information from images based on edge detection algorithms; According to the number of concentric circles, the contours that meet the corresponding conditions are selected in turn; Perform ellipse fitting on the selected contour based on the ellipse fitting algorithm to obtain ellipse parameters; The coordinates of the center of the ellipse are extracted from the ellipse parameters, that is, the center of the ellipse.

5. The infrared binocular camera calibration method according to claim 1, characterized in that: The step of averaging the ellipse centers to obtain the final coordinates of the centers of the concentric circle marks comprises: Collect the coordinates of the center of each ellipse contained in the concentric circle identifier to form a coordinate set; The center coordinates of the ellipses in the coordinate set are averaged to obtain the final center coordinates of the concentric circle markers.

6. The infrared binocular camera calibration method according to claim 1, characterized in that: The step of obtaining the coordinates of the marking points in the camera internal coordinate system by sorting based on the final coordinates of the circle center includes: Determine the coordinate system inside the camera and transform the final coordinates of the center of the circle into the coordinate system inside the camera; According to the final coordinates of the circle center that have been converted to the camera internal coordinate system, calculate the camera internal coordinate value of each final coordinate of the circle center; Sort the coordinate values ​​of the marker points in the camera's internal coordinate system to obtain the coordinates of the marker points in the camera's internal coordinate system.

7. The infrared binocular camera calibration method according to claim 1, characterized in that: The step of obtaining a calibration result by matching the internal coordinate system of the camera with the real reference system of the calibration plate based on the coordinates of the marking points comprises: Use the space transformation matrix to transform the coordinates of the marker points in the camera's internal coordinate system to the calibration board's real reference system; Compare the coordinates of the marking points in the real reference system of the calibration plate with the coordinates of the real marking points in the real reference system of the corresponding calibration plate to obtain a comparison result; Based on the comparison results, the error between the coordinates of the marked points and the actual coordinates of the marked points is calculated, and the error is minimized based on the preset algorithm to obtain the optimization result; Data extraction is performed based on the optimization result to obtain the calibration result.

8. An infrared binocular camera calibration device, characterized in that: include: An image acquisition module, used for acquiring a preset number of photographed images of the marking plate at different object distances; A fitting and sorting module is used to perform straight line fitting and sorting on the concentric circle marks on the captured image; The ellipse fitting module is used to fit the concentric circle marks in the captured image to the ellipse contour and obtain the center of the ellipse; An averaging processing module is used to average the centers of the ellipses to obtain the final coordinates of the centers of the concentric circles; A sorting module is used to sort based on the final coordinates of the circle center to obtain the coordinates of the marking points in the camera's internal coordinate system; The matching module is used to match the coordinates of the marking points through the internal coordinate system of the camera and the real reference system of the calibration board to obtain a calibration result.

9. An infrared binocular camera calibration device, characterized in that: include: processor; Memory for storing computer programs; Wherein, when the processor executes the computer program, the infrared binocular camera calibration method described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the infrared binocular camera calibration method according to any one of claims 1 to 7 is implemented.

Citation Information

Patent Citations

  • Method and device for three-dimensional vision measurement data registration based on planar circle target

    CN102944191A

  • Camera calibration method

    CN110033491A

  • High-precision calibration plate and calibration method

    CN111402343A

  • Infrared camera calibration method and system

    CN113989386A

  • Calibration method of non-coaxial camera

    CN114463442A

Cited By

  • Rapid binocular camera calibration method

    CN120259443A

  • A fast binocular camera calibration method

    CN120259443B

  • Nested identifier design and visual positioning method for target alignment in complex scene

    CN120894419A

  • Structured light camera and ball positioning tool calibration method based on optical positioning system

    CN120894435A

  • Structured light camera calibration method and device, electronic equipment, storage medium and product

    CN121999058A