Infrared binocular camera calibration method, device, equipment and medium

Abstract

The present application relates to the field of infrared binocular camera calibration technology, and discloses an infrared binocular camera calibration method, apparatus, equipment, and medium. The method includes: obtaining a preset number of photographed images of a marker plate at different object distances; performing linear fitting and sorting on the concentric circle marks on the photographed images; performing elliptical contour fitting on the concentric circle marks in the photographed images to obtain the ellipse center; averaging the ellipse centers to obtain the final coordinates of the centers of the concentric circle marks; sorting based on the final coordinates of the centers to obtain the coordinates of the marker points in the camera's internal coordinate system; and obtaining a calibration result based on the marker point coordinates by matching the camera's internal coordinate system with the calibration plate's real reference system. The present application can reduce the detection impact caused by marker point shape changes caused by factors such as camera distortion, calibration plate tilt, or illumination changes, and improve the accuracy and robustness of marker point detection.

Description

Technical Field

[0001] The present application relates to the field of infrared binocular camera calibration technology, and more specifically, to an infrared binocular camera calibration method, device, equipment, and medium. Background Art

[0002] Binocular cameras are widely used in many applications, such as stereo vision, 3D reconstruction, and visual measurement. Binocular cameras simulate human binocular vision, calculating the depth of objects in a scene from image data acquired from two perspectives, thereby enabling 3D visual perception. This helps accurately overlay virtual objects onto the real world, enhancing the realism and accuracy of augmented reality (AR) and virtual reality (VR) applications. Current calibration methods fall into two main categories: geometry-based methods, such as those using specific patterns like a checkerboard grid, calculate camera intrinsic parameters (e.g., focal length, principal point position) and extrinsic parameters (e.g., rotation and translation). Feature-point-based methods detect feature points in the image (e.g., corners, SIFT feature points) to match and calculate the transformation between cameras. Modern binocular camera calibration methods typically require high accuracy and stability to meet the requirements of complex applications such as medical imaging and precision manufacturing. For example, the traditional Zhang's method, a classic binocular camera calibration method, uses intrinsic and extrinsic camera parameters as well as distortion parameters to describe the camera's geometric characteristics. Although this method is widely used and has many successful applications, it also has some disadvantages and limitations. For example, the quality and layout of the calibration plate have a significant impact on the calibration results. If the calibration plate is of poor quality or is placed incorrectly, the accuracy of the calibration results may be reduced.

[0003] Therefore, the existing binocular camera calibration method is greatly affected by the quality and layout of the calibration plate, which easily leads to a decrease in accuracy and needs to be solved urgently. Summary of the Invention

[0004] The main purpose of this application is to provide an infrared binocular camera calibration method, which aims to solve the technical problem that the existing binocular camera calibration method is greatly affected by the quality and layout of the calibration board, which easily leads to a decrease in accuracy.

[0005] The first aspect of the present application provides a method for calibrating an infrared binocular camera, comprising:

[0006] Acquire a preset number of photographed images of the marker plate at different object distances;

[0007] Perform straight line fitting and sorting on the concentric circle marks on the captured image;

[0008] Fit the concentric circle marks in the captured image to an ellipse contour to obtain the ellipse center;

[0009] The ellipse centers are averaged to obtain the final coordinates of the centers of the concentric circles;

[0010] Sort based on the final coordinates of the circle center to obtain the coordinates of the marker points in the camera's internal coordinate system;

[0011] Based on the coordinates of the marker points, the camera's internal coordinate system is matched with the calibration board's real reference system to obtain a calibration result.

[0012] Furthermore, the concentric circle marks are composed of a preset number of concentric circles; and the step of performing straight line fitting and sorting on the concentric circle marks on the captured image includes:

[0013] Use circle detection algorithm to detect and extract the center coordinate information of concentric circles;

[0014] Calculating the center coordinate slope and distance between the concentric circles based on the center coordinate information to perform straight line fitting;

[0015] According to the straight line fitting results, the distance between each concentric circle and the fitting line is calculated and the concentric circles are sorted.

[0016] Furthermore, before the step of fitting the concentric circle marks in the captured image to an ellipse contour to obtain the center of the ellipse, the image preprocessing step is performed on the sorted captured image, and the image preprocessing step includes:

[0017] Gray value and filter denoising of the image;

[0018] Convert the image into a binary image and perform morphological processing on the image.

[0019] Furthermore, the step of fitting the concentric circle marks in the captured image to an ellipse contour to obtain the center of the ellipse includes:

[0020] Extract edge information from images based on edge detection algorithms;

[0021] According to the number of concentric circles, the contours that meet the corresponding conditions are selected in turn;

[0022] Perform ellipse fitting on the selected contour based on the ellipse fitting algorithm to obtain the ellipse parameters;

[0023] The coordinates of the center of the ellipse, that is, the center of the ellipse, are extracted from the ellipse parameters.

[0024] Furthermore, the step of averaging the ellipse centers to obtain the final coordinates of the centers of the concentric circle marks includes:

[0025] Collect the coordinates of the center of each ellipse contained in the concentric circle identifier to form a coordinate set;

[0026] The center coordinates of the ellipses in the coordinate set are averaged to obtain the final center coordinates of the concentric circle markers.

[0027] Furthermore, the step of obtaining the coordinates of the marker points in the camera internal coordinate system by sorting based on the final coordinates of the circle center includes:

[0028] Determine the internal coordinate system of the camera and convert the final coordinates of the circle center to the internal coordinate system of the camera;

[0029] Calculate the camera internal coordinate value of each circle center final coordinate based on the final coordinate of the circle center that has been converted to the camera internal coordinate system;

[0030] Sort the coordinates of the marker points in the camera's internal coordinate system to obtain the coordinates of the marker points in the camera's internal coordinate system.

[0031] Furthermore, the step of obtaining a calibration result by matching the camera internal coordinate system with the real reference system of the calibration plate based on the coordinates of the marker points includes:

[0032] Use the space transformation matrix to transform the coordinates of the marker points in the camera's internal coordinate system to the real reference system of the calibration plate;

[0033] Compare the coordinates of the marker points in the real reference system of the calibration plate with the coordinates of the real marker points in the real reference system of the corresponding calibration plate to obtain a comparison result;

[0034] Based on the comparison results, the error between the coordinates of the marked points and the true coordinates of the marked points is calculated, and the error is minimized based on the preset algorithm to obtain the optimization result;

[0035] Data extraction is performed based on the optimization result to obtain the calibration result.

[0036] The second aspect of the present application provides an infrared binocular camera calibration device, comprising:

[0037] An image acquisition module, used for acquiring a preset number of photographed images of the marking plate at different object distances;

[0038] The fitting and sorting module is used to perform straight line fitting and sorting on the concentric circle marks on the captured image;

[0039] The ellipse fitting module is used to fit the concentric circle marks in the captured image to the ellipse contour and obtain the center of the ellipse;

[0040] An averaging processing module is used to average the centers of the ellipses to obtain the final coordinates of the centers of the concentric circle markers;

[0041] A sorting module is used to sort the final coordinates of the circle center to obtain the coordinates of the marker points in the camera's internal coordinate system;

[0042] The matching module is used to match the coordinates of the mark points through the internal coordinate system of the camera and the real reference system of the calibration plate to obtain a calibration result.

[0043] The third aspect of the present application provides an infrared binocular camera calibration device, comprising:

[0044] processor;

[0045] memory for storing computer programs;

[0046] Wherein, when the processor executes the computer program, any one of the above-mentioned infrared binocular camera calibration methods is implemented.

[0047] A fourth aspect of the present application provides a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, it implements any of the above-mentioned infrared binocular camera calibration methods.

[0048] Beneficial effects:

[0049] This application increases the number and complexity of calibration points by using concentric circle marking plates instead of traditional solid circle marking plates, thereby improving the redundancy and stability of calibration. Image acquisition at different object distances ensures the applicability of the calibration results at various working distances and enhances the generalization ability of the system. Through elliptical contour fitting and circle center averaging processing, the center position of the circle can be determined more accurately even in the case of camera distortion and calibration plate tilt, and it is more resistant to changes in the shape of marker points caused by factors such as camera distortion, calibration plate tilt or illumination changes, thereby improving the accuracy and robustness of marker point detection. The use of multiple concentric circles and elliptical fitting of each circle enable the system to maintain good performance in the presence of noise and partial occlusion. The straight line fitting and sorting steps ensure that each marker point can be correctly identified and processed even in the case of image deformation. In summary, this solution can significantly improve the accuracy, robustness and efficiency of the calibration process. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is a schematic diagram of the steps of a calibration method for an infrared binocular camera according to an embodiment of the present application;

[0051] Figure 2 This is a schematic block diagram of the structure of an infrared binocular camera calibration device according to one embodiment of the present application;

[0052] Figure 3 This is a schematic block diagram of the structure of a computer device according to an embodiment of the present application;

[0053] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0055] Those skilled in the art will appreciate that, unless expressly stated otherwise, the singular forms "a", "an", "above", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of this application refers to the presence of features, integers, steps, operations, elements, modules, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, modules, components, and / or groups thereof. It should be understood that when an element is said to be "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and / or" used herein includes all or any module and all combinations of one or more associated listed items.

[0056] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0057] Reference Figure 1 The present invention provides an infrared binocular camera calibration method, comprising the following steps S1-S6:

[0058] S1, obtaining a preset number of photographed images of the marking plate at different object distances;

[0059] S2. performing straight line fitting and sorting on the concentric circle marks on the captured image;

[0060] S3, performing ellipse contour fitting on the concentric circle marks in the captured image to obtain the center of the ellipse;

[0061] S4, averaging the ellipse centers to obtain the final coordinates of the centers of the concentric circle markers;

[0062] S5. Sort based on the final coordinates of the circle center to obtain the coordinates of the marker points in the camera's internal coordinate system;

[0063] S6. Based on the coordinates of the marking points, the internal coordinate system of the camera is matched with the real reference system of the calibration plate to obtain a calibration result.

[0064] In step S1, a designed marker plate is used. The marker plate has a preset number of concentric circle markings, for example, 50; each concentric circle marking is composed of a preset number of concentric circles, for example, three concentric circles of different sizes. The marker plate should be placed in a flat and evenly lit environment, avoiding shadows and direct strong light to ensure image quality. Different object distances are set according to calibration requirements, such as 1 meter, 1.3 meters, 1.8 meters, and 2 meters. These distances should be selected based on the camera's operating range and the expected calibration accuracy. Using the infrared binocular camera to be calibrated, a certain number of images, for example, 35, are captured at each set object distance. Diverse image data facilitates more accurate estimation of camera parameters, especially when dealing with non-ideal conditions (such as camera distortion or calibration plate tilt). During capture, the camera should be stably fixed to avoid movement or vibration to ensure image clarity and consistency. To improve calibration robustness and accuracy, the marker plate should be captured from different angles and positions to capture its behavior in different poses. This facilitates more accurate estimation of the camera's intrinsic and extrinsic parameters in subsequent steps. Step S1 is the starting point of the entire calibration process and provides raw data for subsequent image processing and analysis. The captured image will be used to detect and locate the marker points on the marker board, and then calculate the camera parameters.

[0065] In step S2, the captured image can be preprocessed, including grayscale, Gaussian filtering, binarization, and morphological processing, to enhance image contrast, reduce noise, and highlight the concentric circle markers. A circle detection algorithm (such as Hough transform or edge detection combined with a mathematical model of a circle) is used to detect the concentric circle markers in the image. The detected center coordinate information will be the basis for subsequent straight line fitting. The slope between the center coordinates of adjacent marker points is calculated, and these slopes and coordinate information are used to perform straight line fitting on the concentric circle markers. Straight line fitting can be achieved through the least squares method or other statistical methods, with the goal of finding a straight line that best represents the arrangement trend of the marker points. Straight line fitting can help locate the position of the marker points more accurately, especially when the shape of the marker points in the image may be distorted due to camera distortion. The concentric circle markers are then sorted according to the straight line fitting results. The sorting can be based on the distance between the concentric circle markers and the fitted straight line, or the center coordinate information of the concentric circle markers, to ensure their correct order, facilitating subsequent ellipse fitting and coordinate calculation. Sorting ensures that the concentric circle markers are processed consistently in subsequent steps. Correct sorting ensures the smooth progress of subsequent steps (such as ellipse fitting and coordinate calculation), ensuring the consistency and effectiveness of the entire calibration process. This step is the first step in extracting useful information from the image data, laying the foundation for subsequent coordinate calculation and calibration parameter optimization. By fitting and sorting lines, the correct correspondence between each marker point can be ensured, which is crucial for accurately calculating the coordinates of the marker points and subsequent calibration.

[0066] In step S3, edge information can be extracted from the preprocessed image using Canny edge detection, the Sobel operator, or other edge detection algorithms. This helps identify the boundaries of the concentric circles. Based on this edge information, contour detection functions in computer vision libraries such as OpenCV (OpenSource Computer Vision Library) are used to identify and select contours corresponding to the concentric circle markers. Note that these contours may be elliptical due to camera distortion. For each detected contour, an ellipse fitting algorithm (such as the least squares method or RANSAC (Random Sample Consensus Algorithm)) is applied to fit an optimal ellipse that matches the contour's actual shape. The goal of ellipse fitting is to find an ellipse equation that minimizes the sum of the squared distances between the ellipse and the contour's edge points. The ellipse center coordinates are extracted from the fitted ellipse parameters. Assuming that each concentric circle marker is actually composed of three concentric circles, three ellipse center coordinates are obtained for each concentric circle marker. Compared to directly using the circle detection results, ellipse fitting can more accurately reflect the true shape of the marker points, especially in the case of large camera distortion. This can significantly improve the accuracy of the circle center coordinates. Ellipse fitting is more adaptable to changes in the shape of concentric circle markers and can obtain reliable circle center coordinates even when the markers are slightly deformed or partially obscured. By improving the accuracy of marker point detection in the early stages, the accumulation of errors in subsequent steps can be reduced, thereby improving the overall accuracy of the entire calibration process.

[0067] In step S4, the coordinates of the centers of the N ellipses of the same concentric circle mark are averaged to obtain a more accurate and stable coordinate of the center. For each concentric circle mark, the x-coordinates and y-coordinates of the three ellipse centers are averaged, that is, the average of the N x-coordinates and the N y-coordinates, to obtain a new x-coordinate and y-coordinate, which will be used as the final coordinate of the center of the concentric circle mark. This step reduces the coordinate error caused by camera distortion, calibration plate tilt or other factors through averaging, thereby providing more accurate basic data for subsequent coordinate sorting and calculation of the spatial transformation matrix. Averaging can reduce coordinate errors. Even when a single ellipse fitting is affected by noise or distortion, the averaged coordinates can be closer to the true center position. That is, in the case where some ellipse fitting results are not ideal, by fusing the information of multiple ellipse centers, relatively accurate center coordinates can be provided, thereby improving the robustness of coordinate calculation.

[0068] In step S5, these final center coordinates are sorted. The purpose of sorting is to ensure the uniqueness and correctness of each marker point in subsequent processing. Sorting can be based on the geometric position of the marker point or the known marker plate layout, for example, in a left-to-right, top-to-bottom order. Create a marker point coordinate list, which contains all the converted and sorted marker point coordinates. These coordinates will provide data support for the next step of spatial transformation matrix calculation in a unified format and order. This step converts the results of the previous image processing and feature point detection into coordinate data that can be directly used for parameter estimation, ensuring the accuracy and consistency of subsequent calibration calculations. Convert to the camera's internal coordinate system. This conversion usually involves the use of the camera's internal parameter matrix (including parameters such as focal length and principal point position) and possible distortion correction. A standardized conversion from image feature points to camera internal coordinate system marker point coordinates is achieved, providing a high-quality data foundation for the final calibration result.

[0069] In step S6, the outputs of all previous steps—that is, the precisely ordered coordinates of the marker points in the camera's internal coordinate system—are matched with the calibration plate's real-world reference coordinate system to determine the final calibration result. The marker point coordinates in the camera's internal coordinate system, obtained in step S5, are paired with the known real-world coordinates on the calibration plate (the calibration plate's real-world reference coordinate system). Using these corresponding point pairs, the camera's intrinsic and extrinsic parameters can be estimated using least squares or other optimization algorithms. These parameters include internal parameters such as the camera's focal length, principal point position, pixel size, and radial distortion coefficient, as well as external parameters such as rotation and translation vectors. Parameter estimation is typically a nonlinear optimization problem because the distortion equations of the camera model are nonlinear. Therefore, iterative optimization techniques such as the Levenberg-Marquardt algorithm are required to solve the problem and minimize the reprojection error. Furthermore, after parameter estimation is complete, the calibration accuracy can be further verified. Specifically, this can be done by reprojecting the known world coordinates (the calibration plate's real-world reference coordinate system) onto the image plane and then comparing the distances between these reprojected points and the actual detected marker points. This step effectively solves the calibration problem by matching the coordinates of the marker points in the camera's internal coordinate system with the real reference coordinates of the calibration plate.

[0070] In one embodiment, the concentric circle marks are composed of a preset number of concentric circles; and the step of performing straight line fitting and sorting on the concentric circle marks on the captured image includes:

[0071] S10, using a circle detection algorithm to detect and extract the center coordinate information of the concentric circles;

[0072] S11, calculating the center coordinate slope and distance between the concentric circles based on the center coordinate information to perform straight line fitting;

[0073] S12. According to the straight line fitting result, calculate the distance between each concentric circle and the fitting straight line, and sort the concentric circles.

[0074] In this embodiment, concentric circle markers are processed to ensure accurate and efficient identification and utilization of these circles during camera calibration. First, a circle detection algorithm is used to detect and extract the center coordinates of the concentric circles. This is because the center coordinates of the concentric circles are key to determining their relative positional relationships. The circle detection algorithm finds edge points in the image and fits these points to a circle model to obtain the center coordinates and radius of the circle. This method is very effective for identifying concentric circle markers because it can accurately locate the center of each concentric circle. Next, based on the center coordinate information of the concentric circles, the slope and distance between the center coordinates are calculated to perform line fitting. Because the concentric circles are arranged according to a certain pattern, their center coordinates will approximate a straight line. By calculating the slope and distance between the center coordinates, the parameters of this line can be obtained, and fitting can be performed using methods such as least squares. The purpose of this step is to determine the relative positional relationships between the concentric circles, providing a basis for subsequent sorting and matching. Finally, based on the results of the line fitting, the distance from each concentric circle to the fitted line is calculated, and the concentric circles are sorted accordingly. The purpose of this step is to arrange the detected concentric circles in the order they appear on the calibration plate, allowing for subsequent alignment with the calibration plate's real-world reference coordinate system. Since the relative positions of the concentric circles have been determined using straight-line fitting, they can be sorted based on their distance from the fitted line. This method is both simple and effective, ensuring accurate sorting.

[0075] In summary, steps S10 to S12 provide an accurate data foundation for the subsequent calibration process by accurately identifying and processing the concentric circle markers. These steps not only improve the accuracy of the calibration, but also enhance the robustness of the system, making the entire calibration process more reliable and efficient.

[0076] In one embodiment, before the step of fitting the concentric circle marks in the captured image to obtain the center of the ellipse, the following steps are included: performing image preprocessing on the sorted captured image, and the image preprocessing step includes:

[0077] S20, graying and filtering the image to remove noise;

[0078] S21. Convert the image into a binary image and perform morphological processing on the image.

[0079] In this embodiment, image preprocessing of the sorted captured images is a crucial step in ensuring the accuracy of subsequent ellipse contour fitting and circle center calculation. Image preprocessing not only simplifies image data and highlights target information, but also effectively eliminates or reduces noise and interference, providing a high-quality data source for subsequent analysis and processing. First, in step S20, the image is grayscaled. This is because the original image is typically in color and contains a large amount of color information, which is not essential for contour recognition and increases processing complexity and computational effort. Grayscale conversion converts a color image into a grayscale image by calculating the average or weighted average of the red, green, and blue color components of each pixel to obtain a grayscale value, thereby forming a grayscale image. The grayscaled image contains only brightness information, facilitating subsequent processing and analysis. Next, in step S20, the grayscaled image is filtered to remove noise. Due to the influence of the shooting environment and equipment, noise and interference may exist in the image, which can affect the recognition of the concentric circle markers and the calculation of the circle center. Filtering and denoising involves processing the image using a specific algorithm or filter to remove or reduce noise and interference. In this embodiment, Gaussian filtering, median filtering, and other algorithms can be used for filtering and denoising to improve the clarity and quality of the image. Here, Gaussian filtering is used as an example. The convolution operation based on the Gaussian function can effectively blur the image, making it smoother and more continuous.

[0080] Gaussian function: (Where, (G(x,y)) is the probability distribution of the point (x,y) in two-dimensional space, which is used to define a filter to blur the noise in the image; σ is the standard deviation of the Gaussian function, which determines the distribution range and peak value of the function); Gaussian filter operator: The Gaussian filter operator convolves the image with the Gaussian function, and its discrete form is Among them, (H(i,j)) is the operation of convolving the Gaussian function with the image, the purpose of which is to smooth the image; i and j are the coordinates of the convolution kernel, which are usually odd numbers, such as 5×5, to maintain central symmetry. Then, in step S21, the filtered and denoised image is converted into a binary image. The grayscale image is converted into a binary image using a global threshold or an adaptive threshold method. The global threshold sets a fixed grayscale threshold, and pixels above this threshold are set to white (foreground), and pixels below this threshold are set to black (background). The adaptive threshold takes into account the brightness differences in local areas and is more suitable for processing images with uneven lighting. In this embodiment, a suitable threshold can be set based on the grayscale characteristics of the concentric circle markers to separate the concentric circle markers from the background to form a binary image. Finally, in step S21, the binary image is morphologically processed. Morphological processing is an image processing technology based on mathematical morphology, which can be used for image segmentation, extraction, smoothing and other operations. In this embodiment, morphological operations such as dilation and erosion can be used to process the binary image to eliminate holes and noise within the concentric circle markers and smooth the edges of the concentric circle markers, providing high-quality image data for subsequent ellipse contour fitting and circle center calculation. For example, in the case of three layers of concentric circles, since the focus is on three concentric circles, the key is to find three contours of decreasing size. Once the three contours are identified, different thresholds or grayscale levels can be set for each circle layer based on their position and size. The outermost layer (largest contours) can be set as the bright area (white), the middle layer (medium-sized contours) as the dark area (black), and the innermost layer (smallest contours) as the bright area (white). For each identified circle layer, the corresponding grayscale or binarization threshold is applied to obtain a light-dark-light pattern from the outside to the inside. After applying the above steps, morphological operations (opening and closing) are required to eliminate noise or disconnected boundaries to ensure the integrity and clarity of each circle layer.

[0081] In summary, this embodiment, through grayscale conversion, filtering and denoising, binarization, and morphological processing, combined with adaptive thresholding and image segmentation techniques, can effectively handle uneven illumination and ensure accurate distinction between light and dark levels in the concentric circle markers, thereby providing high-quality image input for subsequent feature detection and calibration. This provides an accurate and reliable data foundation for subsequent ellipse contour fitting and circle center calculation. This step not only improves calibration accuracy but also enhances the robustness of the system, making the entire calibration process more reliable and efficient.

[0082] In one embodiment, the step of fitting the concentric circle marks in the captured image to an ellipse contour to obtain the ellipse center includes:

[0083] S30, extracting edge information from the image based on an edge detection algorithm;

[0084] S31. According to the number of concentric circles, the contours meeting the corresponding conditions are selected in sequence;

[0085] S32, performing ellipse fitting on the selected contour based on an ellipse fitting algorithm to obtain ellipse parameters;

[0086] S33. Extracting the coordinates of the center of the ellipse from the ellipse parameters, that is, the center of the ellipse.

[0087] In this embodiment, edge detection is a crucial step in image processing for identifying areas of significant brightness variation within an image. The Canny edge detection algorithm is employed in this step. This algorithm detects edges by calculating the gradient strength and direction of each pixel in the image. The Canny algorithm's advantage lies in its ability to simultaneously meet the requirements of low error rate, high localization accuracy, and single edge response, ensuring accurate edge extraction of the concentric circle markers from a complex image background. After edge information is extracted, the contours corresponding to the concentric circle markers must be selected from the numerous contours. The previous morphological processing and binarization steps have yielded three contours of decreasing size, corresponding to the three layers of concentric circles. In this step, appropriate thresholds or conditions, such as contour size and shape, are set to select these three specific contours. Once the correct contours are selected, an ellipse fitting algorithm is used to calculate the ellipse parameters corresponding to each contour. The least squares method is chosen for ellipse fitting because it minimizes fitting errors, resulting in more accurate ellipse parameters. Through ellipse fitting, parameters such as the center position, major and minor axis lengths, and rotation angle of each concentric circle can be determined. The final step is to extract the coordinates of the ellipse center from the ellipse parameters. By extracting the center coordinates of the circle, the center position of the concentric circle markers can be accurately calculated, providing an accurate and reliable data basis for subsequent feature detection and calibration.

[0088] In summary, steps S30 to S33 can accurately extract the center coordinates of the concentric circle markers through precise edge detection, contour screening and ellipse fitting algorithms, providing high-quality image input for subsequent feature detection and calibration, thereby improving the accuracy of calibration and the robustness of the system.

[0089] In one embodiment, the step of averaging the ellipse centers to obtain the final coordinates of the centers of the concentric circle markers includes:

[0090] S40, collecting the coordinates of the center of each ellipse contained in the concentric circle identifiers to form a coordinate set;

[0091] S41 , averaging the center coordinates of the ellipses in the coordinate set to obtain final center coordinates of the concentric circle markers.

[0092] This embodiment ensures high-precision center location even in the presence of image distortion or distorted shooting angles. After performing ellipse fitting and obtaining the center coordinates of each ellipse, these coordinates are collected to form a coordinate set. This is achieved by iterating through all identified ellipses, reading and storing the center coordinates (x, y) of each ellipse. These coordinates may deviate due to shooting angle, lens distortion, or changes in lighting conditions, but they all point to the same physical center. The primary purpose of collecting the coordinate set is to ensure that information is available for all ellipse centers. This allows the subsequent averaging process to utilize all data points, rather than relying on the measurement results of a single ellipse. This improves the stability and reliability of the final calculation result, as the center of a single ellipse may deviate significantly due to poor local image quality or algorithm errors. After obtaining the coordinate set of all ellipse centers, they are averaged to calculate a final center coordinate that integrates the information of all ellipse centers. Averaging is achieved by calculating the arithmetic mean of the x and y coordinates of all coordinate sets. Specifically, all x-coordinates and all y-coordinates are averaged, and the resulting values ​​are the final center coordinates. The primary purpose of averaging is to reduce measurement errors and improve calibration accuracy. Because the center coordinates of each ellipse are calculated independently, they may be affected by various factors and deviate. By calculating the average value, this random error can be offset, resulting in an estimate that is closer to the true center position. Furthermore, this method is robust; even if the fitting results for some ellipses are significantly affected by noise, the final average coordinates can still maintain a high degree of accuracy.

[0093] In one embodiment, the step of obtaining the coordinates of the marker points in the camera internal coordinate system by sorting based on the final coordinates of the circle center includes:

[0094] S50, determining the internal coordinate system of the camera, and converting the final coordinates of the circle center into the internal coordinate system of the camera;

[0095] S51. Calculate the camera internal coordinate value of each final coordinate of the circle center according to the final coordinate of the circle center that has been converted into the camera internal coordinate system;

[0096] S52 , sorting the coordinate values ​​of the marker points in the camera's internal coordinate system to obtain the coordinates of the marker points in the camera's internal coordinate system.

[0097] In this embodiment, the camera's internal coordinate system must first be determined. The camera's internal coordinate system is typically established with the camera's optical axis as the z-axis, the horizontal direction of the image plane as the x-axis, and the vertical direction as the y-axis. The coordinate origin is located at the center of the image, but in practice, the coordinate system origin may also be the upper-left corner of the image, depending on the camera model and coordinate system definition used. Once the coordinate system is determined, the final coordinates of the circle centers are converted to the camera's internal coordinate system. Converting the final coordinates of the circle centers to the camera's internal coordinate system ensures that all subsequent calculations and analyses are performed within a unified coordinate framework, avoiding error accumulation caused by inconsistent coordinate systems. After the coordinate conversion is complete, the final coordinates of each circle center are calculated in the camera's internal coordinate system. This process typically involves simple linear algebra operations, where the converted coordinate values ​​are substituted into the camera model's mathematical expressions to obtain the internal coordinate values ​​of each circle center. Finally, the circle centers are sorted according to their calculated coordinate values ​​in the camera's internal coordinate system. Sorting can be based on the magnitude of the coordinate values, positional relationships, or other logical rules to facilitate subsequent processing and analysis. This embodiment converts the center coordinates into the camera's internal coordinate system and sorts them, so that the camera parameters can be estimated and calibrated more effectively, thereby improving the performance and reliability of the entire system.

[0098] In one embodiment, the step of obtaining a calibration result by matching the camera internal coordinate system with the real reference system of the calibration plate based on the coordinates of the marker points includes:

[0099] S60, using the space transformation matrix, transforming the coordinates of the marker points in the camera's internal coordinate system into the real reference system of the calibration plate;

[0100] S61, comparing the coordinates of the marking points in the real reference system of the calibration plate with the coordinates of the real marking points in the real reference system of the corresponding calibration plate to obtain a comparison result;

[0101] S62. Based on the comparison result, calculate the error between the coordinates of the marked point and the true coordinates of the marked point, and minimize the error based on a preset algorithm to obtain an optimization result;

[0102] S63. Perform data extraction based on the optimization result to obtain the calibration result.

[0103] In this embodiment, a spatial transformation matrix, i.e., a rigid body transformation matrix, is used to transform the coordinates of the marker points in the camera's internal coordinate system into the calibration plate's real reference system. This transformation matrix contains rotation and translation parameters and can accurately describe the relative position and orientation between the camera coordinate system and the world coordinate system (calibration plate's real reference system). The transformation process involves multiplying each marker point coordinate by the transformation matrix to map it from the camera coordinate system to the calibration plate's coordinate system.

[0104] Perform corner detection on each image, assuming the internal parameters of the camera (such as focal length f x 、f y and principal point offset c x 、c y ), as well as external parameters (rotation matrix R and translation vector t) and distortion parameters (radial distortion coefficients k1, k2, tangential distortion coefficients p1, p2). Nonlinear optimization methods (such as least squares) are used to optimize the camera's intrinsic and extrinsic parameters, as well as the distortion parameters, by minimizing the reprojection error (the error between the actual corner points and the calculated corner points).

[0105] The camera projects a point (X, Y, Z) in three-dimensional space to a point (u, v) on the image plane. The projection process can be described by the following formula:

[0106]

[0107] It can be seen that the camera projection matrix P is composed of internal parameters (focal length fx, fy, principal point offset cx, cy), external parameters (rotation matrix R and translation vector t), and distortion parameters. Through this matrix, we can project the point (X, Y, Z) in three-dimensional space to (u, v) on the image plane, that is, convert the coordinates of the marker points in the camera's internal coordinate system to the real reference system of the calibration plate. The purpose of this conversion process is to establish a connection between the camera coordinate system and the real-world coordinate system, allowing us to compare the positions of the marker points observed by the camera with their known positions in the real world. This lays the foundation for subsequent error calculation and parameter optimization.

[0108] Comparing the transformed marker coordinates from the camera coordinate system with the true position coordinates of the markers on the calibration plate allows us to quantify the deviation between the camera observations and the true values. The true marker coordinates are pre-measured and known, and they constitute the "ground truth" during the calibration process. The projection formula shows that the camera projects a point (X, Y, Z) in 3D space onto the image plane (U, V). By comparing the projected (U, V) with the true marker coordinates, we can obtain the projection error, which is the difference between the camera observations and the true values. This comparison reveals the accuracy of the camera calibration parameters, specifically the accuracy of the camera's intrinsic parameters (such as focal length and principal point location) and extrinsic parameters (such as rotation and translation matrices). This comparison is the basis for evaluating calibration results and optimizing parameters. Based on the comparison results obtained in S61, we calculate the error between each marker coordinate and its true position. These errors are typically expressed as distances (such as Euclidean distances) and reflect the deviation between the camera observations and the true position. Subsequently, nonlinear optimization methods (such as the Levenberg-Marquardt algorithm) are used to adjust the camera parameters to minimize the total error until a preset convergence criterion is reached.

[0109] By minimizing the projection error, we can update these parameters to bring them closer to the true values, thereby obtaining optimized camera parameters. After obtaining the optimization results, we extract the final camera parameters from these results, including internal parameters (such as focal length, principal point offset, radial distortion coefficients k1, k2, tangential distortion coefficients p1, p2) and external parameters (rotation and translation matrices). These parameters constitute the final calibration results, which can be used to correct image distortion, enhance the accuracy of 3D reconstruction, and support other advanced vision processing tasks.

[0110] refer to Figure 2 , an infrared binocular camera calibration device, comprising:

[0111] An image acquisition module 100 is used to acquire a preset number of photographic images of the marking plate at different object distances;

[0112] A fitting and sorting module 200 is used to perform straight line fitting and sorting on the concentric circle marks on the captured image;

[0113] The ellipse fitting module 300 is used to fit the concentric circle marks in the captured image to the ellipse contour and obtain the center of the ellipse;

[0114] An averaging processing module 400 is used to average the ellipse centers to obtain the final coordinates of the centers of the concentric circle markers;

[0115] A sorting module 500 is used to sort based on the final coordinates of the circle center to obtain the coordinates of the marker points in the camera's internal coordinate system;

[0116] The matching module 600 is used to match the camera's internal coordinate system with the calibration board's real reference system based on the coordinates of the marker points to obtain a calibration result.

[0117] Furthermore, the fitting sorting module 200 includes:

[0118] A detection unit, configured to detect and extract the center coordinate information of concentric circles using a circle detection algorithm;

[0119] A first fitting unit, configured to calculate the center coordinate slope and distance between concentric circles based on the center coordinate information to perform straight line fitting;

[0120] The first sorting subunit is used to calculate the distance between each concentric circle and the fitting straight line according to the straight line fitting result, and sort the concentric circles.

[0121] Furthermore, the above device also includes a pre-processing module, including:

[0122] Denoising unit, used to grayscale and filter the image to remove noise;

[0123] The processing unit is used to convert the image into a binary image and perform morphological processing on the image.

[0124] Furthermore, the ellipse fitting module 300 further includes:

[0125] An edge detection unit, configured to extract edge information from an image based on an edge detection algorithm;

[0126] The screening unit is used to sequentially screen out contours that meet corresponding conditions according to the number of concentric circle layers;

[0127] A first ellipse fitting subunit is used to perform ellipse fitting on the selected contour based on an ellipse fitting algorithm to obtain ellipse parameters;

[0128] The center coordinate extraction unit is used to extract the center coordinates of the ellipse, that is, the center of the ellipse, from the ellipse parameters.

[0129] Furthermore, the averaging processing module 400 includes:

[0130] a set forming unit for collecting the coordinates of the center of each ellipse contained in the concentric circle identifier to form a coordinate set;

[0131] The final circle center coordinate calculation unit is used to average the circle center coordinates of the ellipses in the coordinate set to obtain the final circle center coordinates of the concentric circle marks.

[0132] Furthermore, the sorting module 500 includes:

[0133] A first conversion unit is used to determine the coordinate system inside the camera and convert the final coordinates of the circle center into the coordinate system inside the camera;

[0134] A first calculation unit is used to calculate the camera internal coordinate value of each final coordinate of the circle center according to the final coordinate of the circle center that has been converted into the camera internal coordinate system;

[0135] The internal coordinate acquisition unit is used to sort the coordinate values ​​of the marker points in the camera's internal coordinate system to obtain the coordinates of the marker points in the camera's internal coordinate system.

[0136] Furthermore, the matching module 600 includes:

[0137] The conversion unit is used to convert the coordinates of the marker points in the camera's internal coordinate system to the real reference system of the calibration plate using the space transformation matrix;

[0138] A comparison unit is used to compare the coordinates of the marking points in the real reference system of the calibration plate with the coordinates of the real marking points in the real reference system of the corresponding calibration plate to obtain a comparison result;

[0139] An optimization unit, configured to calculate the error between the coordinates of the marker point and the true coordinates of the marker point based on the comparison result, and minimize the error based on a preset algorithm to obtain an optimization result;

[0140] An extraction unit is used to extract data based on the optimization result to obtain the calibration result.

[0141] Reference Figure 3 In the embodiment of the present application, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 3 As shown. The computer device includes a processor, memory, a network interface, and a database connected via a bus. The processor is designed to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores operations, computer programs, and a database. The internal memory provides an environment for the operations and computer programs in the non-volatile storage medium. The database of the computer device is used to store data such as an infrared binocular camera calibration method. The network interface of the computer device is used to communicate with an external terminal via a network connection. When executed by the processor, the computer program implements an infrared binocular camera calibration method, comprising the steps of: acquiring a preset number of images of a marker plate at different object distances; performing linear fitting and sorting on the concentric circle markers on the captured images; performing elliptical contour fitting on the concentric circle markers in the captured images to determine the ellipse centers; averaging the ellipse centers to obtain final coordinates of the concentric circle marker centers; sorting based on the final coordinates of the centers to obtain marker point coordinates in the camera's internal coordinate system; and matching the marker point coordinates with the calibration plate's real-world reference system via the camera's internal coordinate system to obtain a calibration result.

[0142] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, a method for calibrating an infrared binocular camera is implemented, comprising the steps of: acquiring a preset number of captured images of a marker plate at different object distances; performing straight line fitting and sorting on the concentric circle marks on the captured images; performing elliptical contour fitting on the concentric circle marks in the captured images to obtain the center of the ellipse; averaging the center of the ellipse to obtain the final coordinates of the center of the concentric circle marks; sorting based on the final coordinates of the center to obtain the coordinates of the marker points in the camera's internal coordinate system; and obtaining a calibration result by matching the camera's internal coordinate system with the real reference system of the calibration plate based on the marker point coordinates.

[0143] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided in this application and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0144] The above description is only a preferred embodiment of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for calibrating an infrared binocular camera, characterized in that: include: Acquire a preset number of photographed images of the marker plate at different object distances; Use a circle detection algorithm to detect and extract the center coordinate information of each concentric circle in the concentric circle mark on the captured image, calculate the center coordinate slope between the concentric circles, and perform straight line fitting on the concentric circle marks. Based on the straight line fitting results, calculate the distance between each concentric circle and the fitted line to determine the relative position relationship between the concentric circle marks, and sort the concentric circles based on the distance between the concentric circle marks and the fitted line; Perform ellipse contour fitting on the concentric circle marks sorted in the captured image to obtain the center of the ellipse; The ellipse centers are averaged to obtain the final coordinates of the centers of the concentric circles; Sort based on the final coordinates of the circle center to obtain the coordinates of the marker points in the camera's internal coordinate system; Based on the coordinates of the marker points, the camera's internal coordinate system is matched with the calibration board's real reference system to obtain a calibration result; The step of fitting the concentric circle marks in the sorted order in the captured image to an ellipse contour to obtain the center of the ellipse includes: Extract edge information from images based on edge detection algorithms; According to the number of concentric circles, the contours that meet the corresponding conditions are selected in turn; Perform ellipse fitting on the selected contour based on the ellipse fitting algorithm to obtain the ellipse parameters; The coordinates of the center of the ellipse, that is, the center of the ellipse, are extracted from the ellipse parameters.

2. The infrared binocular camera calibration method according to claim 1, characterized in that: Before the step of fitting the concentric circle marks in the sorted captured image to obtain the center of the ellipse, the process includes: performing image preprocessing on the sorted captured image, wherein the image preprocessing step includes: Gray value and filter denoising of the image; Convert the image into a binary image and perform morphological processing on the image.

3. The infrared binocular camera calibration method according to claim 1, characterized in that: The step of averaging the ellipse centers to obtain the final coordinates of the centers of the concentric circle marks includes: Collect the coordinates of the center of each ellipse contained in the concentric circle identifier to form a coordinate set; The center coordinates of the ellipses in the coordinate set are averaged to obtain the final center coordinates of the concentric circle markers.

4. The infrared binocular camera calibration method according to claim 1, wherein: The step of obtaining the coordinates of the marker points in the camera internal coordinate system by sorting based on the final coordinates of the circle center includes: Determine the internal coordinate system of the camera and convert the final coordinates of the circle center to the internal coordinate system of the camera; Calculate the camera internal coordinate value of each circle center final coordinate based on the final coordinate of the circle center that has been converted to the camera internal coordinate system; Sort the coordinates of the marker points in the camera's internal coordinate system to obtain the coordinates of the marker points in the camera's internal coordinate system.

5. The infrared binocular camera calibration method according to claim 1, characterized in that: The step of obtaining a calibration result by matching the camera internal coordinate system with the real reference system of the calibration plate based on the coordinates of the marker points includes: Use the space transformation matrix to transform the coordinates of the marker points in the camera's internal coordinate system to the real reference system of the calibration plate; Compare the coordinates of the marker points in the real reference system of the calibration plate with the coordinates of the real marker points in the real reference system of the corresponding calibration plate to obtain a comparison result; Based on the comparison results, the error between the coordinates of the marked points and the true coordinates of the marked points is calculated, and the error is minimized based on the preset algorithm to obtain the optimization result; Data extraction is performed based on the optimization result to obtain the calibration result.

6. An infrared binocular camera calibration device, used to execute the method according to any one of claims 1 to 5, characterized in that: include: An image acquisition module, used for acquiring a preset number of photographed images of the marking plate at different object distances; A fitting and sorting module is used to use a circle detection algorithm to detect and extract the center coordinate information of each concentric circle in the concentric circle marks on the captured image, calculate the center coordinate slope between the concentric circles, and perform straight line fitting on the concentric circle marks. Based on the straight line fitting results, the distance between each concentric circle and the fitted straight line is calculated to determine the relative positional relationship between the concentric circle marks, and sort the concentric circles based on the distance between the concentric circle marks and the fitted straight line or the center coordinate information of the concentric circle marks; The ellipse fitting module is used to fit the ellipse contour of the concentric circle marks sorted in the captured image to obtain the center of the ellipse; An averaging processing module is used to average the centers of the ellipses to obtain the final coordinates of the centers of the concentric circle markers; A sorting module is used to sort the final coordinates of the circle center to obtain the coordinates of the marker points in the camera's internal coordinate system; A matching module is used to match the coordinates of the marker points through the camera's internal coordinate system and the calibration board's real reference system to obtain a calibration result; The step of fitting the concentric circle marks sorted in the captured image to an ellipse contour to obtain the center of the ellipse includes: Extract edge information from images based on edge detection algorithms; According to the number of concentric circles, the contours that meet the corresponding conditions are selected in turn; Perform ellipse fitting on the selected contour based on the ellipse fitting algorithm to obtain the ellipse parameters; The coordinates of the center of the ellipse, that is, the center of the ellipse, are extracted from the ellipse parameters.

7. An infrared binocular camera calibration device, characterized in that: include: processor; memory for storing computer programs; Wherein, when the processor executes the computer program, the infrared binocular camera calibration method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the infrared binocular camera calibration method according to any one of claims 1 to 5 is implemented.

Images