High-precision binocular and multi-view color camera calibration system and calibration method

Abstract

The invention belongs to the technical field of camera calibration, and discloses a high-precision binocular and multi-view color camera calibration system and calibration method. Comprising a calibration platform, a computer and an image acquisition module, the calibration platform comprises an operation table, a support and a calibration plate, and the calibration plate is provided with a known geometric pattern or a calibration point which serves as a reference for calibration and is used for calculating internal parameters and external parameters of a global camera and a local camera; the image acquisition module comprises a projector, at least one global camera and at least one local camera; and the computer is in signal connection with the image acquisition module and the calibration platform, and is used for receiving and processing the panoramic image data and the local area image data acquired by the image acquisition module, and adjusting the angle, the internal reference and the external reference of the color camera. The calibration algorithm provided by the invention can improve the calibration precision under different illumination conditions, and has obvious robustness and adaptability under illumination variation.

Description

Technical Field

[0001] The present invention relates to the technical field of camera calibration, and in particular to a high-precision binocular and multi-view color camera calibration system and calibration method. Background Art

[0002] Camera calibration is a fundamental task in computer vision, aiming to establish an accurate mapping relationship between image coordinates and physical world coordinates by determining the internal and external parameters of the camera. Calibration usually uses a calibration board with a known size, such as a checkerboard. By taking images of the calibration board from different angles, corner points or feature points in the images are extracted. Using the relationship between the physical positions of these points and the image coordinates, the internal and external parameters of the camera are calculated with the help of an optimization algorithm (such as the least squares method). In industrial applications, the calibration method of a monocular camera is commonly used, which has the advantages of simple operation and low cost. A binocular black-and-white camera can provide depth information of an object through stereo vision and has a stronger three-dimensional perception ability. Although its calibration process is more complex and vulnerable to lighting and synchronization problems, it has certain advantages in depth perception. However, a binocular black-and-white camera performs poorly in low-light or high-dynamic-range scenarios and cannot distinguish the details of an object's surface by color. In contrast, a binocular color camera not only has depth perception ability but also can improve the recognition ability of an object's surface features through color information and has great application potential. However, in a complex lighting environment, especially in the case of far and near fields of view, the high-precision calibration of a binocular color camera still faces great challenges and it is difficult to achieve ideal accuracy. Therefore, although a binocular color camera has advantages in function, high-precision calibration is still a technical problem.

[0003] Chinese Invention Patent: Publication No. "CN116309869A", titled "High-precision Calibration Algorithm for Binocular Stereo Vision Camera Based on Deep Reinforcement Learning", discloses a high-precision calibration algorithm for binocular stereo vision camera based on deep reinforcement learning, including the following steps: First step: Determine the vanishing point; Second step: Determine the distortion values in the radial and tangential imaging directions; Third step: Use the data collected by the binocular stereo vision camera as training samples, and determine the number of hidden layers and nodes of the neural network according to the size of the training samples; Fourth step: Construct a target network and update the parameters in hard mode and soft mode. When the size of the network unit must be strictly controlled, it is regarded as hard mode. When the size of the network unit is affected by the size of the overall division unit, it is regarded as soft mode; Fifth step: Collect the data of the binocular stereo vision camera as test data, input it into the deep reinforcement learning structure, calculate the external parameters of the camera, and achieve high-precision calibration of the external parameters of the binocular stereo vision camera. However, this technical solution relies on a large amount of high-quality training data, which leads to a large consumption of computing resources, a complex implementation process, and the need for strong technical support. In addition, this algorithm may face the problem of overfitting, which affects the generalization ability of the model and limits its stability and applicability in practical applications.

[0004] Chinese Invention Patent: Publication No. "CN107144241A", titled "A High-precision Measurement Method for Binocular Vision Based on Depth of Field Compensation", discloses a high-precision measurement method for binocular vision based on depth of field compensation. This method first calibrates the initial positions of the two cameras, then levels the two-dimensional target with the left and right camera planes, solves the distortion coefficients at this position, and solves the structural parameters between the left and right cameras; then, translates the planar target in the direction of the left and right camera planes for parameter calibration; establishes a radial distortion compensation model in the depth of field direction to compensate the measurement accuracy of the measurement results of different depth information, and realizes high-precision measurement of the binocular camera in the depth of field direction. By establishing a distortion model with depth of field direction and combining the calibration information of the binocular camera, this method compensates the distortion of the measured points within the spatial range with depth of field information, realizes the measurement of large-size parts with depth of field direction, and improves the three-dimensional measurement accuracy of binocular vision. However, the compensation effect of this technical solution is not good in scenes with large depths, and the stability of the measurement accuracy is easily affected under different lighting conditions and complex scenes, which limits the practical application of this method and makes it difficult to meet the precise measurement requirements in a changing environment.

[0005] Chinese Invention Patent: Publication No. "CN119251303A", titled "A Spatial Positioning Method for Multimodal Visual Fusion", discloses a spatial positioning method for multimodal visual fusion. Through a sensor joint calibration algorithm, the mutual calibration between a lidar, a short-focus camera, and a long-focus camera is achieved. The stitched images collected by the dual-angle lens of the RGB angular color camera are processed, and the picture data in the power operation scene captured by the jointly calibrated camera system is read. The image is preprocessed, and target detection is performed on the preprocessed image through a deep learning method to achieve the recognition and positioning of target objects. The lidar emits laser beams to detect the target and obtain data, combines the vision program to process the point cloud data, and performs point cloud mapping, fitting reconstruction, and position calculation on the image information obtained by the RGB angular color camera. Through the above method, the accuracy of spatial positioning in complex power operation scenes is improved, and the detection and recognition error of the position is controlled within 10 millimeters. However, this technical solution relies on multiple sensors, requires complex joint calibration, the system implementation is relatively complex, increasing the work difficulty, and this technical solution has high environmental requirements. Changes in factors such as light and reflection will affect the performance of the lidar and the camera, resulting in a decrease in positioning accuracy. Summary of the Invention

[0006] In order to solve the problems in the prior art that the method of camera calibration is very complex, has high requirements for the on-site environment, and is difficult to meet the precise measurement under variable environments, the present invention proposes a high-precision binocular and multi-camera color camera calibration system and calibration method.

[0007] The present invention is realized through the following technical solutions: It includes a calibration platform, a computer, and an image acquisition module.

[0008] The calibration platform includes an operation table, a bracket, and a calibration board. The operation table is used to carry all calibration devices; the bracket is used to support and adjust the positions and angles of the global camera, local camera, and projector. The calibration board has known geometric patterns or calibration points, serving as a reference benchmark for calibration, and is used to calculate the internal and external parameters of the global camera and local camera.

[0009] The image acquisition module includes a projector and at least one global camera and one local camera; the global camera is used to capture images of a large range of scenes and obtain panoramic image data; the local camera is used to capture high-precision local area images and obtain local area image data; the projector is used to project known patterns or light spots onto the calibration board to assist in calibrating the internal and external parameters of the binocular and multi-camera color cameras.

[0010] The computer is signal - connected to the image acquisition module and the calibration platform, and is used to receive and process the panoramic image data and local area image data acquired by the image acquisition module, and adjust the angles, internal parameters, and external parameters of the color cameras.

[0011] As a further preference, the calibration board is composed of a white background board and a number of black circular marks. The number of black circular marks is arranged in a 9×11 array, and the diameters of 4 black circular marks are larger than the rest of the black circular marks.

[0012] As a further preference, the computer is touch - type.

[0013] The present invention also provides a calibration method for the high - precision binocular and multi - binocular color camera calibration system described above, including the following steps:

[0014] S1. Separate and preliminarily calibrate the images acquired by the image acquisition module in three channels to obtain the initial weights of the three channels and the calibration results of the images of the three channels;

[0015] S2. Perform channel fusion and precise calibration on the calibration results of the images of the three channels to obtain the separate calibration results of the two cameras.

[0016] As a further preference, the step S1 includes the following steps:

[0017] S101. The global camera and the local camera of the image acquisition module acquire a pair of synchronous images, and mark the images acquired by the global camera and the local camera respectively;

[0018] S102. Judge the image source. If it comes from the global camera, execute step S103; if it comes from the local camera, execute step S104;

[0019] S103. According to the abscissa Xroi and ordinate Yroi of the upper - left corner of the rectangle and the width width and height height of the rectangle, use the rectangle drawing function to mark the region of interest ROI, and verify the validity of the region of interest ROI;

[0020] S104. Apply bicubic interpolation to the color image to fill the blank areas in the image and obtain a high - resolution color image with smooth transition;

[0021] S105. Split the high - resolution color image with smooth transition obtained in step S104 into three independent color channels of red, green, and blue;

[0022] S106. Denoise and smooth the images of each color channel using Gaussian filtering;

[0023] S107. Binarize the image processed in step S106, convert the image into black and white form, simplify the image and highlight the target area to obtain a black and white image;

[0024] S108. Use the Canny operator for edge detection to detect the significant edges in the black and white image processed in step S107;

[0025] S109. Extract the significant edges detected in step S108 as the contours of the black and white image;

[0026] S110. Calculate the minimum bounding rectangle of each contour;

[0027] S111. Fit an ellipse to each contour to obtain the center coordinates, major axis length, and minor axis length of the ellipse;

[0028] S112. Determine the image source. If it is from the global camera, execute step S113; if it is from the local camera, execute step S114;

[0029] S113. According to the abscissa Xroi and ordinate Yroi of the upper left corner of the region of interest ROI, perform offset compensation on the abscissa X and ordinate Y of the extracted image to obtain the compensated coordinates (X + Xroi, Y + Yroi);

[0030] S114. Screen the target feature points according to the center coordinates, major axis length, and minor axis length of the ellipse obtained in step S111. Select the four black circular marks with larger diameters as the target feature points, sort the four target feature points according to their relative positions, and determine the marked center positions at the four corners of the calibration board;

[0031] S115. Determine whether the number of images is greater than or equal to the set threshold 15. If the number of images is greater than or equal to 15, execute step S116; if the number of images is less than 15, execute steps S101 to S114;

[0032] S116. Perform preliminary calibration using the camera calibration method based on the checkerboard calibration board, and use the Levenberg - Marquardt least squares optimization algorithm to optimize the internal and external parameters of the two cameras to obtain the calibration results of the three - channel images respectively. The formula is as follows:

[0033]

[0034]

[0035] Where, [X l Y l Z l and [X r Yr Z r are the global world coordinates of the left and right camera target feature points respectively; [u l v l and [u r v r are the corresponding normalized pixel coordinates;

[0036] S117. According to the reprojection error of each group of each image, obtain the initial weights of the three channels of this image, and normalize the initial weights to obtain the normalized initial weights of this image. The formula is as follows:

[0037]

[0038] E i = ||X observedi - X predictedi ||

[0039]

[0040] In the formula, W i is the weight of the i-th channel, i = 0, 1, 2, respectively representing the red, green, and blue channels; E i is the reprojection error of the i-th channel, i = 0, 1, 2, respectively representing the red, green, and blue channels; ε is a small constant; X observedi is the actually observed point in the i-th channel of the image, and X predictedi is the point predicted from the three-dimensional point world coordinates through the camera model; is the normalized initial weight;

[0041] S118. Optimize the normalized initial weights in step 117 to obtain the optimized weight W i '.

[0042] As a further preference, step S2 includes the following steps:

[0043] S201. According to the calibration results of the three-channel images obtained in step S116 and the optimized weight W i ' obtained in step S118, fuse them to obtain a unified multi-channel calibration model. The formula is as follows:

[0044]

[0045] Among them, Fused Image represents the fused single-channel image, and Channel i represents the single-channel image of the i-th channel;

[0046] S202. Determine the image source. If it is from the global camera, execute step S203; if it is from the local camera, execute step S204;

[0047] S203. According to the abscissa Xroi and ordinate Yroi of the upper left corner of the image rectangle after channel fusion in step S201, as well as the width width and height height of the rectangle, use the rectangle drawing function to mark the region of interest ROI and verify the validity of the region of interest ROI;

[0048] S204. For the image from the local camera after channel fusion in step S201 and the image after drawing the ROI detection region in step S203, use the bilinear interpolation method to fill the blank region, then perform image preprocessing, and use the Canny operator for edge detection to detect the significant edges of the preprocessed image;

[0049] S205. Extract the significant edges detected in step S204 as the contours of the image;

[0050] S206. Calculate the minimum bounding rectangle of each contour;

[0051] S207. Fit an ellipse using the least squares method to obtain the accurate center coordinates, major axis length, and minor axis length of the ellipse, realizing high-precision ellipse center positioning;

[0052] S208. Determine the image source. If it is from the global camera, execute step S209; if it is from the local camera, execute step S210;

[0053] S209. According to the abscissa Xroi and ordinate Yroi of the upper left corner of the region of interest ROI marked in step S203, perform offset compensation on the abscissa X and ordinate Y of the extracted image to obtain the compensated coordinates (X + Xroi, Y + Yroi);

[0054] S210. Screen the target feature points according to the center coordinates, major axis length, and minor axis length of the ellipse obtained in step S207. Select four black circular marks with larger diameters as the target feature points, sort the four target feature points according to their relative positions, and determine the marked center positions of the four corners of the calibration board;

[0055] S211. Determine whether the number of images is greater than or equal to the set threshold 15. If the number of images is greater than or equal to 15, execute step S212; if the number of images is less than 15, execute steps S201 to S210;

[0056] S212. Use the camera calibration method based on the checkerboard calibration plate to accurately calibrate the fused image, and use the Levenberg-Marquardt least squares optimization algorithm to optimize the internal and external parameters of the two cameras, respectively obtaining the individual calibration results of the two cameras.

[0057] As a further preference, the step S114 includes the following steps:

[0058] S1141. According to the center coordinates, major axis length, and minor axis length of the ellipse obtained in step S111, select four black circular marks with larger diameters on the calibration plate as target feature points;

[0059] S1142. Calculate the sum of the pixel distances from each target feature point to the other three target feature points, and determine the relative position relationship between the four target feature points by comparing the lengths;

[0060] S1143. Select a target feature point with the shortest sum of pixel distances to the other three target feature points as the reference point, that is, the center of the circle, for determining the relative distribution of the other three target feature points;

[0061] S1144. According to the error caused by the image angle, perform slope compensation on the pixel distances of the target feature points to obtain the corrected pixel distances;

[0062] S1145. According to the corrected pixel distances and the relative position relationship with the reference point, clarify the relative distribution of the other three target feature points, and infer the directions of the x-axis and y-axis;

[0063] S1146. Sort the four target feature points according to their relative positions to determine the center positions of the marks at the four corners of the calibration plate.

[0064] As a further preference, the step S118 includes the following steps:

[0065] S1181. Construct the objective function E

[0066] S1182. According to the objective function E Obtain the Jacobian matrix J with respect to the partial derivative of each weight;

[0067] S1183. According to the Jacobian matrix J, obtain the weight change amount, and the formula is as follows:

[0068] ΔW i =(J T J + λ k I) -1 J T ∈

[0069] where, ΔWi is the weight change amount, where i = 0, 1, 2, representing the red, green, and blue channels respectively; J T J is the product of the transpose of the Jacobian matrix and itself; λ k is the damping factor, which controls the optimization step size; I is the identity matrix; ∈ is the residual vector;

[0070] S1184. According to the weight change amount ΔW i , update the weights of each channel, and the formula is as follows:

[0071] W i (t+1) = W i (t) - ΔW i

[0072] where W i (t) is the weight of each channel at the current iteration; W i (t+1) is the weight of each channel for the next iteration;

[0073] S1185. Adjust the damping factor λ according to the error change k . When the error decreases, decrease the damping factor λ k ; when the error increases, increase the damping factor λ k ;

[0074] S1186. Loop steps S1183 to S1185 until the change amount of the objective function E is less than the set threshold or reaches the set maximum number of iterations, and take the final result as the optimized weight W i ' and output it.

[0075] As a further preference, after the step S2, the following steps are further included:

[0076] S3. According to the individual calibration results of the two cameras obtained in step S2, use geometric constraints to obtain the relative external parameters of the two cameras, and the formula is as follows:

[0077]

[0078] T lr = T l - R lr · T r

[0079] where, R lr describes the rotation relationship from the left camera coordinate system to the right camera coordinate system; R r and R lThey are respectively the rotation matrices between the left and right camera world coordinates and the camera coordinates obtained in the individual calibration; T lr Describes the translation transformation from the left camera coordinate system to the right camera coordinate system; T r and T l They are respectively the translation vectors between the left and right camera world coordinates and the camera coordinates obtained in the individual calibration;

[0080] S4. Using the relative external parameters of the two cameras obtained in step S3, optimize the relative external parameters of the two cameras using the artificial bee colony algorithm.

[0081] As a further preference, step S4 includes the following steps:

[0082] S41. Take the average of the relative external parameters of the two cameras obtained in step S3 as the initial parameter, and set the upper limit △max of the search range, the lower limit △min of the search range, the total number of iterations T, and the control parameter α;

[0083] S42. Dynamically adjust the search range according to the upper limit △max of the search range, the lower limit △min of the search range, the total number of iterations T, and the control parameter α. The formula is as follows:

[0084]

[0085] where t is the current number of iterations and T is the total number of iterations;

[0086] S43. Update the bee position according to the following formula:

[0087] x ij (t + 1) = x ij (t) + μ ij △(t)(x kj (t) - x ij (t))

[0088] where x ij (t + 1) represents the position of the i-th bee in the j-th dimension at time t + 1; x ij (t) represents the position of the i-th bee in the j-th dimension at time t; μ ij is a random number in the range of [-1, 1], and x kj (t) represents the position of the reference bee selected for comparison in the j-th dimension at time t;

[0089] S44. Construct the objective function. The formula is as follows:

[0090]

[0091] where: and They are the pixel coordinates of the i-th feature point in the left and right camera images respectively; R is the rotation matrix, and T’ is the translation vector, which describe the relative external parameters of the cameras; is the point in the right camera projected onto the left camera image through the camera model to obtain the corresponding point;

[0092] S45. Compare the error of the objective function to determine whether the artificial bee colony algorithm converges. If it does not converge, continue the iteration; if it converges or reaches the maximum number of iterations, end and output the current result as the optimal relative external parameters of the two cameras.

[0093] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0094] 1. Under different lighting conditions, the calibration accuracy of the method used in the present invention is improved, indicating the robustness and adaptability of the method under lighting changes.

[0095] 2. The calibration method designed in the present invention can better handle different lighting conditions compared with the traditional calibration method. The calibration method designed in the present invention can make the calibration result more stable in low-light and overexposed environments by processing and fusing three-channel images.

[0096] 3. In augmented reality-assisted assembly, the calibration method designed in the present invention significantly improves the accuracy of external parameters and the positioning accuracy of positions, meets the high-precision requirements, and has broad industrial application potential.

[0097] 4. The present invention focuses on the calibration of binocular color cameras, avoids the complexity of coordination between multiple sensors, provides high-precision calibration results through fine image processing and optimization algorithms, simplifies the calibration system at the same time, and improves the calibration efficiency.

[0098] 5. By optimizing the weights of each channel, the present invention can effectively cope with the errors caused by lighting changes, improve the robustness of the calibration process, and even in complex environmental conditions, the system can still provide high-precision calibration results. BRIEF DESCRIPTION OF THE DRAWINGS

[0099] Figure 1 is a schematic diagram of the overall structure of the calibration platform in the calibration system of the present invention.

[0100] Figure 2 is a flow chart of channel separation and basic calibration in the calibration method of the present invention.

[0101] Figure 3 is a schematic diagram of the two-dimensional drawing of the calibration board.

[0102] Figure 4 is a flow chart of calibration accuracy optimization and multi-channel fusion in the calibration method of the present invention.

[0103] Indications in the figure:

[0104] 1-1, Global camera; 1-2, Projector; 1-3, Bracket; 1-4, Local camera; 1-5, Calibration board; 1-6, Operating table; 1-7, Computer. Specific implementation manners

[0105] The advantages and features of the present invention will be illustrated and explained by the following non-limiting description of preferred embodiments, which are given only as examples with reference to the accompanying drawings.

[0106] As Figure 1 shown, the present invention provides a high-precision binocular and multi-view color camera calibration system, including a calibration platform, a computer 1-7, and an image acquisition module.

[0107] The calibration platform includes an operating table 1-6, a bracket 1-3, and a calibration board 1-5. The operating table 1-6 is used to carry all calibration devices including a global camera 1-1, a local camera 1-4, a projector 1-2, and a calibration board 1-5. The bracket 1-3 is used to support and adjust the positions and angles of the global camera 1-1, the local camera 1-4, and the projector 1-2. The adjustable design of the bracket ensures the precise relative positions between the devices and guarantees the stability of the devices during the calibration process. The calibration board 1-5 has known geometric patterns or calibration points, serving as a reference benchmark for calibration, and is used to calculate the internal and external parameters of the global camera 1-1 and the local camera 1-4. The pattern design of the calibration board 1-5 can help accurately measure the geometric parameters of the color camera.

[0108] The image acquisition module includes a projector 1-2, at least one global camera 1-1, and one local camera 1-4. The global camera 1-1 is used to capture scene images in a large range and obtain panoramic image data. The global camera 1-1 can provide a sufficient field of view to obtain the panoramic image data required for binocular and multi-view color camera calibration. The local camera 1-4 is used to capture high-precision local area images and obtain local area image data to ensure precise capture and analysis of details during the calibration process. The projector 1-2 is used to project known patterns or light spots onto the calibration board 1-5 to assist in calibrating the internal and external parameters of the binocular and multi-view color cameras and ensure calibration accuracy.

[0109] The computer 1-7 is signal-connected to the image acquisition module and the calibration platform, and is used to store the high-precision binocular and multi-view color camera calibration method of the present invention, receive and process the panoramic image data and local area image data obtained by the image acquisition module, and adjust the angles, internal parameters, and external parameters of the color camera. The computer 1-7 can be set to be touch-type, and the touch interface is easy to operate and can monitor the calibration progress.

[0110] As Figures 2 to 4 shown, the present invention provides a high-precision binocular and multi-view color camera calibration method, including the following steps:

[0111] Step 1: Separate and preliminarily calibrate the images obtained by the image acquisition module for three channels to obtain the initial weights of the three channels and the calibration results of the images of the three channels;

[0112] Step 101: The global camera 1-1 and the local camera 1-4 of the image acquisition module acquire a pair of synchronous images, and mark the images acquired by the global camera 1-1 and the local camera 1-4 respectively. The images contain a calibration pattern with known geometric features.

[0113] Using the binocular color camera system composed of the global camera 1-1 and the local camera 1-4 as Figure 1 shown, acquire a pair of synchronous images. The images contain a calibration pattern with known geometric features. The calibration pattern is as Figure 2 shown, and a calibration board 1-5 with a 9×11 non-uniform diameter circular array is adopted. The calibration board 1-5 consists of two parts. One part is a white background board, and the other part is several black circular marks. Among the black circular marks on the calibration board 1-5, there are 4 larger circles, which are distributed in the central area of the calibration board 1-5 according to specific rules, and the remaining smaller circular marks are evenly distributed on the entire white background board. The relative positions and diameters of each black circular mark have been predetermined during the calibration process and are used as geometric references. To ensure the calibration accuracy, during the calibration process, the global camera 1-1 and the local camera 1-4 will take pictures of the calibration board from 15 different perspectives to obtain multiple groups of calibration board images at different positions. By taking pictures of the calibration pattern from different perspectives, the changes in the geometric features on the calibration board can be comprehensively captured, so as to ensure the accurate calculation of the internal and external parameters of the camera.

[0114] During the shooting process, the calibration board 1-5 is usually placed on the operating table 1-6, and the binocular camera is fixed by the bracket 1-3. To facilitate the control and adjustment of the calibration process, the operator adjusts the position of the calibration board 1-5 by moving it, and combines with the computer 1-7 to control the calibration process, so as to ensure the acquisition of high-quality images from different perspectives.

[0115] Step 102: Judge the image source. If it comes from the global camera 1-1, execute Step 103; if it comes from the local camera 1-4, execute Step 104.

[0116] The global camera 1-1 can also be called a wide-angle camera. This step is used to split the image data transmitted from the image acquisition module. If the image data is captured by the wide-angle camera, the drawing of the ROI detection area is performed to highlight the target area. However, if the image data is not captured by the wide-angle camera, the image processing can be directly carried out.

[0117] Step 103: According to the abscissa Xroi and ordinate Yroi of the upper left corner of the rectangle, as well as the width width and height height of the rectangle, use the rectangle drawing function to mark the region of interest ROI and verify the validity of the region of interest ROI.

[0118] This step needs to determine the rectangular area parameters of the region of interest ROI (Region of Interest), where Xroi and Yroi are the abscissa and ordinate of the upper left corner of the rectangle, and width and height are the width and height of the rectangle. Use the rectangle drawing function to draw a rectangle on the image according to these parameters, such as the rectangle function in OpenCV, to identify the region of interest ROI, and select appropriate colors and line widths to ensure clear visibility. Subsequently, verify the validity of the region of interest ROI to ensure that it is within the image range and accurately covers the target area, and record the parameters of the region of interest ROI.

[0119] Step 104: Apply bicubic interpolation to the color image to fill the missing areas in the image and obtain a high-resolution color image with smooth transitions.

[0120] The image generated in this step is visually continuous and smooth, and significantly reduces the jagged effect, providing clearer and more accurate image data for subsequent processing.

[0121] Step 105: Split the high-resolution color image with smooth transitions obtained in Step 104 into three independent color channels: red, green, and blue.

[0122] This step splits the original color image into three independent color channels, and processes each color channel independently to optimize its color information. After this processing, more accurate single-channel image data is obtained, making the color details in the image clearer and providing more accurate color information for subsequent image analysis and processing.

[0123] Step 106: Denoise and smooth the image of each color channel using Gaussian filtering. The weight formula of Gaussian filtering is as follows:

[0124]

[0125] Among them, W(a, b) is the weight of Gaussian filtering; σ is the standard deviation, which controls the extent of filter expansion; a and b are the coordinates relative to the center of the filter.

[0126] Through Gaussian filtering, the image can be smoothed, the influence of noise can be reduced, and the details in the image can be made clearer.

[0127] Step 107: Perform binarization processing on the image processed in step 106, convert the image into a black-and-white form, simplify the image and highlight the target area to obtain a black-and-white image.

[0128] Step 108: Use the Canny operator for edge detection to detect the significant edges in the black-and-white image processed in step 107.

[0129] Step 109: Extract the significant edges detected in step 108 as the contour of the black-and-white image.

[0130] The Canny operator can detect all edges in the image, feedback all significant edge information in the image and extract it. The extracted significant edges are represented by a set of points and integrated into the contour of the image, which can accurately represent the boundary of the object in the image.

[0131] Step 110: Calculate the minimum bounding rectangle of each contour.

[0132] Calculate the minimum bounding rectangle for each contour, and this rectangle can precisely enclose the contour completely with the minimum area.

[0133] Step 111: Perform ellipse fitting on each contour to obtain the center coordinates, major axis length, and minor axis length of the ellipse.

[0134] Perform ellipse fitting on each contour to precisely represent the shape of the object. After fitting, the center coordinates, major axis length, and minor axis length of the ellipse can be obtained.

[0135] Step 112: Determine the image source. If it comes from the global camera 1-1, then execute step 113; if it comes from the local camera 1-4, then execute step 114.

[0136] This step diverts the image data transmitted by the image acquisition module again, corresponding to steps 102 and 103. If the image data comes from a wide-angle camera, that is, the global camera 1-1, then compensate the ROI detection area drawn in step 103, that is, execute step 113; if the image data does not come from a wide-angle camera, that is, comes from the local camera 1-4, then skip step 113 and directly perform feature point sorting, that is, execute step 114.

[0137] Step 113: According to the abscissa Xroi and ordinate Yroi of the upper left corner of the region of interest ROI, perform offset compensation on the abscissa X and ordinate Y of the extracted image to obtain the compensated coordinates (X + Xroi, Y + Yroi).

[0138] Step 114: Screen the target feature points according to the center coordinates, major axis length, and minor axis length of the ellipse obtained in Step 111. Select four black circular marks with larger diameters as the target feature points, and sort the four target feature points according to their relative positions to determine the marked center positions of the four corners of the calibration plate 1-5. The specific steps are as follows:

[0139] Step 1141: According to the center coordinates, major axis length, and minor axis length of the ellipse obtained in Step 111, select four black circular marks with larger diameters on the calibration plate 1-5 as the target feature points.

[0140] Step 1142: Calculate the sum of the pixel distances from each target feature point to the other three target feature points, and determine the relative position relationship between the four target feature points by comparing the lengths.

[0141] Step 1143: Select a target feature point with the shortest sum of pixel distances to the other three target feature points as the reference point, that is, the center of the circle, to determine the relative distribution of the other three target feature points.

[0142] Step 1144: According to the error caused by the image angle, perform slope compensation on the pixel distances of the target feature points to obtain the corrected pixel distances.

[0143] Step 1145: According to the corrected pixel distances and the relative position relationship with the reference point, clarify the relative distribution of the other three target feature points, and infer the directions of the x-axis and y-axis.

[0144] Step 1146: Sort the four target feature points according to their relative positions to determine the marked center positions of the four corners of the calibration plate 1-5.

[0145] Step 115: Determine whether the number of images is greater than or equal to the set threshold 15. If the number of images is greater than or equal to 15, execute Step 116; if the number of images is less than 15, execute Steps 101 to 114.

[0146] The number of images mentioned here refers to the number of color images, and each color image contains three independent color channels: red, green, and blue.

[0147] Step 116: Conduct preliminary calibration using the camera calibration method based on the checkerboard calibration board, and optimize the internal and external parameters of the two cameras using the Levenberg-Marquardt least squares optimization algorithm to obtain the calibration results of the three-channel images respectively. The calibration results include the preliminary poses, distortion parameters, internal and external parameters of the two cameras. The formula is as follows:

[0148]

[0149] Among them, [X l Y l Z l and [X r Y r Z r (unit: mm) are the global world coordinates of the target feature points of the left and right cameras respectively; [u l v l and [u r v r are the corresponding normalized pixel coordinates, measured in pixels. Here, the left and right cameras mentioned are not clearly defined as the global camera 1-1 on the left and the local camera 1-4 on the right. The opposite setting is also possible.

[0150] The camera calibration method based on the checkerboard calibration board establishes the mapping relationship between the coordinates of the calibration board 1-5 in the world coordinate system and the image pixel coordinates by extracting the target feature points in the left and right camera images. Considering the field of view differences and global lens distortion of the left and right cameras, the Levenberg-Marquardt least squares optimization algorithm is used to optimize the internal and external parameters of the camera and estimate the internal and external parameter matrices of the camera. This process provides preliminary parameter estimates for the subsequent fine calibration and determines the preliminary pose and distortion parameters of the camera.

[0151] Step 117: Obtain the initial weights of the three channels of the image according to the reprojection error of each group of each image, and normalize the initial weights to obtain the normalized initial weights of the image. The formula is as follows:

[0152]

[0153] E i =||X observedi -X predictedi ||

[0154]

[0155] In the formula, W i is the weight of the i-th channel, where i = 0, 1, 2, representing the red, green, and blue channels respectively; E iis the reprojection error of the i-th channel, where i = 0, 1, 2, representing the red, green, and blue channels respectively; ε is a small constant, e.g., ε = 10 -6 , to ensure the numerical stability of weight calculation. X observedi is the actually observed point in the i-th channel of the image, and X predictedi is the point predicted from the three-dimensional point world coordinates through the camera model; is the initial weight after normalization.

[0156] It should be noted that for each group of reprojection errors of each image in this step, there are at least 15 images captured by the binocular color camera. After each image is split into three channels, each channel has a reprojection error, and the reprojection errors of the three channels of each image form a group of reprojection errors of the image. The relationship between the reprojection error and the channel weight can be analyzed, and the weight of each channel can be calculated using the reprojection error of each channel. To calculate the initial weights of the three channels, first initialize the weights according to the reprojection error of each channel. Calculate according to the following formula:

[0157]

[0158] where W i is the weight of the i-th channel, where i = 0, 1, 2, representing the red, green, and blue channels respectively; E i is the reprojection error of the i-th channel, which can be calculated by the formula E i = ||X observedi - X predictedi ||; ε is a small constant to prevent division-by-zero errors (e.g., ε = 10^-6), ensuring the numerical stability of weight calculation; X observedi is the actually observed point in the i-th channel of the image, and X predictedi is the point predicted from the three-dimensional point world coordinates through the camera model. Normalize the initialized weights so that the sum of the weights of all channels is 1, ensuring that the influence of each channel on the reprojection error is in a balanced state. The normalization formula is:

[0159]

[0160] This can ensure that the relative contributions between the weights are reasonable and avoid instability in optimization caused by an overly large weight of a certain channel.

[0161] Step 118: Optimize the initial weights after normalization in Step 117 to obtain the optimized weight W i '.

[0162] Step 1181: Construct the objective function E

[0163] Objective function E is the reprojection error of the fused image, which is calculated by the error between the projection results of the fused image and the real image. Therefore, the optimization objective of the present invention is to minimize the reprojection error after fusion, that is, the objective function E has the minimum value. This objective function depends on the weights of each channel and is obtained by calculating the difference between the projection error of the weighted fusion of each channel image on the image plane and the real image.

[0164] Step 1182: Obtain the Jacobian matrix J according to the partial derivative of the objective function E with respect to each weight.

[0165] Step 1183: Obtain the weight change amount according to the Jacobian matrix J. The formula is as follows:

[0166] ΔW i =(J T J + λ k I) -1 J T ∈

[0167] where ΔW i is the weight change amount, i = 0, 1, 2, respectively representing the red, green, and blue channels; J T J is the product of the transpose of the Jacobian matrix and itself; λ k is the damping factor, which controls the optimization step size; I is the identity matrix; ∈ is the residual vector.

[0168] The above formula calculates the optimal step size at each update by minimizing the sum of the squares of the reprojection errors.

[0169] Step 1184: Update the weights of each channel according to the weight change amount ΔW i . The formula is as follows:

[0170] W i (t+1) =W i (t) -ΔW i

[0171] where W i (t) is the weight of each channel at the current iteration; W i (t+1) is the weight of each channel at the next iteration.

[0172] Step 1185: Adjust the damping factor λ k according to the error change. When the error decreases, reduce the damping factor λ k, accelerate convergence; when the error increases, increase the damping factor λ k , to avoid excessive overshoot in optimization.

[0173] Step 1186, loop through steps 1183 to 1185 until the change in the reprojection error, i.e., the objective function E is less than a set threshold or reaches the set maximum number of iterations, and use the final result as the optimized weight W i ' for output.

[0174] Step 2: Perform channel fusion and precise calibration on the calibration results of the three-channel images to obtain the individual calibration results of the two cameras. The specific process is as Figure 4 shown.

[0175] Step 201: Based on the calibration results of the three-channel images obtained in step 116 and the optimized weight W i ' obtained in step 118, fuse to obtain a unified multi-channel calibration model. The fusion formula is:

[0176]

[0177] where Fused Image represents the fused single-channel image, and Channel i represents the single-channel image of the i-th channel.

[0178] This fusion process can effectively combine the information of each channel and provide more comprehensive camera calibration data.

[0179] Step 202: Determine the image source. If it is from the global camera 1-1, execute step 203; if it is from the local camera 1-4, execute step 204.

[0180] This step is the same as step 102, aiming to split the image data captured by the global camera 1-1 and the local camera 1-4, so as to highlight the target area.

[0181] Step 203: Based on the abscissa Xroi and ordinate Yroi of the upper left corner of the image rectangle after channel fusion in step 201, as well as the width width and height height of the rectangle, use the rectangle drawing function to mark the region of interest ROI and verify the validity of the region of interest ROI.

[0182] This step is basically the same as step 103.

[0183] Step 204: For the images from local cameras 1-4 after channel fusion in Step 201 and the images after drawing ROI detection regions in Step 203, use bilinear interpolation to fill in the blank regions, then perform image preprocessing, and use the Canny operator for edge detection to detect the significant edges of the preprocessed images.

[0184] In this step, bilinear interpolation method is used to fill in the blank regions of the fused single-channel images to achieve sub-pixel level detection. Subsequently, Gaussian filtering is applied for denoising to reduce the influence of noise and irregular contours on the accuracy of feature extraction. Then, the image is binarized and the Canny operator is used for edge detection to accurately identify the significant edges.

[0185] Step 205: Extract the significant edges detected in Step 204 as the contours of the image.

[0186] Step 206: Calculate the minimum bounding rectangle of each contour.

[0187] Step 207: Fit an ellipse using the least squares method to obtain the accurate center coordinates, major axis length, and minor axis length of the ellipse, realizing high-precision ellipse center positioning.

[0188] Step 208: Determine the image source. If it is from global cameras 1-1, execute Step 209; if it is from local cameras 1-4, execute Step 210.

[0189] This step is basically the same as Step 112. If the image data is from a wide-angle camera, compensate the ROI detection region drawn in Step 203, that is, execute Step 209; if the image data is not from a wide-angle camera, skip Step 209 and execute Step 210.

[0190] Step 209: According to the abscissa Xroi and ordinate Yroi of the upper left corner of the region of interest ROI marked in Step 203, perform offset compensation on the abscissa X and ordinate Y of the extracted image to obtain the compensated coordinates (X+Xroi, Y+Yroi).

[0191] Step 210: Screen the target feature points according to the center coordinates, major axis length, and minor axis length of the ellipse obtained in Step 207. Select the four black circular marks with larger diameters as the target feature points, sort the four target feature points according to their relative positions, and determine the marked center positions of the four corners of the calibration plate 1-5.

[0192] Step 211: Determine whether the number of images is greater than or equal to the set threshold of 15. If the number of images is greater than or equal to 15, execute Step 212; if the number of images is less than 15, execute Steps 201 to 210.

[0193] Step 212: Use the camera calibration method based on the checkerboard calibration board to accurately calibrate the fused image, and use the Levenberg-Marquardt least squares optimization algorithm to optimize the internal and external parameters of the two cameras, respectively obtaining the individual calibration results of the two cameras. The individual calibration results of the two cameras include the preliminary poses, distortion parameters, internal parameters, and external parameters of the two cameras.

[0194] The Levenberg-Marquardt algorithm introduces a quadratic approximation on the basis of gradient descent, making the update of the weight parameters more robust. Combining the original Levenberg-Marquardt least squares optimization algorithm for calibration parameters such as camera internal parameters and distortion parameters in the traditional camera calibration method based on the checkerboard calibration board, thus completing the optimization of both the image and parameters in camera calibration. This comprehensive optimization strategy aims to achieve a higher level of camera calibration accuracy to meet the higher requirements for accurate imaging.

[0195] Step 3: According to the individual calibration results of the two cameras obtained in Step 2, use geometric constraints to obtain the relative pose of the two cameras, that is, the relative external parameters of the two cameras. The formula is as follows:

[0196]

[0197] T lr =T l -R lr ·T r

[0198] where R lr describes the rotation relationship from the left camera coordinate system to the right camera coordinate system. R r and R l are the rotation matrices between the world coordinates and camera coordinates of the left and right cameras obtained in the individual calibration respectively. T lr describes the translation transformation from the left camera coordinate system to the right camera coordinate system. T r and T l are the translation vectors between the world coordinates and camera coordinates of the left and right cameras obtained in the individual calibration respectively. The relative external parameters of the two cameras here are the relative external parameters of the left camera with respect to the right camera.

[0199] This step is the calculation of the initial value of the relative pose of the binocular cameras. After obtaining the calibration results of the monocular cameras, that is, the individual calibration results of the two cameras, use geometric constraints to calculate the relative pose between the binocular cameras. The relative pose includes the rotation matrix and the translation vector. The left and right cameras here are only different in orientation, and there is no limitation on whether they must be the global camera 1-1 or the local camera 1-4.

[0200] The complete transformation between two camera coordinate systems can be represented as a transformation matrix by combining the rotation matrix R lr and the translation vector T lr together. The formula for their complete transformation is as follows:

[0201]

[0202] By comparing the calibration patterns captured by the binocular cameras, the initial relative pose between the cameras, i.e., the initial relative extrinsic parameters, is accurately calculated to provide an initial value for the subsequent optimization process.

[0203] Step 4: According to the relative extrinsic parameters of the two cameras obtained in Step 3, use the Artificial Bee Colony (ABC) algorithm to optimize the relative extrinsic parameters of the two cameras.

[0204] To further improve the accuracy of binocular camera calibration, during the calibration process, the Artificial Bee Colony (ABC) algorithm is used to optimize the extrinsic parameters of the binocular cameras. The Artificial Bee Colony algorithm effectively avoids the trap of local optimal solutions by simulating the foraging process of bees and combining local search and global search strategies, thus finding the globally optimal calibration result. This algorithm has significant advantages in optimizing the relative pose of the cameras and the feature matching of the calibration board 1-5.

[0205] The specific process of the Artificial Bee Colony (ABC) algorithm is as follows:

[0206] Step 41: Take the average of the relative extrinsic parameters of the two cameras obtained in Step 3 as the initial parameter, and set the upper limit of the search range △max, the lower limit of the search range △min, the total number of iterations T, and the control parameter α. The control parameter α is used to adjust the reduction speed of the search range.

[0207] Step 42: Dynamically adjust the search range according to the upper limit of the search range △max, the lower limit of the search range △min, the total number of iterations T, and the control parameter α. The formula is as follows:

[0208]

[0209] where t is the current iteration number and T is the total number of iterations.

[0210] In each iteration, the search range △ is dynamically adjusted according to the iteration number. The search range starts from △max and gradually decreases to △min according to the above formula.

[0211] Step 43: Update the bee position according to the following formula:

[0212] x ij(t + 1)= x ij (t)+ μ ij △(t)(x kj (t)- x ij (t))

[0213] Wherein, x ij (t + 1) represents the position of the i-th bee in the j-th dimension at time t + 1; x ij (t) represents the position of the i-th bee in the j-th dimension at time t; μ ij is a random number within the range of [-1, 1], and x kj (t) represents the position of the reference bee selected for comparison in the j-th dimension at time t.

[0214] Foraging bees and observing bees update their positions according to the dynamically adjusted search range. The new position is given by the following formula: x ij (t + 1)= x ij (t)+ μ ij △(t)(x kj (t)- x ij (t))

[0215] Step 44: Construct the objective function, and the formula is as follows:

[0216]

[0217] Where: and are respectively the pixel coordinates of the i-th feature point in the left and right camera images; R is the rotation matrix, and T’ is the translation vector, which describes the relative external parameters of the cameras; is the corresponding point obtained by projecting the point in the right camera onto the left camera image through the camera model.

[0218] Optimized by the Artificial Bee Colony (ABC) algorithm. In order to optimize the relative external parameters of the cameras, it is necessary to establish an objective function. Assume that N calibration images are obtained at different positions. The objective function can be expressed as the above formula.

[0219] Step 45: Compare the error of the objective function to determine whether the Artificial Bee Colony (ABC) algorithm converges. If it does not converge, continue the iteration; if it converges or reaches the maximum number of iterations, end and output the current result as the optimal relative external parameters of the two cameras.

[0220] Except for the above embodiments, the present invention may also have other implementation manners. All technical solutions formed by equivalent replacement or equivalent transformation fall within the protection scope required by the present invention.

Claims

1. A high-precision binocular and multi-eye color camera calibration system, comprising a calibration platform, a computer (1--7) and an image acquisition module, characterized in that: The calibration platform comprises an operating table (1-6), a bracket (1-3) and a calibration plate (1-5); the operating table (1-6) is used to carry all calibration equipment; the bracket (1-3) is used to support and adjust the position and angle of the global camera (1-1), the local camera (1-4) and the projector (1-2); the calibration plate (1-5) has a known geometric pattern or calibration points as a reference for calibration, and is used to calculate the internal and external parameters of the global camera (1-1) and the local camera (1-4); The image acquisition module comprises a projector (1-2) and at least one global camera (1-1) and a local camera (1-4); the global camera (1-1) is used to capture a large-scale scene image and obtain panoramic image data; the local camera (1-4) is used to capture a high-precision local area image and obtain local area image data; the projector (1-2) is used to project a known pattern or light spot onto a calibration plate (1-5) to assist in the calibration of the internal and external parameters of binocular and multi-eye color cameras; The computer (1-7) is connected to the image acquisition module and the calibration platform via signals, and is used to receive and process the panoramic image data and local area image data acquired by the image acquisition module, and to adjust the angle, internal parameters and external parameters of the color camera.

2. The high-precision binocular and multi-eye color camera calibration system according to claim 1, characterized in that: The calibration plate (1-5) consists of a white background plate and a plurality of black circular marks, wherein the plurality of black circular marks are arranged in a 9×11 array, wherein the diameters of four black circular marks are larger than the diameters of the remaining black circular marks.

3. The high-precision binocular and multi-eye color camera calibration system according to claim 2, characterized in that: The computer (1-7) is of touch type.

4. A calibration method for a high-precision binocular and multi-eye color camera calibration system according to claim 2 or 3, characterized in that: The steps include: S1, performing three-channel separation and preliminary calibration on the image acquired by the image acquisition module to obtain initial weights of the three channels and calibration results of the three-channel images; S2. Perform channel fusion and precise calibration on the calibration results of the three channel images to obtain separate calibration results of the two cameras.

5. The calibration method of the high-precision binocular and multi-eye color camera calibration system according to claim 4, characterized in that: The step S1 comprises the following steps: S101, the global camera (1-1) and the local camera (1-4) of the image acquisition module acquire a pair of synchronous images, and mark the images acquired by the global camera (1-1) and the local camera (1-4) respectively; S102, determine the image source, if it comes from the global camera (1-1), execute step S103; if it comes from the local camera (1-4), execute step S104; S103, marking the region of interest ROI using a rectangle drawing function according to the horizontal coordinate Xroi and the vertical coordinate Yroi of the upper left corner of the rectangle and the width width and the height height of the rectangle, and verifying the validity of the region of interest ROI; S104, applying bicubic interpolation to the color image to fill in the vacant areas in the image to obtain a high-resolution color image with a smooth transition; S105, splitting the smoothly transitioned high-resolution color image obtained in step S104 into three independent color channels of red, green and blue; S106, performing denoising and smoothing processing on the image of each color channel by using Gaussian filtering; S107, performing binarization processing on the image processed in step S106, converting the image into black and white, simplifying the image and highlighting the target area, and obtaining a black and white image; S108, using the Canny operator to perform edge detection to detect significant edges in the black and white image processed in step S107; S109, extracting the significant edges detected in step S108 as the outline of the black and white image; S110, calculating the minimum circumscribed rectangle of each contour; S111, performing ellipse fitting on each contour to obtain the center coordinates, major axis length, and minor axis length of the ellipse; S112, determine the image source, if it comes from the global camera (1-1), execute step S113; if it comes from the local camera (1-4), execute step S114; S113, according to the upper left corner horizontal coordinate Xroi and vertical coordinate Yroi of the region of interest ROI, offset-compensate the extracted image horizontal coordinate X and vertical coordinate Y to obtain compensated coordinates (X+Xroi, Y+Yroi); S114, filtering the target feature points according to the center coordinates, major axis length and minor axis length of the ellipse obtained in step S111, selecting four black circular marks with larger diameters as target feature points, sorting the four target feature points according to their relative positions, and determining the center positions of the marks at the four corners of the calibration plate (1-5); S115, determining whether the number of images is greater than or equal to a set threshold of 15, if the number of images is greater than or equal to 15, executing step S116; if the number of images is less than 15, executing steps S101 to S114; S116. Perform preliminary calibration using a camera calibration method based on a checkerboard calibration plate, and use the Levenberg-Marquardt least squares optimization algorithm to optimize the intrinsic and extrinsic parameters of the two cameras to obtain calibration results of the three channel images respectively; the formula is as follows: Among them, [X l Y l Z l ] and [X r Y r Z r ] are the global world coordinates of the target feature points of the left and right cameras respectively; [u l v l ] and [u r v r ] are the corresponding normalized pixel coordinates; S117, according to each group of reprojection errors of each image, the initial weights of the three channels of the image are obtained, and the initial weights are normalized to obtain the normalized initial weights of the image, and the formula is as follows: E i =||X observedi -X predictedi || Where W i is the weight of the i-th channel, i = 0, 1, 2, representing the red, green, and blue channels respectively; E i is the reprojection error of the i-th channel, i = 0, 1, 2, representing the red, green, and blue channels respectively; ε is a small constant; X observedi is the point actually observed in the image in the i-th channel, X predictedi To predict the initial point from the 3D point world coordinates through the camera model; is the normalized initial weight; S118, normalizing the initial weights in step 117 Optimize and get the optimized weight W i '.

6. The calibration method of the high-precision binocular and multi-eye color camera calibration system according to claim 5, characterized in that: The step S2 comprises the following steps: S201, based on the calibration results of the three channel images obtained in step S116 and the optimized weights W obtained in step S118 i ', and a unified multi-channel calibration model is obtained by fusion. The formula is as follows: Among them, Fused Image Represents the fused single-channel image, Channel i Represents a single-channel image of the i-th channel; S202, determine the image source, if it comes from the global camera (1-1), execute step S203; if it comes from the local camera (1-4), execute step S204; S203, according to the abscissa Xroi and ordinate Yroi of the upper left corner of the image rectangle after the channel fusion in step S201, as well as the width width and height height of the rectangle, use a rectangle drawing function to mark the region of interest ROI, and verify the validity of the region of interest ROI; S204, using the quadratic interpolation method to fill the vacant area of ​​the image from the local camera (1-4) after the channel fusion in step S201 and the image after the ROI detection area is drawn in step S203, and then performing image preprocessing, and using the Canny operator to perform edge detection to detect the significant edges of the preprocessed image; S205, extracting the significant edges detected in step S204 as the contours of the image; S206, calculating the minimum circumscribed rectangle of each contour; S207, fitting the ellipse using the least square method to obtain accurate center coordinates, major axis length, and minor axis length of the ellipse, thereby achieving high-precision center positioning of the ellipse; S208, determine the image source, if it is from the global camera (1-1), execute step S209; if it is from the local camera (1-4), execute step S210; S209, according to the upper left corner abscissa Xroi and ordinate Yroi of the region of interest ROI marked in step S203, offset-compensate the extracted image abscissa X and ordinate Y to obtain compensated coordinates (X+Xroi, Y+Yroi); S210, filtering the target feature points according to the center coordinates, major axis length and minor axis length of the ellipse obtained in step S207, selecting four black circular marks with larger diameters as target feature points, sorting the four target feature points according to their relative positions, and determining the center positions of the marks at the four corners of the calibration plate (1-5); S211, determining whether the number of images is greater than or equal to a set threshold of 15, if the number of images is greater than or equal to 15, executing step S212; if the number of images is less than 15, executing steps S201 to S210; S212. A camera calibration method based on a checkerboard calibration plate is used to accurately calibrate the fused image, and the Levenberg-Marquardt least squares optimization algorithm is used to optimize the intrinsic and extrinsic parameters of the two cameras to obtain separate calibration results for the two cameras.

7. The calibration method of the high-precision binocular and multi-eye color camera calibration system according to claim 6, characterized in that: The step S114 comprises the following steps: S1141, according to step S111, the center coordinates, the length of the major axis and the length of the minor axis of the ellipse are obtained, and four black circular marks with larger diameters on the calibration plate (1-5) are selected as target feature points; S1142, calculating the sum of the pixel distances from each target feature point to the other three target feature points, and determining the relative position relationship between the four target feature points by comparing the lengths; S1143, selecting a target feature point with the shortest sum of pixel distances to the other three target feature points as a reference point, i.e., the center of the circle, for determining the relative distribution of the other three target feature points; S1144, performing slope compensation on the pixel distance of the target feature point according to the error caused by the image angle to obtain a corrected pixel distance; S1145, according to the corrected pixel distance and the relative position relationship with the reference point, the relative distribution of the remaining three target feature points is determined, and the directions of the x-axis and the y-axis are inferred; S1146. Sort the four target feature points by relative position, and determine the mark center positions of the four corners of the calibration plate (1-5).

8. The calibration method of the high-precision binocular and multi-eye color camera calibration system according to claim 6, characterized in that: The step S118 includes the following steps: S1181, construct objective function E S1182, according to the objective function E The partial derivatives with respect to each weight give the Jacobian matrix J; S1183. According to the Jacobian matrix J, the weight change is obtained, and the formula is as follows: ΔW i =(J T J+λ k I) -1 J T ∈ Where, ΔW i is the weight change, i = 0, 1, 2, representing the red, green, and blue channels respectively; J T J is the product of the transpose of the Jacobian matrix and itself; λ k is the damping factor, controlling the optimization step size; I is the identity matrix; ∈ is the residual vector; S1184, according to the weight change ΔW i , update the weight of each channel, the formula is as follows: IN i (t+1) =In i (t) -ΔW i Where W i (t) is the weight of each channel under the current number of iterations; W i (t+1) The weight of each channel for the next iteration; S1185, adjust the damping factor λ according to the error change k , when the error decreases, reduce the damping factor λ k ; When the error increases, increase the damping factor λ k ; S1186, loop steps S1183 to S1185 until the objective function E The change in the value is less than the set threshold or reaches the set maximum number of iterations, and the final result is used as the optimized weight W i 'Output.

9. The calibration method of the high-precision binocular and multi-eye color camera calibration system according to any one of claims 7 or 8, characterized in that: The step S2 further includes the following steps: S3. Based on the individual calibration results of the two cameras obtained in step S2, the relative external parameters of the two cameras are obtained by using geometric constraints. The formula is as follows: T lr =T l -R lr ·T r Among them, R lr Describes the rotation relationship from the left camera coordinate system to the right camera coordinate system; R r and R l are the rotation matrices between the world coordinates and the camera coordinates of the left and right cameras obtained in separate calibration; T lr Describes the translation transformation from the left camera coordinate system to the right camera coordinate system; T r and T l They are the translation vectors between the world coordinates and the camera coordinates of the left and right cameras obtained in separate calibration; S4. According to the relative extrinsic parameters of the two cameras obtained in step S3, an artificial bee colony algorithm is used to optimize the relative extrinsic parameters of the two cameras.

10. The calibration method of the high-precision binocular and multi-eye color camera calibration system according to claim 9, characterized in that: The step S4 comprises the following steps: S41, taking the average value of the relative external parameters of the two cameras obtained in step S3 as the initial parameters, setting the search range upper limit △max, the search range lower limit △min, the total number of iterations T and the control parameter α; S42, dynamically adjust the search range according to the search range upper limit △max, the search range lower limit △min, the total number of iterations T and the control parameter α, the formula is as follows: Among them, t is the current iteration number, T is the total iteration number; S43, update the bee position according to the following formula: x ij (t+1)=x ij (t)+μ ij △(t)(x kj (t)-x ij (t)) Among them, x ij (t+1) represents the position of the i-th bee in the j-th dimension at time t+1; x ij (t) represents the position of the ith bee in the jth dimension at time t; μ ij is a random number in the range [-1,1], x kj (t) represents the position of the reference bee selected for comparison at time t in the jth dimension; S44. Construct the objective function, the formula is as follows: in: and are the pixel coordinates of the i-th feature point in the left and right camera images, respectively; R is the rotation matrix, and T' is the translation vector, which describes the relative extrinsic parameters of the camera; The point in the right camera is transformed through the camera model The corresponding points obtained by projecting onto the left camera image; S45. Compare the error of the objective function to determine whether the artificial bee colony algorithm has converged. If not, continue to iterate. If it has converged or the maximum number of iterations has been reached, end the process and output the current result as the optimal relative external parameter of the two cameras.

Images