Camera intrinsic parameter calibration method, system and device based on multi-dimensional diversity evaluation

Abstract

The application discloses a camera intrinsic parameter calibration method, system and equipment based on multi-dimension diversity evaluation, and belongs to the technical field of camera calibration. The method comprises the following steps: acquiring a plurality of image frames in an image frame sequence collected by a camera; dividing a plurality of evaluation dimensions of a calibration board in the image frames into a plurality of intervals respectively, wherein the plurality of evaluation dimensions comprise relative size and inclination angle; performing calibration board corner point detection on a current image frame and screening out effective frames; determining the intervals to which the effective frames belong on each evaluation dimension and accumulating count values, and determining a comprehensive collection progress accordingly; when the comprehensive collection progress reaches a preset progress threshold or the number of effective frames reaches a preset maximum frame number, solving an intrinsic parameter matrix and distortion coefficients of the camera according to image coordinates and corresponding world coordinates of calibration board corner points of all effective frames. The application can quantitatively guide calibration data collection, ensure data diversity and automatically trigger calibration calculation, thereby improving calibration precision and efficiency.

Description

Technical Field

[0001] This invention belongs to the field of camera calibration technology, specifically relating to a camera intrinsic parameter calibration method, system, and device based on multi-dimensional diversity evaluation. Background Technology

[0002] Camera calibration is a fundamental task in applications such as robot vision, industrial automation inspection, and 3D reconstruction. Its purpose is to obtain the camera's intrinsic parameters, and the accuracy of the intrinsic parameter calibration directly affects the accuracy of subsequent image measurement and spatial calculation.

[0003] In existing technologies, camera intrinsic parameter calibration typically employs an offline processing mode. An operator holds a calibration board and moves it within the camera's field of view to acquire multiple images, which are then imported into a calibration tool for batch processing. This calibration method lacks quantitative guidance for the data acquisition process. Operators cannot easily ascertain in real time how well the acquired data meets the calibration requirements, potentially leading to data redundancy or the loss of crucial information. Furthermore, low-quality images are difficult to remove during the acquisition phase, affecting the final calibration accuracy. Additionally, the entire calibration process suffers from poor interactivity and a long calibration cycle. Summary of the Invention

[0004] To address the aforementioned problems in the existing technology, this invention provides a camera intrinsic parameter calibration method, system, and device based on multi-dimensional diversity evaluation. The technical problem to be solved by this invention is achieved through the following technical solution: In a first aspect, the present invention provides a camera intrinsic parameter calibration method based on multi-dimensional diversity assessment, comprising: Acquire multiple image frames from a sequence of image frames captured by the camera; The calibration board is divided into multiple intervals according to multiple evaluation dimensions in the image frame. The multiple evaluation dimensions include the relative size and tilt angle of the calibration board in the image frame. One of the multiple image frames is taken as the current image frame in sequence. The current image frame is subjected to calibration board corner point detection. The current image frame is then filtered based on the detected calibration board corner points. The current image frame that passes the filter is determined as a valid frame. Determine the interval to which the valid frame belongs in each evaluation dimension, and accumulate the count value of the interval; determine the overall acquisition progress based on the count value of the interval corresponding to each evaluation dimension; When the overall acquisition progress reaches a preset progress threshold, or when the number of valid frames reaches a preset maximum number of frames, the camera's intrinsic parameter matrix and distortion coefficients are solved based on the image coordinates of the calibration board corner points and the corresponding world coordinates of all valid frames.

[0005] In one embodiment of the present invention, the plurality of evaluation dimensions further include the horizontal and vertical coordinates of the calibration plate's position in the image frame; The step of dividing the calibration board into multiple intervals for each of the multiple evaluation dimensions in the image frame includes: The value range of each evaluation dimension is evenly divided into a preset number of intervals, and a counter is set for each interval.

[0006] In one embodiment of the present invention, after the step of sequentially using one of the plurality of image frames as the current image frame and performing calibration board corner detection on the current image frame, the method further includes: Iterate through adjacent corner point pairs in the calibration board corner points; For each pair of adjacent corner points, calculate the distance between the two corner points of the calibration plate, and take the minimum value among the obtained distances; The search window radius for subpixel refinement of the calibration board corner points is determined based on the minimum value. The calibration board corner points are refined to sub-pixel accuracy according to the search window radius to obtain image coordinates of the calibration board corner points with an accuracy better than one pixel.

[0007] In one embodiment of the present invention, the step of filtering the current image frame based on the detected calibration board corner points and determining the filtered current image frame as a valid frame includes: Based on the image coordinates of the detected calibration board corner points, the current image frame is subjected to quality checks, and the current image frames that pass the quality checks are determined as candidate frames; Based on the multiple evaluation dimensions, a diversity check is performed on the candidate frames, and the candidate frames that pass the diversity check are determined as valid frames.

[0008] In one embodiment of the present invention, the step of performing a quality check on the current image frame based on the image coordinates of the detected calibration board corner points, and determining the current image frame that passes the quality check as a candidate frame, includes: Based on the detected calibration board corner points and their image coordinates, it is determined whether the current image frame meets the first preset condition; if yes, the current image frame passes the quality check and is determined as a candidate frame; if no, the current image frame is discarded. The first preset condition includes: The number of detected calibration plate corner points is consistent with the theoretical number of corner points; The bounding box determined by the image coordinates of the corner points of the calibration board maintains a preset boundary margin with the boundary of the current image frame; The relative size of the calibration plate in the current image frame is within a preset size range; The calibration plate is tilted relative to the camera's optical axis at a preset angle range.

[0009] In one embodiment of the present invention, the step of performing a diversity check on the candidate frames based on the multiple evaluation dimensions, and determining the candidate frames that pass the diversity check as valid frames, includes: Based on the image coordinates of the corner points of the calibration board, the feature values ​​of the candidate frame in the multiple evaluation dimensions are calculated; Based on the feature values ​​on the multiple evaluation dimensions, it is determined whether the candidate frame meets the second preset condition; if yes, the candidate frame passes the diversity check and is determined to be a valid frame; if no, the candidate frame is discarded. The second preset condition includes: For each of the M candidate frames that most recently passed the quality check, the absolute value of the difference between the feature values ​​in each evaluation dimension and the candidate frame is greater than the corresponding preset difference threshold.

[0010] In one embodiment of the present invention, the step of determining the comprehensive acquisition progress based on the count values ​​of the intervals corresponding to each evaluation dimension includes: For each evaluation dimension, the interval where the corresponding count value reaches the preset count threshold is defined as the full interval, and the ratio of the number of full intervals to the total number of intervals under the evaluation dimension is used as the collection progress of the evaluation dimension. The average of the collection progress of each evaluation dimension is determined as the comprehensive collection progress.

[0011] In one embodiment of the present invention, the step of solving the camera's intrinsic parameter matrix and distortion coefficients based on the image coordinates of the calibration board corner points and the corresponding world coordinates of all valid frames includes: With the goal of minimizing the reprojection error, the intrinsic parameter matrix and distortion coefficients of the camera are iteratively optimized and solved. The reprojection error is the error between the projected coordinates obtained by projecting the world coordinates and the image coordinates.

[0012] Secondly, the present invention provides a camera intrinsic parameter calibration system based on multi-dimensional diversity assessment, comprising: The image acquisition module is used to acquire multiple image frames from the image frame sequence captured by the camera; The interval division module is used to divide the multiple evaluation dimensions of the calibration board in the image frame into multiple intervals, wherein the multiple evaluation dimensions include the relative size and tilt angle of the calibration board in the image frame. The corner detection and filtering module is used to sequentially take one of the multiple image frames as the current image frame, perform calibration board corner detection on the current image frame, filter the current image frames that have detected calibration board corners, and determine the current image frames that pass the filtering as valid frames. The interval counting module is used to determine the interval to which the valid frame belongs in each evaluation dimension, and to accumulate the count value of the interval to which it belongs; and to determine the comprehensive acquisition progress based on the count value of the interval corresponding to each evaluation dimension. The calibration solution module is used to solve the camera's intrinsic parameter matrix and distortion coefficients based on the image coordinates of the calibration board corner points and the corresponding world coordinates of all valid frames when the overall acquisition progress reaches a preset progress threshold or the number of valid frames reaches a preset maximum number of frames.

[0013] Thirdly, the present invention also provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the camera intrinsic parameter calibration method based on multidimensional diversity assessment as described in the first aspect.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. By dividing the multiple evaluation dimensions of the calibration board in the image frame into multiple intervals, and determining the overall acquisition progress by the interval count value under each evaluation dimension, the question of "whether the data has been collected sufficiently" that originally relied on the operator's experience is transformed into a quantifiable and real-time statistical progress indicator. The operator can supplement the missing pose according to the progress prompt, thereby reducing the reliance on the operator's professional level and shortening the calibration cycle.

[0015] 2. Through two-layer screening of quality check and diversity check, low-quality image frames are first eliminated under the first preset condition, and then redundant image frames that are not significantly different from the candidate frames that have passed the quality check are eliminated under the second preset condition. Only high-value valid frames are retained for calibration calculation, thereby reducing invalid calculations and ensuring sufficient coverage of image frames participating in calibration calculation in terms of position, size and pose, thus improving calibration accuracy and robustness.

[0016] 3. By iteratively optimizing the intrinsic parameter matrix and distortion coefficients with the goal of minimizing reprojection error, and by providing real-time feedback on the overall acquisition progress during the acquisition process, and automatically triggering the solution when the preset progress threshold is reached or the number of effective frames reaches the preset maximum number of frames, automated camera intrinsic parameter calibration is achieved, improving calibration efficiency.

[0017] 4. By adaptively determining the search window radius based on the minimum distance between adjacent corner points, the subpixel refinement search window automatically adjusts with the actual size of the calibration board in the image frame. This ensures the accuracy of the calibration corner point positioning while avoiding manual adjustment of the search window parameters, thus improving the versatility of the method.

[0018] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0019] Figure 1 This is a schematic flowchart of a camera intrinsic parameter calibration method based on multi-dimensional diversity assessment provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the interval division and counting of each evaluation dimension provided in the embodiments of the present invention; Figure 3 This is a schematic diagram of a camera intrinsic parameter calibration system based on multi-dimensional diversity evaluation provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0020] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0021] Figure 1 This is a schematic flowchart of a camera intrinsic parameter calibration method based on multi-dimensional diversity assessment provided in an embodiment of the present invention. Figure 1 As shown, this embodiment of the invention provides a camera intrinsic parameter calibration method based on multi-dimensional diversity assessment, including: S1. Obtain multiple image frames from the image frame sequence captured by the camera.

[0022] A camera refers to an imaging device that requires internal parameter calibration, including but not limited to industrial cameras, automotive cameras, and robot vision cameras; an image frame refers to a single image captured by the camera, and multiple image frames are acquired sequentially according to the order of acquisition to form an image frame sequence. In this embodiment, the camera continuously acquires field-of-view images to obtain multiple image frames. The acquisition of image frames can be achieved through video streaming, timed snapshots, or other methods, and this invention does not limit this.

[0023] S2. Divide the calibration board into multiple intervals for multiple evaluation dimensions in the image frame. The multiple evaluation dimensions include the relative size and tilt angle of the calibration board in the image frame.

[0024] The evaluation dimensions are used to characterize the imaging state of the calibration board in an image frame. In this embodiment, multiple evaluation dimensions include the relative size and tilt angle of the calibration board in the image frame. The relative size refers to the proportion of the area of ​​the calibration board's imaging range in the image frame to the total area of ​​the image frame, and the tilt angle refers to the tilt angle of the calibration board relative to the camera's optical axis. In addition, multiple evaluation dimensions may also include the horizontal and vertical coordinates of the calibration board's position in the image frame, corresponding to the normalized positions of the calibration board's center in the width and height directions of the image frame, respectively, used to characterize the positional coverage of the calibration board in the field of view. The aforementioned relative size, tilt angle, and horizontal and vertical coordinates together characterize the distance, attitude, and field of view position of the calibration board relative to the camera, comprehensively reflecting the information contribution of a single image frame to the camera's intrinsic parameter discernibility.

[0025] Optionally, the value range of each evaluation dimension can be evenly divided into a preset number of intervals. For example... Figure 2 As shown, the horizontal coordinate of the position is the normalized position of the center of the calibration board along the width of the image frame, with a value range of [0,1]. This value is evenly divided into 5 intervals: [0,0.2), [0.2,0.4), [0.4,0.6), [0.6,0.8), and [0.8,1.0], corresponding to the 5 regions of the image frame from left to right. The vertical coordinate of the position is the normalized position of the center of the calibration board along the height of the image frame, with a value range of [0,1]. This value is also evenly divided into 5 intervals, corresponding to the 5 regions of the image frame from top to bottom. The relative size ranges from [0.1,0.6] and is evenly divided into 5 intervals, covering different distance segments of the calibration board from far to near. The tilt angle is measured in degrees, with a value range of [10°,70°] and is evenly divided into 5 intervals, covering different posture segments of the calibration board from nearly facing to significantly tilted. A counter is set for each of the above intervals and is reset to zero during initialization.

[0026] It should be noted that the number of intervals and the endpoints of the value range are exemplary settings and can be adjusted according to the actual camera parameters, calibration plate specifications and operating scenarios.

[0027] S3. Sequentially, take one of the multiple image frames as the current image frame, perform calibration board corner point detection on the current image frame, and filter the current image frames based on the detected calibration board corner points. The current image frames that pass the filter are determined as valid frames.

[0028] In step S3, each image frame from the multiple image frames is taken as the current image frame, and the following processing is performed: First, calibration board corner detection is performed on the current image frame to obtain the image coordinates of all calibration board corner points in the current image frame. A calibration board refers to a planar pattern calibration object with known specifications. The corner points in the pattern that can be used as feature points are the calibration board corner points. In this embodiment, the calibration board is a checkerboard calibration board, and the calibration board corner points are the interior corner points of the checkerboard calibration board. The calibration board corner point detection can use a checkerboard corner point detection algorithm. This algorithm locates the interior corner points based on the image gradient features at the intersection of black and white squares and outputs the image coordinates of each interior corner point. The image coordinates are the coordinates of the calibration board corner points in the pixel coordinate system of the image frame. For example, the pixel coordinate system has the upper left corner of the image frame as the origin, the horizontal axis along the width direction of the image frame, and the vertical axis along the height direction of the image frame.

[0029] Next, the current image frames that detect the calibration board corner points are filtered, including quality checks and diversity checks. Specifically, a quality check is performed first, and the current image frames that pass the quality check are determined as candidate frames; then a diversity check is performed on the candidate frames, and the candidate frames that pass the diversity check are determined as valid frames.

[0030] In this embodiment, the quality check is achieved by determining whether the current image frame meets a first preset condition, which includes the following four sub-conditions: First, the number of detected calibration board corner points must match the theoretical number of corner points. The theoretical number of corner points on the calibration board is determined by its known specifications. Taking a checkerboard calibration board as an example, the detected number of interior corner points should be the product of the number of interior corners in the row direction and the number of interior corners in the column direction, which is consistent with the theoretical number of corner points. This sub-condition is used to exclude image frames where corner points are missing due to the calibration board being obstructed or partially moved out of the field of view. If the number of detected calibration board corner points in the current image frame is not equal to the theoretical number of corner points, it indicates that the calibration board imaging is incomplete and does not meet the first preset condition.

[0031] Secondly, the bounding box determined by the image coordinates of the calibration board corner points maintains a preset boundary margin with the boundary of the current image frame. The bounding box of the calibration board is used to characterize the imaging range of the calibration board in the current image frame. The bounding box is the smallest bounding rectangle that covers the image coordinates of the detected calibration board corner points. Optionally, the bounding box is the smallest bounding rectangle along the coordinate axes of the pixel coordinate system of the current image frame, with its four sides parallel to the four sides of the current image frame, determined by the minimum, maximum, minimum, and maximum values ​​of the abscissa, ordinate, and ordinate of all calibration board corner points. This sub-condition is used to exclude current image frames where the calibration board is close to the edge, as this will cause the calibration board corner points near the boundary of the current image frame to be severely distorted or exceed the imaging range. For example, the preset boundary margin is 20 pixels, meaning that the minimum distance between the four sides of the bounding box and the corresponding boundary of the current image frame is not less than 20 pixels.

[0032] Third, the relative size of the calibration board in the current image frame is within a preset size range. The relative size is the ratio of the bounding box area to the area of ​​the current image frame. This sub-condition excludes current image frames where the calibration board is too far or too close. If the calibration board is too far, the spacing between the corner points of the calibration board is too small, resulting in insufficient sub-pixel positioning accuracy; if the calibration board is too close, the imaging range exceeds the field of view. For example, the preset size range is [0.1, 0.8], meaning that the ratio of the bounding box area to the image frame area is not less than 0.1 and not more than 0.8.

[0033] Fourth, the tilt angle of the calibration board relative to the camera optical axis is within a preset angle range. This sub-condition can exclude current image frames where the calibration board is excessively tilted or where the corner features of the calibration board are difficult to extract stably. It should be noted that the tilt angle can be obtained as follows: based on the initial intrinsic parameter matrix of the camera and the correspondence between the image coordinates of the calibration board corners and their corresponding world coordinates, the pose of the calibration board relative to the camera is solved; the normal vector of the calibration board is obtained from the rotation component in the pose; the angle between the normal vector and the camera optical axis is the tilt angle. Here, the initial intrinsic parameter matrix of the camera is the estimated value of the camera intrinsic parameters used to solve the pose, which can be estimated from the resolution of the current image frame. For example, the initial focal length is set to equal the width of the current image frame, and the initial value of the distortion coefficient is set to zero. For example, the preset angle range is... .

[0034] During the quality check, the current image frame is determined to pass the quality check and is identified as a candidate frame if and only if all four sub-conditions in the first preset condition are met simultaneously; if any sub-condition is not met, the current image frame is determined to fail the quality check and is discarded.

[0035] Furthermore, a diversity check is performed on the candidate frames that pass the quality check. The diversity check is achieved by determining whether the candidate frames meet the second preset condition. To perform the diversity check, the feature values ​​of the candidate frames in multiple evaluation dimensions are first calculated based on the image coordinates of the calibration board corner points. The feature values ​​include the specific values ​​of each evaluation dimension, such as the horizontal coordinate of the candidate frame's position, the vertical coordinate of its position, its relative size, and its tilt angle.

[0036] In this embodiment, a sliding window that records candidate frames in chronological order can be maintained. The sliding window stores the M most recent candidate frames that have passed the quality check. For example, if M is 5, the sliding window stores the 5 most recent candidate frames that have passed the quality check. It should be noted that even if a candidate frame fails the diversity check and is discarded, it is still retained in the sliding window when it passes the quality check, so as to participate in the diversity comparison of subsequent candidate frames, until the updated candidate frame is squeezed out of the sliding window.

[0037] The second preset condition includes: the absolute value of the difference between the feature values ​​of the candidate frame and each candidate frame in the sliding window in each evaluation dimension is greater than the corresponding preset difference threshold. Specifically, the absolute value of the difference between the feature values ​​of the candidate frame and each candidate frame in the sliding window in each evaluation dimension (position x-coordinate, position y-coordinate, relative size, tilt angle) is calculated. If the absolute value of the difference between the feature values ​​of the candidate frame and each candidate frame in the sliding window in each evaluation dimension is greater than the corresponding preset difference threshold, the candidate frame is determined to meet the second preset condition, that is, the candidate frame has significant differences in position, size, and pose compared with the M recently acquired candidate frames, and has sufficient information. Conversely, if the difference between the feature values ​​of a candidate frame in the sliding window and that candidate frame in a certain evaluation dimension does not exceed the corresponding preset difference threshold, the candidate frame is determined not to meet the second preset condition. The candidate frame is too close to the recently acquired candidate frames and is a redundant image frame, so it is discarded.

[0038] As an example, the preset difference thresholds for each evaluation dimension are as follows: the preset difference thresholds for the horizontal and vertical coordinates of the location are both 0.002; the preset difference threshold for relative size is 0.002; and the preset difference threshold for tilt angle is... Of course, those skilled in the art can adjust the above-mentioned preset difference threshold according to the actual camera resolution, calibration board specifications, and acquisition scenario, and the embodiments of the present invention do not limit this.

[0039] In this embodiment, an adjacent corner pair refers to two adjacent corner points of the calibration board along the row or column direction in the checkerboard calibration board. All the calibration board corner points detected in the current image frame are arranged according to the checkerboard row and column topology, so adjacent corner pairs can be determined based on the corner detection results. The distance between adjacent corner pairs refers to the Euclidean distance between two of the calibration board corner points.

[0040] In step S3, after taking one image frame from multiple image frames as the current image frame and performing calibration board corner detection on the current image frame, sub-pixel refinement can be further performed to improve the corner positioning accuracy. Specifically, this includes: traversing adjacent corner point pairs in the calibration board corner points; for each adjacent corner point pair, calculating the distance between the two calibration board corner points and taking the minimum value among the obtained distances; determining the search window radius for sub-pixel refinement of the calibration board corner points based on the minimum value; and performing sub-pixel refinement on the calibration board corner points according to the search window radius to obtain image coordinates of the calibration board corner points with an accuracy better than one pixel.

[0041] Optionally, when determining the search window radius, half of the minimum distance between the two calibration board corner points in the aforementioned adjacent corner point pair is rounded up as the search window radius, and the search window radius is not less than one pixel. Since sub-pixel refinement uses image gradient information in the neighborhood of the corner point for iterative solution, if the search window covers the adjacent corner point, the gradient of the adjacent corner point will interfere with the solution of the current calibration board corner point, causing the positioning result to be biased towards the adjacent corner point. By taking half of the minimum distance between the two calibration board corner points, it can be ensured that the search window does not cross the nearest adjacent corner point, and the effective gradient sample range can be expanded as much as possible while avoiding corner point confusion. When the calibration board is far from the camera, the spacing between the calibration board corner points becomes smaller, and the search window radius automatically decreases accordingly, preventing the fixed window from crossing the adjacent corner point in long-distance conditions. Subpixel refinement can be achieved through iterative solutions based on gradient orthogonality constraints: within the search window, an ideal calibration board corner point satisfies the property that the vector pointing from any pixel within the search window to that corner point is orthogonal to the image gradient direction at that pixel. Based on this, a system of equations is established and iteratively solved using least squares until a preset convergence condition is met or a preset number of iterations is reached, outputting the coordinates of the calibration board corner point with an accuracy better than that of a single pixel. After subpixel refinement, the corner point coordinate accuracy is improved from the pixel level to the subpixel level, for example, from an error of 0.5 pixels to an error of 0.05 to 0.1 pixels, providing high-precision coordinate input for subsequent calibration solutions.

[0042] S4. Determine the interval to which the valid frame belongs in each evaluation dimension, and accumulate the count value of the interval to which it belongs; determine the comprehensive acquisition progress based on the count value of the interval corresponding to each evaluation dimension.

[0043] For a valid frame, determine the interval it belongs to in each evaluation dimension, namely, the horizontal coordinate of the position, the vertical coordinate of the position, the relative size, and the tilt angle. That is, for the feature value of each evaluation dimension of the valid frame, find the interval division of the evaluation dimension established in step S2, determine the interval in which the feature value falls, and increment the count value of the interval by one.

[0044] Here, this embodiment explains the interval counting and acquisition progress calculation as follows. A counter is pre-set for each interval of each evaluation dimension, and a preset counting threshold is set for each evaluation dimension. For example, the preset counting threshold is 2, meaning that an interval is considered full when two valid frames are acquired. When the count value of an interval's counter reaches the preset counting threshold, that interval is marked as a full interval. For each evaluation dimension, the ratio of the number of full intervals under that evaluation dimension to the total number of intervals under that evaluation dimension is used as the acquisition progress for that evaluation dimension; then, the average of the acquisition progress for each evaluation dimension is used as the overall acquisition progress. For example, if an evaluation dimension is divided into 5 intervals, and the count value of 3 intervals reaches 2, then the acquisition progress for that evaluation dimension is 3 / 5; the acquisition progress of the four evaluation dimensions is recorded as follows: , , , The overall data collection progress is then the average of the data collection progress across all evaluation dimensions. .

[0045] In this embodiment, the acquisition progress of each evaluation dimension, including the horizontal coordinate of the position, the vertical coordinate of the position, the relative size, and the tilt angle, as well as the overall acquisition progress, can be displayed in the graphical interface. This allows the operator to see the acquisition progress in real time, enabling them to move the calibration board in a targeted manner to fill in the field of view, distance segment, or attitude segment corresponding to the uncovered interval, thus efficiently completing the data acquisition.

[0046] S5. When the overall acquisition progress reaches the preset progress threshold, or the number of valid frames reaches the preset maximum number of frames, the camera's intrinsic parameter matrix and distortion coefficients are solved based on the image coordinates of the calibration board corner points of all valid frames and the corresponding world coordinates.

[0047] The calibration calculation is triggered when either the overall acquisition progress reaches a preset progress threshold or the number of valid frames reaches a preset maximum number of frames. The preset maximum number of frames is used to prevent the acquisition process from failing to terminate when certain intervals are consistently difficult to fill; for example, the preset maximum number of frames is set to 100. Camera intrinsic parameters include an intrinsic parameter matrix and distortion coefficients. The intrinsic parameter matrix contains the focal length and principal point coordinates, while the distortion coefficients describe the radial and tangential distortion of the lens.

[0048] In step S5, when solving for the camera's intrinsic parameter matrix and distortion coefficients, the goal is to minimize the reprojection error. The intrinsic parameter matrix and distortion coefficients are iteratively optimized. The reprojection error is the error between the projected coordinates obtained by projecting world coordinates and the image coordinates. For each calibration board corner point in all valid frames selected from the start of acquisition to the triggering of calibration calculation, the world coordinates of the calibration board corner point are projected onto the image plane according to the camera imaging model based on the current intrinsic parameter matrix and distortion coefficients, resulting in projected coordinates. The distance between the projected coordinates and the image coordinates of that calibration board corner point is the reprojection error of that calibration board corner point. The sum (or mean square error) of the reprojection errors of the calibration board corner points in all valid frames is the objective function. By iteratively optimizing the intrinsic parameter matrix and distortion coefficients, this objective function is minimized. The converged intrinsic parameter matrix and distortion coefficients are the calibration results. The Levenburg-Marquardt algorithm can be used for iterative optimization.

[0049] In this embodiment, the image coordinates are the image coordinates of the calibration plate corner points with a precision better than one pixel obtained by sub-pixel refinement in step S3, and the world coordinates are the three-dimensional coordinates of the calibration plate corner points in the calibration plate coordinate system with the plane where the calibration plate is located as the reference, which are determined by the known specifications of the calibration plate.

[0050] After calibration, the intrinsic parameter matrix and distortion coefficients can be saved to a configuration file in a preset format for direct use in subsequent image measurements and spatial calculations. In this embodiment, the calibration results are saved as a configuration file, and the reprojection error statistics of the calibration process are output to measure the calibration quality.

[0051] In some other embodiments of the present invention, in order to improve the real-time performance of acquisition and processing, a multi-threaded architecture can be adopted. The first thread performs image frame acquisition and calibration board corner detection, the second thread performs the solution of intrinsic parameter matrix and distortion coefficient when the above triggering conditions are met, and the third thread performs graphical interface rendering and acquisition progress display. The threads synchronize data through thread-safe queues and atomic variables, thereby avoiding the calibration solution from blocking image acquisition and ensuring the smooth real-time feedback of the interface.

[0052] In one specific embodiment of the present invention, a resolution of [resolution value missing] is used. Industrial cameras, with an interior corner point count of A checkerboard calibration board with a grid size of 25mm was used. The four evaluation dimensions—position x-coordinate, position y-coordinate, relative size, and tilt angle—were each evenly divided into 5 intervals. A preset counting threshold of 2 and a preset progress threshold of 0.8 were set. The five most recent candidate frames that passed the quality check were recorded in a sliding window. The quality check margin was set to 20 pixels, and the preset size interval was set to... , preset angle range set Relative size dimension range division coverage Tilt angle dimensional interval division coverage Under the above settings, the operator moves the calibration board according to the progress prompts on the interface. The system filters valid frames and updates the progress in real time. When the overall acquisition progress reaches 0.8, the calibration calculation is automatically triggered.

[0053] Actual testing showed that, under the above configuration, high-quality calibration data acquisition and calibration calculation can be completed within 30 seconds, and the reprojection error of the calibration results can be stably controlled within 0.5 pixels.

[0054] Figure 3 This is a schematic diagram of a camera intrinsic parameter calibration system based on multi-dimensional diversity assessment provided in an embodiment of the present invention. Figure 3 As shown, this embodiment of the invention also provides a camera intrinsic parameter calibration system 300 based on multi-dimensional diversity assessment, comprising: Image acquisition module 301 is used to acquire multiple image frames in an image frame sequence captured by the camera; The interval division module 302 is used to divide the multiple evaluation dimensions of the calibration board in the image frame into multiple intervals. The multiple evaluation dimensions include the relative size and tilt angle of the calibration board in the image frame. The corner detection and filtering module 303 is used to sequentially take one of multiple image frames as the current image frame, perform calibration board corner detection on the current image frame, filter the current image frames that have detected calibration board corners, and determine the current image frames that pass the filtering as valid frames. The interval counting module 304 is used to determine the interval to which the valid frame belongs in each evaluation dimension, and to accumulate the count value of the interval to which it belongs; and to determine the comprehensive acquisition progress based on the count value of the interval corresponding to each evaluation dimension. The calibration solution module is used to solve the camera's intrinsic parameter matrix and distortion coefficients based on the image coordinates of the calibration board corner points and the corresponding world coordinates of all valid frames when the overall acquisition progress reaches a preset progress threshold or the number of valid frames reaches a preset maximum number of frames.

[0055] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. For example... Figure 4 As shown, this embodiment of the invention also provides an electronic device, including a processor 401, a communication interface 402, a memory 403, and a communication bus 404, wherein the processor 401, the communication interface 402, and the memory 403 communicate with each other through the communication bus 404. Memory 403 is used to store computer programs; When processor 401 executes the program stored in memory 403, it performs the following steps: Acquire multiple image frames from a sequence of image frames captured by the camera; The calibration board's multiple evaluation dimensions in the image frame are divided into multiple intervals. The multiple evaluation dimensions include the relative size and tilt angle of the calibration board in the image frame. One of the multiple image frames is taken as the current image frame in sequence. The calibration board corner point is detected on the current image frame, and the current image frame is filtered according to the detected calibration board corner points. The current image frame that passes the filter is determined as a valid frame. Determine the interval to which the valid frame belongs in each evaluation dimension, and accumulate the count value of the interval; determine the overall acquisition progress based on the count value of the interval corresponding to each evaluation dimension. When the overall acquisition progress reaches the preset progress threshold, or the number of valid frames reaches the preset maximum number of frames, the camera's intrinsic parameter matrix and distortion coefficients are solved based on the image coordinates of the calibration board corner points and the corresponding world coordinates of all valid frames.

[0056] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0057] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0058] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0059] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0060] The method provided in this invention can be applied to electronic devices. Specifically, the electronic device can be a desktop computer, a portable computer, a smart mobile terminal, a server, etc. No limitation is made herein; any electronic device that can implement this invention falls within the protection scope of this invention.

[0061] It should be noted that the system and electronic device in the embodiments of the present invention are respectively systems and electronic devices that apply the above-mentioned camera intrinsic parameter calibration method based on multi-dimensional diversity assessment. Therefore, all embodiments of the above-mentioned camera intrinsic parameter calibration method based on multi-dimensional diversity assessment are applicable to the system and electronic device, and can achieve the same or similar beneficial effects.

[0062] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0063] The use of terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples" indicates that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0064] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A camera intrinsic parameter calibration method based on multi-dimensional diversity assessment, characterized in that, include: Acquire multiple image frames from a sequence of image frames captured by the camera; The calibration board is divided into multiple intervals according to multiple evaluation dimensions in the image frame. The multiple evaluation dimensions include the relative size and tilt angle of the calibration board in the image frame. One of the multiple image frames is taken as the current image frame in sequence. The current image frame is subjected to calibration board corner point detection. The current image frame is then filtered based on the detected calibration board corner points. The current image frame that passes the filter is determined as a valid frame. Determine the interval to which the valid frame belongs in each evaluation dimension, and accumulate the count value of the interval; determine the overall acquisition progress based on the count value of the interval corresponding to each evaluation dimension; When the overall acquisition progress reaches a preset progress threshold, or when the number of valid frames reaches a preset maximum number of frames, the camera's intrinsic parameter matrix and distortion coefficients are solved based on the image coordinates of the calibration board corner points and the corresponding world coordinates of all valid frames.

2. The camera intrinsic parameter calibration method based on multi-dimensional diversity assessment according to claim 1, characterized in that, The multiple evaluation dimensions also include the horizontal and vertical coordinates of the calibration board's position in the image frame; The step of dividing the calibration board into multiple intervals for each of the multiple evaluation dimensions in the image frame includes: The value range of each evaluation dimension is evenly divided into a preset number of intervals, and a counter is set for each interval.

3. The camera intrinsic parameter calibration method based on multi-dimensional diversity assessment according to claim 1, characterized in that, After the step of sequentially using one of the plurality of image frames as the current image frame and performing calibration board corner detection on the current image frame, the method further includes: Iterate through adjacent corner point pairs in the calibration board corner points; For each pair of adjacent corner points, calculate the distance between the two corner points of the calibration plate, and take the minimum value among the obtained distances; The search window radius for subpixel refinement of the calibration board corner points is determined based on the minimum value. The calibration board corner points are refined to sub-pixel accuracy according to the search window radius to obtain image coordinates of the calibration board corner points with an accuracy better than one pixel.

4. The camera intrinsic parameter calibration method based on multi-dimensional diversity assessment according to claim 3, characterized in that, The step of filtering the current image frames based on the detected calibration board corner points and determining the filtered current image frames as valid frames includes: Based on the image coordinates of the detected calibration board corner points, the current image frame is subjected to quality checks, and the current image frames that pass the quality checks are determined as candidate frames; Based on the multiple evaluation dimensions, a diversity check is performed on the candidate frames, and the candidate frames that pass the diversity check are determined as valid frames.

5. The camera intrinsic parameter calibration method based on multi-dimensional diversity assessment according to claim 4, characterized in that, The step of performing a quality check on the current image frame based on the detected image coordinates of the calibration board corner points, and determining the current image frames that pass the quality check as candidate frames, includes: Based on the detected calibration board corner points and their image coordinates, it is determined whether the current image frame meets the first preset condition; if yes, the current image frame passes the quality check and is determined as a candidate frame; if no, the current image frame is discarded. The first preset condition includes: The number of detected calibration plate corner points is consistent with the theoretical number of corner points; The bounding box determined by the image coordinates of the corner points of the calibration board maintains a preset boundary margin with the boundary of the current image frame; The relative size of the calibration plate in the current image frame is within a preset size range; The calibration plate is tilted relative to the camera's optical axis at a preset angle range.

6. The camera intrinsic parameter calibration method based on multi-dimensional diversity assessment according to claim 4, characterized in that, The step of performing a diversity check on the candidate frames based on the multiple evaluation dimensions, and determining the candidate frames that pass the diversity check as valid frames, includes: Based on the image coordinates of the corner points of the calibration board, the feature values ​​of the candidate frame in the multiple evaluation dimensions are calculated; Based on the feature values ​​on the multiple evaluation dimensions, it is determined whether the candidate frame meets the second preset condition; if yes, the candidate frame passes the diversity check and is determined to be a valid frame; if no, the candidate frame is discarded. The second preset condition includes: For each of the M candidate frames that most recently passed the quality check, the absolute value of the difference between the feature values ​​in each evaluation dimension and the candidate frame is greater than the corresponding preset difference threshold.

7. The camera intrinsic parameter calibration method based on multi-dimensional diversity assessment according to claim 1, characterized in that, The steps for determining the overall data collection progress based on the count values ​​of the intervals corresponding to each evaluation dimension include: For each evaluation dimension, the interval where the corresponding count value reaches the preset count threshold is defined as the full interval, and the ratio of the number of full intervals to the total number of intervals under the evaluation dimension is used as the collection progress of the evaluation dimension. The average of the collection progress of each evaluation dimension is determined as the comprehensive collection progress.

8. The camera intrinsic parameter calibration method based on multi-dimensional diversity assessment according to claim 1, characterized in that, The steps for solving the camera's intrinsic parameter matrix and distortion coefficients based on the image coordinates of the calibration board corner points and their corresponding world coordinates from all valid frames include: With the goal of minimizing the reprojection error, the intrinsic parameter matrix and distortion coefficients of the camera are iteratively optimized and solved. The reprojection error is the error between the projected coordinates obtained by projecting the world coordinates and the image coordinates.

9. A camera intrinsic parameter calibration system based on multi-dimensional diversity assessment, characterized in that, include: The image acquisition module is used to acquire multiple image frames from the image frame sequence captured by the camera; The interval division module is used to divide the multiple evaluation dimensions of the calibration board in the image frame into multiple intervals, wherein the multiple evaluation dimensions include the relative size and tilt angle of the calibration board in the image frame. The corner detection and filtering module is used to sequentially take one of the multiple image frames as the current image frame, perform calibration board corner detection on the current image frame, filter the current image frames that have detected calibration board corners, and determine the current image frames that pass the filtering as valid frames. The interval counting module is used to determine the interval to which the valid frame belongs in each evaluation dimension, and to accumulate the count value of the interval to which it belongs; and to determine the comprehensive acquisition progress based on the count value of the interval corresponding to each evaluation dimension. The calibration solution module is used to solve the camera's intrinsic parameter matrix and distortion coefficients based on the image coordinates of the calibration board corner points and the corresponding world coordinates of all valid frames when the overall acquisition progress reaches a preset progress threshold or the number of valid frames reaches a preset maximum number of frames.

10. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the camera intrinsic parameter calibration method based on multidimensional diversity assessment as described in any one of claims 1-8.

Images