Camera internal reference calibration adaptive angular point method and system
Through the adaptive corner point detection method, combined with grayscale processing and internal angle removal strategies, the difficulty of corner point extraction in traditional methods when the checkerboard is incomplete is solved, and high-precision inner corner point extraction and coverage improvement are achieved.
Patent Information
- Application Number
- CN202510172536.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-17
- Publication Date
- 2025-05-30
AI Technical Summary
Traditional camera internal parameter calibration methods are difficult to accurately extract the inner corner points when the checkerboard is incomplete, resulting in a reduced corner point coverage and affecting system performance and reliability.
Adaptive corner point detection method is adopted to reduce image complexity through grayscale processing and thresholding processing, corner point detection algorithm based on the function database detects corner points, and interfering corner points are removed through the internal angle removal strategy, and the three-dimensional coordinates of corner points are finally determined.
In the case of incomplete checkerboard grids, the inner corner points are accurately extracted, the corner points coverage rate of internal parameter calibration is improved, and the accuracy and reliability of the system are improved.
Smart Images
Figure CN120070593A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of camera testing, and particularly to an adaptive corner point method and system for camera internal parameter calibration. Background Art
[0002] In today's digital age, cameras, as important image acquisition devices, have been widely used in multiple fields. Mobile phone cameras provide users with convenient shooting and recording functions; vehicle-mounted cameras play a key role in intelligent driving, such as ADAS (Advanced Driver Assistance System), AVM (Around View Monitor), etc., which can monitor the vehicle's surrounding environment in real time and assist drivers in driving safely; cameras in the medical field are used for medical image acquisition, surgical assistance, etc., providing key bases for disease diagnosis and treatment; cameras carried by drones assist in aerial photography, mapping, inspection, etc.
[0003] In the application of vehicle-mounted cameras, for systems such as ADAS and AVM to achieve accurate environmental perception and image analysis, internal parameter calibration must be carried out, and the distortion correction work is a key link. Internal parameter calibration can accurately determine the internal parameters of the camera, reduce image distortion, thereby improving the image quality and the accuracy of visual perception, which is of great significance for ensuring driving safety and enhancing the driving experience.
[0004] However, with the continuous progress of technology and the intensification of market competition, customers' requirements for camera calibration indicators are becoming increasingly stringent. Among them, the requirement for image coverage rate is particularly prominent, striving for as high a coverage rate as possible to comprehensively obtain information about the vehicle's surrounding environment. To achieve this goal, it is usually necessary to cover the board image as much as possible to the entire image, which inevitably leads to the incompleteness of the calibration card in the edge area. In this case, traditional corner point extraction and processing methods face severe challenges. If the edge corner points are not properly processed, it is very easy for the edge corner points to be erroneously removed, which in turn leads to inaccurate extraction of interior corner points, and ultimately reduces the corner point coverage rate of internal parameter calibration, seriously affecting the performance and reliability of the vehicle-mounted camera system.
[0005] Based on this, there is an urgent need for an adaptive corner point method and system for camera internal parameter calibration, which can accurately extract interior corner points in the case of an incomplete checkerboard and improve the corner point coverage rate of internal parameter calibration. Summary of the Invention
[0006] One of the purposes of the present invention is to provide an adaptive corner point method and system for camera internal parameter calibration, which can accurately extract interior corner points in the case of an incomplete checkerboard and improve the corner point coverage rate of internal parameter calibration.
[0007] To achieve the above purpose, an adaptive corner point method for camera internal parameter calibration is provided, including the following steps:
[0008] S1. Select the corresponding calibration card from the calibration card library according to the calibration requirements; the calibration library includes checkerboards and checkerboards with circles.
[0009] S2. Obtain an image.
[0010] S3. Perform image grayscale processing on the obtained image to obtain the grayscale image corresponding to the image.
[0011] S4. Perform thresholding on the grayscale image corresponding to the image to obtain the corresponding binary image.
[0012] S5. According to the grayscale image corresponding to the image, based on each function in the function database and the preset function call execution strategy, detect each corner point corresponding to the grayscale image, and draw and display the detected corner points on the image.
[0013] S6. According to the corner points displayed on the image, find the reference corner point and record the corner points adjacent to the reference corner point above, below, left, and right.
[0014] S7. Set constraint conditions according to the reference corner point and the corner points adjacent to the reference corner point above, below, left, and right. The constraint conditions are the adjacent spacing between each corner point and its adjacent corner point above or below, left or right, and the inclination angle of the reference corner point.
[0015] S8. According to the corner points displayed on the image, the constraint conditions, and the corresponding binary image, based on the preset inner corner elimination strategy, judge the corner points displayed on the image to determine whether each corner point meets the inner corner points. If not, eliminate the corner points that do not meet the inner corner points; otherwise, continue to display on the image.
[0016] S9. According to the corner points displayed on the image at this time, find the nearest adjacent blocks adjacent to the upper left, upper right, lower left, and lower right of each corner point.
[0017] S10. According to the nearest adjacent blocks adjacent to the upper left, upper right, lower left, and lower right corresponding to each corner point, match the corner points adjacent to the upper, lower, left, and right of each corner point.
[0018] S11. According to the corner points adjacent to the upper, lower, left, and right of each corner point matched, obtain the first corner points of the rows and columns, sort them in sequence for each row, and correct the rows and columns.
[0019] S12. According to the preset sorting strategy, sort the corrected rows and columns, and determine the three-dimensional coordinates corresponding to each corner point after the sorting is completed.
[0020] Technical principle and effect of this solution: In this solution, a suitable calibration card is selected from a calibration card library containing checkerboards and checkerboards with circles according to the calibration requirements, and then an image of the corresponding calibration card is obtained. Different types of calibration cards are suitable for different application scenarios and calibration requirements. For example, checkerboard calibration cards are commonly used for general camera internal parameter calibration, while checkerboard calibration cards with circles may be used in some scenarios with higher precision requirements or special environments.
[0021] Convert the obtained color image into a grayscale image to reduce the complexity of the image. Since a grayscale image has only one channel, it is more efficient to process than a color image and can also retain the key information required for corner detection.
[0022] Perform thresholding on the grayscale image to obtain a binary image. Binarization can divide the pixel values in the image into two categories, usually foreground and background, which can highlight the area where the corners are located and facilitate subsequent corner detection and processing.
[0023] Based on each function in the function database and the preset function invocation execution strategy, perform corner detection on the grayscale image. The function database may contain various corner detection algorithms. Select a suitable function for corner detection according to different requirements and image characteristics, and draw and display the detected corners on the image. Find a reference corner among the detected corners and record its adjacent corners above, below, left, and right. The reference corner can be used as a reference point for subsequent processing. Set constraint conditions according to the reference corner and its adjacent corners, including the adjacent spacing between each corner and its adjacent corner above or below, left or right, and the inclination angle of the reference corner. These constraint conditions can be used to screen and judge the validity of the corners.
[0024] Based on the corners displayed on the image, the constraint conditions, and the corresponding binary image, judge whether each corner meets the requirements of interior corners according to the preset interior corner elimination strategy. If not, eliminate it and only retain the interior corners that meet the requirements. This can remove some interfering corners and improve the accuracy of calibration. According to the remaining corners, find the nearest adjacent blocks above, right, below, and left of each corner. The information of the adjacent blocks can help determine the relative position relationship between the corners. According to the information of the adjacent blocks, match the adjacent corners above, below, left, and right of each corner to further clarify the topological structure between the corners. According to the matched adjacent corners, obtain the first corner of each row and column, sort the corners in each row in sequence, and correct the rows and columns at the same time. This step can arrange the corners according to certain rules to make them more in line with the actual calibration requirements. According to the preset sorting strategy, sort the corrected rows and columns to finally determine the three-dimensional coordinates corresponding to each corner. These three-dimensional coordinates can be used for subsequent camera internal parameter calibration calculations.
[0025] 1. Traditional corner point detection and calibration methods are easily affected by factors such as complex backgrounds and lighting changes, which can lead to false or missed corner point detections, thus affecting calibration accuracy. This solution innovatively proposes an inner corner point elimination strategy, which effectively identifies and eliminates interfering corner points based on strict constraints, such as the spacing between adjacent corner points and the tilt angle of reference corner points. This makes the corner points involved in the final calibration calculation more accurate and reliable, greatly improving the accuracy of the camera's intrinsic calibration, and can accurately extract inner corner points when the chessboard is incomplete, thereby improving the corner point coverage of the intrinsic calibration.
[0026] 2. Traditional methods often suffer from low efficiency due to complex computational processes and redundant processing steps when processing images. This solution cleverly uses grayscale processing and threshold processing in the image preprocessing stage, which greatly reduces the complexity of the image and lays an efficient foundation for subsequent corner point detection and processing. In the corner point processing process, a step-by-step screening and processing method is adopted to avoid a large amount of unnecessary calculations and significantly shorten the processing time.
[0027] 3. Most calibration methods on the market are not flexible enough to cope with diverse application scenarios and different calibration requirements. This solution takes a different approach and builds a rich library of calibration cards, including chessboards and chessboards with circles, which can accurately match the most suitable calibration card according to actual needs.
[0028] Further, the S4 also includes:
[0029] When the label card is a chessboard with circles, the binarized circles are filled with inverted colors to obtain the corresponding processed binarized image.
[0030] Beneficial effect: In the checkerboard label with a circle, the corners of the circle and the checkerboard are the key features of calibration. The binarized circle is filled with inverted colors, which can significantly improve the contrast between the circle and the checkerboard background. The circle, which may not be easily distinguished from the background in the binary image, has a clearer and sharper outline after inverted color filling, making the shape features of the circle more prominent in the image, providing a clearer target for the subsequent corner detection algorithm based on shape features, greatly enhancing the recognizability of key features.
[0031] Inverse color filling changes the pixel distribution of the circle area, making the boundary between the circle and the surrounding checkerboard clearer. During the corner point detection process, the algorithm can more accurately capture the corner point position where the circle and the checkerboard intersect. This not only reduces the false detection and missed detection of corner points caused by edge blur or noise interference, but also improves the accuracy of corner point positioning, thereby providing a more accurate data basis for the subsequent camera intrinsic calibration, and improving the reliability and accuracy of the entire calibration process.
[0032] Furthermore, the preset function call execution strategy is:
[0033] Retrieve the first detection function from the function database and set the first function parameters corresponding to the first detection function. The first function parameters include the maximum number of detected corner points, the corner point quality threshold, and the minimum distance between corner points.
[0034] Use the grayscale image gray corresponding to the image ij as the input data and input it into the first detection function to output the preliminary corner point set corresponding to the descending sorting rule of the quality degree. Here, i represents the row and j represents the column.
[0035] Retrieve the second optimization and adjustment function from the function database and set the second function parameters corresponding to the second optimization and adjustment function. The second function parameters include the search window size and the refinement accuracy.
[0036] Use the preliminary corner point set and the grayscale image gray corresponding to the image ij as the input data and input it into the second optimization and adjustment function to output the sub-pixel level corner point set that meets the refinement accuracy, and draw and display each corner point in the sub-pixel level corner point set on the image.
[0037] Beneficial effects: In this solution, this strategy calls functions with different functions in stages, significantly improving the accuracy of corner detection. First, through the first detection function combined with carefully set first function parameters, such as the maximum number of detected corner points, the corner point quality threshold, and the minimum distance between corner points, it is possible to quickly screen out possible corner points in the grayscale image and generate a preliminary corner point set. This screening mechanism based on the quality threshold and distance limit effectively excludes low-quality and overly dense corner points, reducing redundant information in subsequent processing. Then, the second optimization and adjustment function uses the preliminary corner point set and the grayscale image, and by setting second function parameters such as the search window size and the refinement accuracy, refines the preliminary corner points at the sub-pixel level and outputs the sub-pixel level corner point set that meets the refinement accuracy. This refinement operation makes the corner point positioning more accurate, can capture more subtle features in the image, provides high-precision corner point data for the calibration of the camera internal parameters, and greatly improves the accuracy of the calibration result.
[0038] In the optimization and adjustment stage, by adjusting the search window size and the refinement accuracy, it is possible to balance the calculation efficiency and the corner point accuracy. This way of flexibly adjusting parameters according to the actual situation not only ensures the high efficiency of the algorithm but also can adapt to diverse images and application scenarios, enhancing the practicality and adaptability of the entire calibration solution.
[0039] Furthermore, S6 includes:
[0040] S60: Randomly obtain one of the corner points displayed on the image as the current corner point point c, and the four corner points point adjacent to the current corner point above, below, left, and right u , point d , point l , point r ;
[0041] S61. Based on the current corner point and the four corner points adjacent to the current corner point above, below, left, and right, and based on a preset judgment rule, determine whether the current corner point is satisfied. If so, the current corner point is the reference corner point point[i], and record the corner points adjacent to the reference corner point point[i] above, below, left, and right. Otherwise, continue to execute S60 until the reference corner point point[i] is found;
[0042] The preset judgment rule is:
[0043]
[0044] In the formula, D u , D d , D l , D r respectively represent the distances adjacent to the above, below, left, and right, point u .x and point u .y represent the two-dimensional coordinates of the corner point point u .
[0045] Beneficial effect: In this solution, by randomly selecting the current corner point and determining the reference corner point according to a preset strict judgment rule, it is possible to accurately screen out the representative key corner points from among the many detected corner points.
[0046] By setting a strict judgment rule to synthesize multiple dimensions, this multi-dimensional setting makes the selected reference corner point better reflect the geometric characteristics of the checkerboard or other calibration cards in the image. This method avoids the errors that may be caused by blindly selecting corner points, provides a reliable reference point for the subsequent calibration process, and thus improves the accuracy and reliability of the entire calibration.
[0047] Furthermore, the constraint condition is:
[0048]
[0049] In the formula, UD is the adjacent spacing range between the corner point and the adjacent corner point above or below, LR is the adjacent spacing range between the corner point and the adjacent corner point on the left or right, and Δα is the inclination angle of the reference corner point.
[0050] Beneficial effects: Reasonable setting of constraint conditions can reduce the calibration error caused by improper selection of corner points or abnormal distribution of corner points. Through strict screening and limitation, it is ensured that the corner points participating in the calibration calculation have high quality and accuracy, thereby improving the reliability of the calibration results.
[0051] Furthermore, the preset interior angle rejection strategy is as follows:
[0052] S80. Initialize the decision flag flag and the offset pixel values x offsrt , y offset ;
[0053]
[0054] S81. According to the characteristics of interior angle points and the initialized offset pixel values x offsrt , y offset , find the central coordinates of the 3×3 pixel blocks corresponding to each corner point shown in the image in the four quadrants;
[0055] The central coordinates are as follows:
[0056]
[0057] In the formula, A ∈ [45, 135, 225, 315], a ∈ [-1, 1], b ∈ [-1, 1];
[0058] S82. According to the binary image corresponding to the image, judge whether the pixel values corresponding to each 3×3 pixel block all meet 0 or 225. If they meet, judge that the corresponding 3×3 pixel blocks are all black blocks or white blocks, and execute S83. If they do not meet, execute S84;
[0059] S83. Judge whether the colors of the 3×3 pixel blocks corresponding to the first quadrant and the third quadrant are the same, and whether the colors of the 3×3 pixel blocks corresponding to the second quadrant and the fourth quadrant are the same. At the same time, the colors of the 3×3 pixel blocks corresponding to the first quadrant and the third quadrant are opposite to the colors of the 3×3 pixel blocks corresponding to the second quadrant and the fourth quadrant. If so, this corner point is an interior angle point and is retained. Otherwise, execute S84;
[0060] S84. Make a decision according to the decision flag;
[0061] If the decision flag flag = true, re - execute S81. At this time, A ∈ [30, 120, 210, 300]; when re - executing S81 next time, the corresponding A ∈ [60, 150, 240, 330];
[0062] If the decision flag flag = false, determine that this corner point is not an interior angle point and reject it.
[0063] Beneficial effects: In this solution, when finding the central coordinates of the 3×3 pixel blocks corresponding to each corner point in the four quadrants, the calculation of the angle and offset can more comprehensively cover the pixel area around the corner point. According to the binary image to judge whether the pixel values of the pixel blocks are all 0 or 225, and the specific relationship of the colors of the pixel blocks in different quadrants (the first quadrant and the third quadrant are the same, the second quadrant and the fourth quadrant are the same, and these two groups of colors are opposite), the true interior corner points can be effectively screened out from many corner points, excluding the interfering corner points that do not conform to these characteristics, thereby improving the accuracy of interior corner point recognition and providing more reliable corner point data for subsequent calibration.
[0064] For different images, even if the spacing between corner points is different, this strategy can accurately find the pixel block area for judgment through reasonable offset calculation. In addition, when the judgment flag flag is true, by changing the value range of the angle A for multiple judgments, the flexibility of the strategy is further increased, enabling it to cope with various complex image situations and improving the adaptability of the strategy to different image features. Specifically, the loop step is one of the core mechanisms of the preset interior angle elimination strategy. When the judgment flag flag = true, S81 is re-executed, which means that the corner points that are not clearly determined to be interior corner points in the initial judgment are reviewed again. Each loop is not a simple repetition, but by adjusting the value range of the angle A, the 3×3 pixel blocks around each corner point are detected from different angles. This multi-round detection mechanism ensures that the review of each corner point is comprehensive and detailed enough, without missing any situation that may be an interior corner point, effectively reducing the missed detection and false detection caused by the limitations of single detection. By detecting by adjusting the angle multiple times, each time the pixel block is judged from a different angle, and by synthesizing the detection results of these different angles, it is possible to more accurately judge whether a corner point is an interior corner point. This multi-round and multi-angle detection method greatly improves the accuracy and reliability of interior corner point detection, ensuring that only the true interior corner points are retained, and providing high-quality corner point data for subsequent camera internal parameter calibration.
[0065] Further, the S9 includes:
[0066] S90. According to the corner points shown on the image at this time, find up to 12 nearest corner points of each corner point to form the nearest corner point set corresponding to each corner point; this corner point is the basic corner point;
[0067] S91. Select three corner points from the nearest corner point set in turn, and based on the preset nearest condition constraint, judge whether the three selected corner points meet one of the preset nearest conditions. If not, re-select three corner points from the nearest corner point set. If so, execute the next step;
[0068] S92. Based on the three selected corner points and under the corresponding closest constraint conditions, judge whether the three selected corner points meet the requirements according to the second judgment rule. If so, proceed to the next step; if not, repeat S91;
[0069] S93. Based on the three selected corner points and their corresponding basic corner points, determine the adjacent blocks between each corner point. According to the determined adjacent blocks, calculate the midpoints of the four sides of each adjacent block respectively, and form a new quadrilateral corresponding to the adjacent block based on the four calculated midpoints;
[0070] S94. Determine the position of the new quadrilateral formed in S93 on the image, extract the pixel values of all pixels within the quadrilateral, calculate the average value of the pixel values of all pixels within the quadrilateral, and judge whether the average value is less than the preset first threshold or greater than the preset second threshold. If so, the basic corner point and the three selected corner points form the corresponding closest adjacent block; otherwise, repeat S91;
[0071] S95. Based on the closest adjacent blocks corresponding to the basic corner point and the three selected corner points, determine the positions of the closest adjacent blocks in the image and classify each closest adjacent block.
[0072] Advantageous effects: By strictly screening out three suitable corner points from the up to 12 closest corner points of each corner point, and then determining the adjacent blocks based on these corner points, the adjacent regions closely related to each basic corner point can be accurately found. This method does not randomly determine the adjacent relationship, but based on the preset closest constraint conditions and the second judgment rule, through multiple rounds of screening and judgment, it ensures that the determined adjacent blocks are the regions that truly have a close geometric relationship with the basic corner point, thereby improving the accuracy of adjacent block positioning and providing more reliable basic data for subsequent calibration calculations.
[0073] Perform multiple conditional judgments on the selected three corner points. If the preset conditions are not met, reselect. This process effectively excludes those corner point combinations that do not meet specific geometric or feature relationships, avoiding the determination of incorrect adjacent blocks due to incorrect corner point combinations. Through this strict screening mechanism, the influence of interference factors on the determination of adjacent blocks can be reduced, making the finally determined adjacent blocks more accurately reflect the actual structure of the calibration card, and improving the stability and reliability of the entire calibration process.
[0074] The present invention also provides a camera internal parameter calibration adaptive corner point system, which uses the above-mentioned camera internal parameter calibration adaptive corner point method. BRIEF DESCRIPTION OF THE DRAWINGS
[0075] Figure 1 It is a flowchart of the camera internal parameter calibration adaptive corner point method in Embodiment 1 of the present invention;
[0076] Figure 2 It is a display diagram of the corner points detected in the first embodiment of the present invention on the image;
[0077] Figure 3 It is a schematic diagram of the reference corner points and their adjacent corner points above, below, left, and right in the first embodiment of the present invention on the image;
[0078] Figure 4 It is the inner corner point diagram after removing the non - conforming inner corner points in the first embodiment of the present invention;
[0079] Figure 5 It is a schematic diagram of the new quadrilateral formed by adjacent blocks in the first embodiment of the present invention;
[0080] Figure 6 It is a schematic diagram of a certain corner point after sorting in the first embodiment of the present invention. Detailed implementation method
[0081] The following is a further detailed description through specific implementation methods:
[0082] Embodiment 1
[0083] An adaptive corner point method for camera internal parameter calibration is basically as Figure 1 shown, and includes the following steps:
[0084] S1. According to the calibration requirements, select the corresponding calibration card from the calibration card library; the calibration library includes checkerboards and checkerboards with circles;
[0085] S2. According to the selected calibration card, obtain the corresponding image;
[0086] S3. Perform image grayscale processing on the obtained image to obtain the grayscale image corresponding to the image;
[0087] S4. Perform thresholding processing on the grayscale image corresponding to the image to obtain the corresponding binary image;
[0088] The S4 also includes: when the calibration card is a checkerboard with circles, fill and invert the color of the binary circles to obtain the processed binary image.
[0089] S5. According to the grayscale image corresponding to the image, based on each function in the function database and the preset function calling execution strategy, detect each corner point corresponding to the grayscale image, and draw and display the detected corner points on the image; as Figure 2 shown, the display of the detected corner points on the image.
[0090] The preset function calling execution strategy is:
[0091] Retrieve the first detection function from the function database and set the first function parameters corresponding to the first detection function. The first function parameters include the maximum number of detected corner points, the corner quality threshold, and the minimum distance between corner points. In this embodiment, the first detection function is the goodFeaturesToTrack function. The first function parameters also include the block size for calculating the derivative covariance matrix of each pixel neighborhood.
[0092] Take the grayscale image gray corresponding to the image ij as the input data and input it into the first detection function to output the preliminary corner point set sorted in descending order of quality. Here, i represents the row and j represents the column.
[0093] Retrieve the second optimization and adjustment function from the function database and set the second function parameters corresponding to the second optimization and adjustment function. The second function parameters include the search window size and the refinement accuracy. Generally, set the maximum number of detected corner points to 500, the corner quality threshold to 0.1, the minimum distance between corner points to 10, the block size to 7 to 11 (default is 7), the search window size to 11, and the refinement accuracy to 0.01.
[0094] Take the preliminary corner point set and the grayscale image gray corresponding to the image ij as the input data and input it into the second optimization and adjustment function to output the sub-pixel level corner point set that meets the refinement accuracy, and draw and display each corner point in the sub-pixel level corner point set on the image. In this embodiment, the second optimization and adjustment function is the cornerSubPix function.
[0095] S6. Based on the corner points shown on the image, find the reference corner point and record the corner points adjacent to the reference corner point above, below, left, and right. As Figure 3 shown, the reference corner point and the corner points adjacent to it above, below, left, and right.
[0096] The S6 includes:
[0097] S60. Based on the corner points shown on the image, randomly obtain one of the corner points as the current corner point point c , and the four corner points adjacent to the current corner point above, below, left, and right, namely point u , point d , point l , point r ;
[0098] S61. Based on the current corner point and the four corner points adjacent to it above, below, left, and right, and based on a preset judgment rule, determine whether the current corner point is satisfied. If so, the current corner point is the reference corner point point[i], and record the corner points adjacent to the reference corner point point[i] above, below, left, and right. Otherwise, continue to execute S60 until the reference corner point point[i] is found;
[0099] The preset judgment rule is:
[0100]
[0101] In the formula, D u , D d , D l , D r represent the distances to the adjacent points above, below, left, and right respectively, and point u .x and point u .y represent the two-dimensional coordinates of the corner point point u .
[0102] S7. Set constraint conditions based on the reference corner point and the corner points adjacent to it above, below, left, and right. The constraint conditions are the adjacent spacing between each corner point and its adjacent corner point above or below, left or right, and the inclination angle of the reference corner point;
[0103] The constraint conditions are:
[0104]
[0105] In the formula, UD is the range of the adjacent spacing between a corner point and its adjacent corner point above or below, LR is the range of the adjacent spacing between a corner point and its adjacent corner point left or right, and Δα is the inclination angle of the reference corner point.
[0106] S8. Based on the corner points shown in the image, the constraint conditions, and the corresponding binary image, and based on a preset interior angle rejection strategy, judge the corner points shown in the image to determine whether each corner point conforms to an interior angle point. If not, eliminate the corner points that do not conform to the interior angle points. Otherwise, continue to display them on the image; As Figure 4 shown, the interior angle point image after eliminating the corner points that do not conform to the interior angle points.
[0107] The preset interior angle rejection strategy is:
[0108] S80. Initialize the judgment flag flag and the offset pixel values x offsrt , y offset ;
[0109]
[0110] S81. According to the characteristics of the interior angle points and the initialized offset pixel values xoffsrt and y offset Find the central coordinates of the 3×3 pixel blocks corresponding to each corner point shown on the image in the four quadrants;
[0111] The central coordinates are as follows:
[0112]
[0113] where A ∈ [45, 135, 225, 315], a ∈ [-1, 1], and b ∈ [-1, 1];
[0114] S82. According to the binary image corresponding to the image, determine whether the pixel values corresponding to each 3×3 pixel block all satisfy being 0 or 225. If so, determine that the corresponding 3×3 pixel block is all black or white, and execute S83. If not, execute S84;
[0115] S83. Determine whether the colors of the 3×3 pixel blocks corresponding to the first quadrant and the third quadrant are the same, and whether the colors of the 3×3 pixel blocks corresponding to the second quadrant and the fourth quadrant are the same. At the same time, the colors of the 3×3 pixel blocks corresponding to the first quadrant and the third quadrant are opposite to the colors of the 3×3 pixel blocks corresponding to the second quadrant and the fourth quadrant. If so, this corner point is an interior corner point and is retained. Otherwise, execute S84;
[0116] S84. Make a judgment according to the judgment flag;
[0117] If the judgment flag flag = true, re - execute S81. At this time, A ∈ [30, 120, 210, 300]; when re - executing S81 for the next time, the corresponding A ∈ [60, 150, 240, 330];
[0118] If the judgment flag flag = false, determine that this corner point is not an interior corner point and eliminate it.
[0119] S9. According to the corner points shown on the image at this time, find the nearest adjacent blocks adjacent to the upper - left, upper - right, lower - left, and lower - right of each corner point;
[0120] The S9 includes:
[0121] S90. According to the corner points shown on the image at this time, find up to 12 nearest corner points of each corner point to form the nearest corner point set corresponding to each corner point; this corner point is the basic corner point;
[0122] S91. Select three corner points from the set of nearest corner points in sequence. Based on the pre-set nearest constraint condition, determine whether the three selected corner points satisfy one of the preset nearest condition. If not, re-select three corner points point[m], point[j], point[k] from the set of nearest corner points. If so, proceed to the next step;
[0123] The nearest constraint condition is:
[0124] Lower left block:
[0125]
[0126] Lower right block:
[0127]
[0128] Upper left block:
[0129]
[0130] Upper right block:
[0131]
[0132] S92. Based on the three selected corner points and under the corresponding nearest constraint condition, judge whether the three selected corner points meet the requirements according to the second judgment rule. If so, proceed to the next step. If not, repeat S91;
[0133] The second judgment rule is:
[0134]
[0135] Where β is set to 2 to 10, and the default value is 5.
[0136] S93. Determine the adjacent blocks between the selected three corner points and the corresponding basic corner points. According to the determined adjacent blocks, calculate the midpoints of the four sides of each adjacent block respectively, and form a new quadrilateral corresponding to the adjacent block based on the four calculated midpoints; as Figure 5 shown, the schematic diagram of the new quadrilateral formed by the corresponding adjacent blocks.
[0137] S94. Determine the position of the new quadrilateral formed in S93 on the image, extract the pixel values of all pixels within the quadrilateral, calculate the average value of the pixel values of all pixels within the quadrilateral, and judge whether the average value is less than the preset first threshold or greater than the preset second threshold. If so, the basic corner point and the three selected corner points form the corresponding nearest adjacent block. Otherwise, repeat S91;
[0138] S95. Determine the positions of the nearest neighbor blocks corresponding to the basic corner points and the three selected corner points in the image, and classify each nearest neighbor block.
[0139] S10. Match the upper, lower, left, and right adjacent corner points of each corner point according to the nearest neighbor blocks adjacent to the upper left, upper right, lower left, and lower right of each corner point.
[0140] S11. Obtain the first corner points of the rows and columns according to the upper, lower, left, and right adjacent corner points of each matched corner point, sort them in sequence for each row, and correct the rows and columns.
[0141] In this embodiment, for rows: in the case where there is no left adjacent corner point and it serves as the first corner point of each row, sequentially find the right adjacent corner point of this corner point as the next corner point of this row, and thus find the corner points of each row.
[0142] For columns: in the case where there is no upper adjacent corner point and it serves as the first corner point of each column, sequentially find the lower adjacent corner point of this corner point as the next corner point of this column.
[0143] When making corrections, according to the arranged corner points of each row, extract the last two corner points of a certain row and the first corner points of other rows, as well as the first corner point of this row and the first two corner points of other rows, and judge them against a preset threshold. As Figure 6 shown, a schematic diagram of corner points after a certain sorting, where the 5th point and the 6th point are not adjacent. Without correction, the 0th to 5th corner points and the 6th to 8th corner points are not in the same row, which is obviously problematic. Similarly, this situation will also occur for columns, so corrections are needed. Calculate using the 4th, 5th, 6th, and 7th corner points (the corner points are p[4], p[5], p[6], p[7])
[0144]
[0145] Among them,
[0146] S12. Sort the corrected rows and columns according to a preset sorting strategy, and determine the three-dimensional coordinates corresponding to each corner point after the sorting is completed. The ascending sorting of rows is confirmed according to a group of the longest columns; the ascending sorting of columns is confirmed according to a group of the longest rows.
[0147] Determine the number of rows and columns according to the corner points of the rows and columns. Since the obtained corner points may not enclose a convex hull, assume that the second row is the temporarily longest row row, and then sequentially confirm the corner points to be supplemented according to the first corner point and the last corner point of the second row.
[0148] First, find out whether there are other corner points in the rows in front of the corner points in the column where the first corner point in the second row is located. For example, there are none in the first row, none in the third row, and one in the fourth row. Thus, count the corner points with the largest number and place them in front of row;
[0149] Similarly, find out whether there are corner points in the rows behind the corner points in the column where the last corner point in the second row is located. For example, there are none in the first row, none in the third row, and none in the fourth row. Thus, the corner points with no such points are placed behind row;
[0150] In this way, use the found longest row and longest column for ascending sorting of rows and columns. For example, in step 14.1, the longest row is found and column sorting is performed.
[0151] If the data of the first corner point in the first column is at the first corner point of the longest row, it is arranged at the very front, and then arranged in ascending order in sequence.
[0152] The above are only embodiments of the present invention. Specific structures and common knowledge such as characteristics in the solution are described in too much detail here. Those of ordinary skill in the art know all the common technical knowledge in the technical field to which the invention belongs before the application date or priority date, can know all the existing technologies in this field, and have the ability to apply the conventional experimental means before this date. Those of ordinary skill in the art can, under the inspiration given in this application, combine their own abilities to improve and implement this solution. Some typical well-known structures or well-known methods should not become obstacles for those of ordinary skill in the art to implement this application. It should be noted that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can still be made, and these should also be regarded as the protection scope of the present invention, and these will not affect the implementation effect of the present invention and the practicality of the patent. The protection scope required by this application should be subject to the content of its claims, and the specific implementation manners and the like recorded in the specification can be used to interpret the content of the claims.
Claims
1. A camera intrinsic parameter calibration adaptive corner point method, characterized by: The following steps are involved: S1. According to the calibration requirements, a corresponding calibration card is selected from a calibration card library; the calibration card library includes a chessboard and a chessboard with a circle; S2, acquiring an image; S3, performing grayscale processing on the acquired image to obtain a grayscale image corresponding to the image; S4, performing threshold processing on the grayscale image corresponding to the image to obtain a corresponding binary image; S5. According to the grayscale image corresponding to the image, based on each function in the function database and the preset function call execution strategy, each corner point corresponding to the grayscale image is detected, and the detected corner points are drawn and displayed on the image; S6. Find the reference corner point according to the corner point displayed on the image and record the corner points adjacent to the reference corner point above, below, left and right; S7, according to the reference corner point and the corner points adjacent to the reference corner point above, below, left and right, set constraint conditions, wherein the constraint conditions are the adjacent spacing between each corner point and the adjacent corner point above, below, left or right, and the inclination angle of the reference corner point; S8, judging the corner points displayed on the image, the constraint conditions and the corresponding binary image based on the preset inner corner elimination strategy, and judging whether each corner point meets the requirements of the inner corner point. If not, the corner points that do not meet the requirements of the inner corner point are eliminated, otherwise they continue to be displayed on the image; S9, according to the corner points displayed on the image at this time, find the nearest neighboring blocks of the upper left, upper right, lower left and lower right of each corner point; S10, matching the upper, lower, left and right adjacent corner points of each corner point according to the upper left, upper right, lower left and lower right adjacent nearest neighboring blocks corresponding to each corner point; S11, obtaining the first corner point of the row and column according to the upper, lower, left and right adjacent corner points of each matched corner point, sorting each row in sequence, and correcting the row and column; S12. Sort the corrected rows and columns according to a preset sorting strategy, and determine the three-dimensional coordinates corresponding to each corner point after the sorting is completed.
2. The camera intrinsic parameter calibration adaptive corner point method according to claim 1, characterized in that: The S4 further comprises: When the label card is a chessboard with circles, the binarized circles are filled with inverted colors to obtain the corresponding processed binarized image.
3. The camera intrinsic parameter calibration adaptive corner point method according to claim 2, characterized in that: The preset function call execution strategy is: Retrieving a first detection function from a function database, and setting first function parameters corresponding to the first detection function, wherein the first function parameters include a maximum number of detected corner points, a corner point quality threshold, and a minimum distance between corner points; The grayscale image corresponding to the image ij As input data, input into the first detection function, and output the preliminary corner point set corresponding to the descending order of quality degree; Where i is the row and j is the column; Retrieving a second optimization adjustment function from a function database, and setting second function parameters corresponding to the second optimization adjustment function, wherein the second function parameters include a search window size and a refinement precision; The grayscale image corresponding to the preliminary corner point set and the image ij As input data, it is input into the second optimization adjustment function, and a sub-pixel corner point set that meets the refinement accuracy is output, and each corner point in the sub-pixel corner point set is drawn and displayed on the image.
4. The camera intrinsic parameter calibration adaptive corner point method according to claim 3, characterized in that: The S6 includes: S60, according to the corner points displayed on the image, randomly obtain one of the corner points as the current corner point c , and the four corner points adjacent to the current corner point above, below, left and right u 、point d 、point l 、point r ; S61, judging whether the current corner point satisfies the conditions based on the preset judgment rule according to the four corner points adjacent to the current corner point in the upper, lower, left and right directions. If so, the current corner point is the reference corner point point[i], and the corner points adjacent to the reference corner point point[i] in the upper, lower, left and right directions are recorded. Otherwise, S60 is continued until the reference corner point point[i] is found. The preset judgment rule is: Where D u , D d , D l , D r Respectively represent the distances to the upper, lower, left and right adjacent points. u .x and point u .y represents the corner point u The two-dimensional coordinates of .
5. The camera intrinsic parameter calibration adaptive corner point method according to claim 4, characterized in that: The constraints are: Where UD is the adjacent spacing range between the corner point and the adjacent corner point above or below, LR is the adjacent spacing range between the corner point and the adjacent corner point on the left or right, and Δα is the inclination angle of the reference corner point.
6. A camera intrinsic parameter calibration adaptive corner point method according to claim 5, characterized in that: The preset inner corner culling strategy is: S80, initializing the decision flag flag and the offset pixel value x offsrt ,y offset ; S81, according to the inner corner point characteristics and the initialized offset pixel value x offsrt ,y offset , find the center coordinates of the 3*3 pixel blocks corresponding to each corner point shown on the image in the four quadrants; The center coordinates are as follows: Where A∈[45,135,225,315], a∈[-1,1], b∈[-1,1]; S82, judging whether the pixel values corresponding to each 3*3 pixel block are all 0 or 225 according to the binarized image corresponding to the image, if so, judging whether the corresponding 3*3 pixel blocks are all black blocks or white blocks, and executing S83, if not, executing S84; S83, determine whether the colors of the 3*3 pixel blocks corresponding to the first quadrant and the third quadrant are the same, and whether the colors of the 3*3 pixel blocks corresponding to the second quadrant and the fourth quadrant are the same, and the colors of the 3*3 pixel blocks corresponding to the first quadrant and the third quadrant are opposite to the colors of the 3*3 pixel blocks corresponding to the second quadrant and the fourth quadrant. If so, the corner point is an inner corner point and is retained. Otherwise, execute S84; S84. Make a judgment according to the judgment mark; If the judgment flag flag = true, then re-execute S81, and A∈[30,120,210,300] at this time; when S81 is re-executed next time, the corresponding A∈[60,150,240,330]; If the flag is set to false, the corner point is determined not to be an inner corner point and is removed.
7. A camera intrinsic parameter calibration adaptive corner point method according to claim 6, characterized in that: The S9 includes: S90, according to the corner points displayed on the image at this time, find at most 12 nearest corner points of each corner point to form a nearest corner point set corresponding to each corner point; the corner point is a basic corner point; S91, selecting three corner points from the nearest corner point set in sequence, and judging whether the three selected corner points meet one of the preset nearest conditions based on the preset nearest condition. If not, reselecting three corner points from the nearest corner point set. If yes, executing the next step. S92, judging whether the three selected corner points meet the requirements based on the second judgment rule according to the three selected corner points and the corresponding nearest constraint condition, if yes, executing the next step, if no, repeating S91; S93, determining adjacent blocks between the three selected corner points and the corresponding basic corner points, calculating the midpoints of the four sides of each adjacent block based on the determined adjacent blocks, and constructing a new quadrilateral corresponding to the adjacent block based on the calculated four midpoints; S94, determining the position of the new quadrilateral formed in S93 on the image, extracting the pixel values of all the pixels in the quadrilateral, calculating the average value of the pixel values of all the pixels in the quadrilateral, and judging whether the average value is less than a preset first threshold value or greater than a preset second threshold value. If so, the basic corner point and the three selected corner points constitute a corresponding nearest neighboring block. Otherwise, re-execute S91. S95. Determine the position of the nearest neighbor block in the image according to the nearest neighbor blocks corresponding to the basic corner point and the three selected corner points, and classify each nearest neighbor block.
8. A camera intrinsic parameter calibration adaptive corner point system, characterized by: A camera intrinsic parameter calibration adaptive corner point method using any one of claims 1 to 7 above.