Cross intersection corner detection method suitable for structured light three-dimensional reconstruction system calibration

Abstract

This invention discloses a method for detecting cross-shaped corner points suitable for calibration of structured light 3D reconstruction systems. The method includes the following steps: acquiring a cube target image by capturing an image; initially locating similar corner points in the cube target image through corner point localization; and then precisely locating the corner points, eliminating false corner points, to complete the corner point localization of the cube target. Compared to the findChessboardCorners() corner detection method provided by OpenCV, this invention significantly improves robustness, especially in cases where the angle between the cube target plane and the camera imaging plane is greater than 45° or where the cube target is partially occluded. It effectively improves the success rate, robustness, and efficiency of calibration for structured light 3D reconstruction systems.

Description

Technical Field

[0001] The present invention relates to the fields of image processing, binocular camera system or camera and projector system calibration technology, and specifically to a cross-shaped corner detection method applicable to the calibration of structured light 3D reconstruction systems. Background Technology

[0002] Among traditional methods for detecting corner points of block targets, the most commonly used method in industry is the findChessboardCorners() method provided by OpenCV. This method detects the texture features of the block target, mainly identifying and locating the corner points, ultimately achieving camera calibration or the calibration of imaging system parameters.

[0003] Based on existing literature such as "Learning OpenCV 3_Computer vision in C++ with the OpenCV library", the Automatic chessboard corner detection method, and the camera calibration algorithm based on the Harris-Zhang Zhengyou plane calibration method, for the calibration of structured light 3D reconstruction systems, when the intersection angle between the cube target plane and the camera imaging plane is greater than 45° or the cube target is partially occluded, the findChessboardCorners() corner detection method provided by OpenCV (Rufli M,Scaramuzza D,Siegwart R.Automatic detection of checkerboards on blurred and distorted images[C] / / 2008IEEE / RSJ International Conference on Intelligent Robots and Systems.IEEE,2008:3121-3126.) often fails to detect, resulting in low calibration efficiency of the structured light 3D reconstruction system. Summary of the Invention

[0004] This invention addresses the frequent failures of the `findChessboardCorners()` corner detection method provided by OpenCV in calibrating structured light 3D reconstruction systems when the intersection angle between the target plane and the camera imaging plane exceeds 45° or when the target is partially occluded. The invention proposes a method that abandons the approach of detecting corners by detecting the shape of the target blocks, instead using only the geometric features of the corners within the target blocks for detection. This invention effectively solves the problem of frequent corner detection failures of `findChessboardCorners()` when the intersection angle between the target plane and the camera imaging plane exceeds 45° or when the target is partially occluded, thus significantly improving the success rate and robustness of structured light 3D reconstruction system calibration.

[0005] The objective of this invention is achieved by at least one of the following technical solutions.

[0006] A method for detecting crossroads corner points applicable to the calibration of structured light 3D reconstruction systems includes the following steps:

[0007] S1. Obtain images of the cube target by taking pictures;

[0008] S2. By initially locating the corner points, the positions of similar corner points in the square target image are initially located;

[0009] S3. By accurately locating corner points and eliminating false corner points, the corner point location of the target block is completed.

[0010] Further, step S2 includes the following steps:

[0011] S2.1. Extract the edges of the square target image captured by the camera to obtain the edge results of the square target image;

[0012] S2.2. Based on the edge results of the block target image, candidate endpoints are detected to obtain the candidate endpoint set A;

[0013] S2.3. Using the corner and edge feature localization algorithm, false endpoints in the candidate endpoint set A are initially eliminated to obtain the initial corner set B.

[0014] Further, in step S2.2, based on the edge results of the square target image, candidate endpoints are detected to obtain a candidate endpoint set A, as follows:

[0015] Based on the geometric features of the corner points of the target block, the pixel value of edge pixels is set to 1, and the pixel value of non-edge pixels is set to 0. First, the positional relationship of each pixel in the eight-neighborhood of the pixel is defined. The center pixel in the eight-neighborhood of the pixel is defined as p1, the pixel below the center pixel is defined as p2 and used as the starting point, and the pixels around the center pixel in a clockwise direction are defined as p3 to p9 respectively.

[0016] The 'approximate cross structure' feature is constructed as follows:

[0017] When p1 in the eight-neighborhood of a pixel is an edge pixel, at least one of p3, p4 and p5 is an edge pixel, and p6 to p9 and p2 are non-edge pixels;

[0018] When pixel p1 and its eight neighbors satisfy the above characteristics, p1 is considered a candidate endpoint.

[0019] Define a 'cross-shaped' mathematical matrix, as shown by K1 in Equation 1:

[0020]

[0021] When the dot product of a pixel in the edge image of the target obtained in step S2.1 and its eight neighbors with the mathematical matrix of the 'quasi-cross structure' reaches the set condition, that is, when the value of the corresponding position of the matrix of the dot product also satisfies the 'quasi-cross structure' feature, the pixel is considered as a candidate endpoint, and then the candidate endpoint set A is obtained.

[0022] However, the candidate endpoint set A obtained above may also contain spurious endpoints. The dot product of spurious endpoints and their neighborhoods with the "cross-shaped structure" mathematical matrix will also satisfy the set feature conditions. They are generated by edge distortion caused by factors such as image noise, pixel resolution, placement angle, and position, resulting in spurious edge endpoints.

[0023] Furthermore, in step S2.3, the localization algorithm for corner points and edge features is based on the fact that correct candidate endpoints have the feature of intersecting the boundary lines of the squares; in contrast, false endpoints are located on the boundary lines of the squares and do not have the 'cross' feature.

[0024] Therefore, based on the feature that the corner points of the square target image are formed by the intersection of two mutually perpendicular straight lines, a 13*13 boundary scan rectangular window template K2 is defined to determine the 'cross intersection' feature. The boundary scan rectangular window template K2 is shown in Equation 2.

[0025]

[0026] Where, p iThe position of the candidate endpoint is represented by 1, which indicates the pixel that needs to be traversed, and the pixel that is 0 does not need to be traversed.

[0027] Traverse all candidate endpoints in the candidate endpoint set A, and make the center point of the boundary scanning rectangle window template coincide with each candidate endpoint in the candidate endpoint set A to identify the features of the boundary of the region near the candidate endpoint.

[0028] The process involves traversing the four boundaries of the overlapping area between the candidate endpoint and the boundary scanning rectangular window template in the order of top, bottom, left, and right. When an edge point (i.e., an edge pixel with a pixel value of 1) is found during the traversal of a boundary, the traversal of the current boundary is stopped, the total boundary value of the candidate endpoint is incremented by 1, and the traversal of the next boundary begins. After traversing all four boundaries, the total boundary value of the candidate endpoint is recorded. If the total boundary value of the candidate endpoint is 4, it means that there are two intersecting lines near the candidate endpoint, and the candidate endpoint is a correct candidate endpoint. Conversely, if the total boundary value of the candidate endpoint is less than 4, the candidate endpoint is a false endpoint.

[0029] By scanning all candidate endpoints in the candidate endpoint set A and removing false endpoints, the set is updated to obtain the initial corner point set B.

[0030] After the initial corner point localization operation, the resulting initial corner point set B may contain a real corner point surrounded by two or more initial corner points. In other words, these initial corner points are not spurious corner points, but rather approximations of the real corner points. Therefore, the precise corner point localization algorithm needs to select the initial corner point from the initial corner point set B that is closest to the real corner point.

[0031] Further, in step S3, a 'crosshair aiming response value' matrix K3 of size 7*7 is defined, as shown in Equation 4:

[0032]

[0033] By performing dot product and non-maximum suppression on the 'cross-point response value' matrix and one or more initial corner points in the same local region of the initial corner point set B, the accurate positioning of the true corner point is achieved.

[0034] Further, step S3 includes the following steps:

[0035] S3.1. Traverse each initial corner point in the initial corner point set B and calculate the Euclidean distance between each initial corner point and other initial corner points. If there are two or more initial corner points whose Euclidean distance is less than the set threshold, then the two or more initial corner points whose Euclidean distance is less than the set threshold are considered to be in the local region of the same theoretical corner point, and are all approximate points of the real corner point. Mark these approximate points in the local region of the same theoretical corner point.

[0036] S3.2. Perform dot product operations on the obtained approximate points and their neighborhoods in the same region with the predefined 'cross-aiming response value' matrix to obtain the response matrix corresponding to the approximate point.

[0037] S3.3 Summing the response matrix obtained in step S3.2 yields the response values ​​of each approximation point. The higher the response value, the higher the similarity between the corresponding approximation point and the true corner point. Therefore, the approximation point with the largest response value in the current local area is retained as the final correct positioning corner point.

[0038] S3.4 Return to step S3.1 until all initial corner points in the initial corner point set B have been traversed, and the correct corner point set C of the square target image is obtained.

[0039] Compared with the prior art, the advantages of the present invention are:

[0040] A survey of parameter calibration methods applied to structured light 3D reconstruction systems revealed that the most commonly used method is to detect corner points by examining the shape of a cube on a target for parameter calibration. This method can be implemented by calling the OpenCV function `findChessboardCorners()`. However, this method has poor robustness in corner point finding, especially when the angle between the target plane and the camera's imaging plane is greater than 45° or when the target is partially occluded. In these cases, `findChessboardCorners()` frequently fails to find corner points, resulting in low success rate, poor robustness, and low efficiency in structured light 3D reconstruction system calibration. Furthermore, during the calibration process, it is often necessary to place the target at a large angle (i.e., a large angle between the target plane and the camera's imaging plane) to ensure that the 3D reconstruction performance of the measured 3D object at large angles does not significantly degrade later. Therefore, it is urgent to solve the problem that corner detection often fails when the intersection angle between the target plane and the camera imaging plane is greater than 45°, and to improve the performance and efficiency of the structured light 3D reconstruction system calibration.

[0041] This invention abandons the basic idea of ​​the OpenCV-provided findChessboardCorners() corner detection method, which identifies corners by recognizing the squares of a target. The proposed "cross-shaped aiming method" works as follows: First, edges are extracted from the target image. Then, a "cross-shaped structure" mathematical matrix is ​​used to detect and filter candidate endpoints from the edge results, obtaining a candidate corner set A. Next, a 13*13 "boundary scanning rectangular window template" is used to remove false endpoints from candidate endpoint set A, obtaining the first updated initial corner set B. Then, a "cross-shaped aiming response value" matrix is ​​used to perform dot product and summation operations with multiple initial corners within the same local region to obtain the response values ​​of each approximate corner. Only the approximate corner with the largest response value within the same local region is retained, i.e., non-maximum suppression is performed. After calculating the response values ​​and performing non-maximum suppression on all initial corners in initial corner set B, the correct corner set C is obtained. By utilizing the corner point information in set C, the internal and external parameters of the structured light 3D reconstruction system can be accurately calibrated.

[0042] Compared to the `findChessboardCorners()` corner detection method provided by OpenCV, the proposed "cross-shaped corner detection method" is based on the "cross-shaped" geometric features of the cube target corners. Therefore, when the angle between the cube target plane and the camera imaging plane is greater than 45°, the local changes in the corner areas are small in the captured image of the cube target, and the impact of large-angle placement is also relatively small. This avoids the corner detection failure caused by the large cube deformation of the cube target when the angle between the cube target plane and the camera imaging plane is greater than 45°.

[0043] This is also because it utilizes the local geometric features of the target cube's corners. When part of the target cube is occluded, as long as the local features of the corners are not obscured, this method can still detect the corners. However, the basic idea behind OpenCV's findChessboardCorners() is to detect the cubes of the target cube. When some cubes of the target cube are partially occluded, it will fail to detect the corners within the cubes.

[0044] Therefore, compared with the findChessboardCorners() corner detection method provided by OpenCV, the robustness of this invention is greatly improved. In particular, for cases where the angle between the target plane and the camera imaging plane is greater than 45° and the target plane is partially occluded, this invention can effectively improve the success rate, robustness and efficiency of the calibration of the structured light 3D reconstruction system. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the chessboard edge result in Embodiment 1 of the present invention;

[0046] Figure 2 This is a schematic diagram of the geometric features of the corner points of the chessboard in Embodiment 1 of the present invention. Figure a is a schematic diagram of the perfect cross structure of the corner points of the chessboard. Figures b, c and d are schematic diagrams of several geometric features derived from the corner points of the chessboard after edge extraction due to distortion.

[0047] Figure 3 This is an example diagram of the eight-neighborhood of a pixel in Embodiment 1 of the present invention;

[0048] Figure 4 Figure 1 is a schematic diagram of the geometric features of candidate endpoints in Embodiment 1 of the present invention, wherein Figure a is a schematic diagram of correct candidate endpoints and Figure b is a schematic diagram of incorrect candidate endpoints;

[0049] Figure 5 This is a schematic diagram showing the total number of correct candidate endpoints and false endpoints in Embodiment 1 of the present invention;

[0050] Figure 6 This is a schematic diagram of the initial corner points in a local region of the same theoretical chessboard corner point in Embodiment 1 of the present invention;

[0051] Figure 7 This is a schematic diagram of the initial corner point response values ​​in Embodiment 1 of the present invention;

[0052] Figure 8 This is a flowchart illustrating the steps of the cross-shaped corner detection method applicable to the calibration of a structured light 3D reconstruction system in this embodiment of the invention.

[0053] Figure 9 This is a schematic diagram showing the total number of correct candidate endpoints and false endpoints in Embodiment 2 of the present invention;

[0054] Figure 10 This is a schematic diagram of the initial corner point response values ​​in Embodiment 2 of the present invention;

[0055] Figure 11 This is a schematic diagram illustrating the total number of boundary values ​​for correct candidate endpoints and false endpoints in Embodiment 3 of the present invention;

[0056] Figure 12 This is a schematic diagram of the initial corner point response values ​​in Embodiment 3 of the present invention. Detailed Implementation

[0057] This method addresses the issue of low calibration efficiency in structured light 3D reconstruction systems when the corner detection method provided by OpenCV, `findChessboardCorners()`, fails to detect corners when the angle between the target plane and the camera's imaging plane is greater than 45° or when the target is partially occluded. The design concept of this method abandons the basic idea of ​​`findChessboardCorners()`, which detects corners by detecting texture features. Instead, it detects corners by detecting the geometric features of the target. This approach allows the method to detect corners even when the angle between the target plane and the camera's imaging plane is greater than 45° or when the target is partially occluded, thus improving the calibration efficiency of the structured light 3D reconstruction system.

[0058] Example 1:

[0059] In this embodiment, a typical checkerboard calibration board in a cube target will be used as an example, with the angle between the plane of the checkerboard calibration board and the camera imaging plane being 20°. This method is applicable to the cross-shaped corner detection method in the calibration of a structured light 3D reconstruction system. Figure 8 As shown, it includes the following steps:

[0060] S1. Obtain an image of the chessboard calibration board by taking a picture;

[0061] S2. Initially locate similar corner points in the chessboard calibration board image through corner point localization, including the following steps:

[0062] S2.1. Perform edge extraction on the chessboard calibration board image captured by the camera to obtain the edge result of the chessboard calibration board image;

[0063] This is because the basic idea of ​​this invention only requires the geometric features of the chessboard corners to detect them. Therefore, apart from the geometric information of the chessboard corners, all other information in the chessboard calibration board image is redundant. After edge extraction of the chessboard calibration board image, only the geometric information of the edge results of the chessboard calibration board image is retained. Duplicate information within the squares of the chessboard calibration board is removed. In subsequent detection, redundant information can be avoided from having additional impact on corner detection, and the amount of subsequent computational data can also be reduced, thus improving the computational speed of the method.

[0064] After edge extraction is performed on the chessboard calibration board image captured by the camera, the edge result of the chessboard calibration board image can be obtained. The edge result of the chessboard calibration board image after edge extraction is as follows: Figure 1 As shown.

[0065] S2.2. Based on the edge results of the chessboard calibration board image, candidate endpoints are detected to obtain a candidate endpoint set A, as follows:

[0066] Based on the geometric characteristics of the corner points of the square target, a 'cross-like structure' mathematical matrix was defined;

[0067] In this embodiment, due to random variations in camera resolution and grayscale values, and the insufficient sharpness of the chessboard image edges, the extracted geometric features of the chessboard corner points will exhibit some distortion, resulting in various different geometric feature structures. Theoretically, if the above-mentioned influences did not exist, a chessboard corner point should be located in a position such as... Figure 2 The perfect local cross structure shown in Figure a. Figure 1 As shown in the actual chessboard edge results, statistical analysis of the geometric features of the chessboard corner points in the edge results of the actual chessboard image reveals several geometric features derived from the distortion of the chessboard corner points after edge extraction, such as... Figure 2 Figures b, c, and d are shown in the diagram. Figure 2 Images b, c, and d in the diagram represent classic geometric features of the chessboard corner points after edge extraction. They all share a common geometric structure, which can be used to detect chessboard corner points by retrieving this structure from the edge results of the chessboard image. This geometric structure is referred to as the 'approximate cross structure' feature in this method.

[0068] In this embodiment, edge pixels are represented by white with a pixel value of 1, and non-edge pixels are represented by black with a pixel value of 0; for example Figure 3 As shown, when p1 is an edge pixel, at least one of p3, p4, and p5 is an edge pixel, and p6 to p9 and p2 are non-edge pixels, when the eight neighbors of pixel p1 satisfy the above features, this method considers p1 as a candidate endpoint. The above features are the 'approximate cross structure' features. Therefore, this method defines a 'quasi-cross structure' mathematical matrix, as shown by K1 in Equation 1:

[0069]

[0070] Traverse the target edge image pixels obtained in step S2.1. When the dot product of a pixel and its eight neighbors in the edge result of the square target image with the 'quasi-cross structure' mathematical matrix meets the set condition, that is, when the value of the corresponding position in the matrix of the dot product also satisfies the "approximate cross structure" feature, that is, in this embodiment, perform a dot product operation on K1 with all pixels in the edge result of the chessboard image and the eight neighbors of that pixel. When the matrix of the dot product satisfies the following condition... Figure 3 When p1 is 1, at least one of p3, p4 and p5 is 1, and p6 to p9 and p2 are 0, the current pixel and its eight neighboring pixels are considered to satisfy the "approximate cross structure" feature, and the pixel is considered to be a candidate endpoint.

[0071] In this embodiment, a coordinate system is established with the first pixel at the top left corner of the chessboard calibration board image as the origin, and a set of candidate endpoints A is constructed.

[0072] However, the candidate endpoint set A obtained above may still contain false endpoints. Correct candidate endpoints satisfy the 'approximate cross structure' characteristic and are located in the local region of a chessboard corner point; false endpoints satisfy the 'approximate cross structure' characteristic but are not located in the local region of a chessboard corner point; for example... Figure 4 As shown in Figure a, pixel 1, marked by the black circle, satisfies the following conditions: pixel 1 is an edge pixel, and at least one of its eight neighboring pixels 3, 4, and 5 is an edge pixel. Pixels 2, 6, to 9 are non-edge pixels, satisfying the 'approximate cross structure' characteristic. Furthermore, it is located near a local area of ​​a chessboard corner point, making it a correct candidate endpoint. And as... Figure 4 As shown in Figure b, although the pixels marked by the black circles also satisfy the 'approximate cross structure' feature, their positions are not near the local area of ​​the chessboard corners, so they are incorrect endpoints and need to be removed.

[0073] S2.3. Using the corner and edge feature localization algorithm, false endpoints in the candidate endpoint set A are initially eliminated to obtain the initial corner set B;

[0074] The localization algorithm for corner and edge features is based on the fact that correct candidate endpoints have the feature of intersecting the boundary lines of the squares; in contrast, false endpoints are on the boundary lines of the squares and do not have the 'cross' feature.

[0075] A correct candidate endpoint should be located near the local area of ​​a chessboard corner, while a false endpoint is not located near the local area of ​​a chessboard corner. The most obvious characteristic of a chessboard corner is that it is formed by the intersection of two orthogonal lines. Therefore, this characteristic can be used to distinguish between correct candidate endpoints and false endpoints.

[0076] Therefore, based on the feature that the corner points of the square target image are formed by the intersection of two mutually perpendicular straight lines, a 13*13 boundary scanning rectangular window template K2 is defined to determine the 'cross intersection' feature. Template K2 is shown in Equation 2.

[0077]

[0078] In this template, p i The position represents the candidate endpoint. The position of 1 represents the pixel that needs to be traversed, and the pixel at the position of 0 does not need to be traversed.

[0079] Iterate through all candidate endpoints in the candidate endpoint set A, and scan the boundary scan rectangle window template K2 for each endpoint. iEach of these endpoints coincides with each candidate endpoint in the candidate endpoint set A, and features of the boundary of the region near the candidate endpoint are identified.

[0080] Once the positions overlap, it is necessary to traverse the pixels with a value of 1 that coincide with the bounding rectangle window template K2. The traversal order is the top boundary, bottom boundary, left boundary, and right boundary. When an edge point (i.e., an edge pixel with a pixel value of 1) is found during the traversal of a certain boundary, the traversal of the current boundary is stopped, the total boundary value of the candidate endpoints of the current traversal is incremented by 1, and the traversal of the next boundary begins. After traversing all four boundaries, the total boundary value of the candidate endpoints of the current traversal is recorded. If the total boundary value of the candidate endpoints of the current traversal is 4, it means that there are two intersecting lines near the candidate endpoints of the current traversal, and the candidate endpoints of the current traversal are considered to be in the local area near the corner of the chessboard, and the candidate endpoints of the current traversal are correct candidate endpoints. Conversely, if the total boundary value of the candidate endpoints of the current traversal is less than 4, the candidate endpoints of the current traversal are considered not to be in the local area near the corner of the chessboard, and the candidate endpoints of the current traversal are false endpoints.

[0081] like Figure 4 In Figure a, the candidate endpoint marked with a black circle has a total of 4 boundary values ​​after the above steps, which can be considered a correct candidate endpoint. Figure 4 The candidate endpoint marked by the black circle in Figure b has a total boundary value of 2 after the above steps, and is considered a false endpoint.

[0082] By scanning all candidate endpoints in the candidate endpoint set A and removing false endpoints, the set is updated to obtain the initial corner point set B.

[0083] like Figure 5 As shown, Figure 5 The image shows the result of a set of candidate endpoints A for a chessboard image after false endpoint detection and removal. The white circles mark the correct candidate endpoints, which are located near the local area of ​​the chessboard corner. The white × marks the false endpoints, which are not located near the local area of ​​the chessboard corner. The lower right corner of the white circles and white ×s shows the total number of boundary values ​​of the corresponding candidate endpoints.

[0084] After the initial corner point localization operation, the resulting initial corner point set B may contain a real corner point surrounded by two or more initial corner points. In other words, these initial corner points are not spurious corner points, but rather approximations of the real corner points. Therefore, the precise corner point localization algorithm needs to select the initial corner point from the initial corner point set B that is closest to the real corner point.

[0085] like Figure 6As shown, pixel points A and B marked by black circles are both initial corner points of the initial corner point set B. They both belong to the approximate points of the same theoretical chessboard corner point, and the only difference is their similarity degrees with the theoretical chessboard corner point. Therefore, the accurate corner point positioning operation needs to calculate the similarity degrees between these approximate points belonging to the same theoretical chessboard corner point and the theoretical chessboard corner point, and retain the approximate corner point with the maximum similarity degree.

[0086] S3. Propose the positioning of false corner points through accurate corner point positioning to complete the positioning of the corner points of the square target;

[0087] Define a 'crosshair response value' matrix with a size of 7*7. After performing dot multiplication and non-maximum suppression processing on the 'crosshair response value' matrix and one or more initial corner points in the same local area in the initial corner point set B, the accurate positioning of real corner points is achieved, including the following steps:

[0088] S3.1. Traverse each initial corner point in the initial corner point set B, and calculate the Euclidean distance between each initial corner point and other initial corner points according to Equation 3:

[0089] |(x i -x j) +(y i -y j )|<threshold Equation 3

[0090] In Equation 3, x i , x j , y i and y j respectively represent the abscissas and ordinates of two different initial corner points, i and j are the subscripts of the corresponding initial corner points, and threshold is the distance threshold.

[0091] It is necessary to set the Euclidean distance threshold between initial corner points according to the boundary length of each square in the image of the square target imaged by the camera. Since the device of the structured light three-dimensional reconstruction system is basically fixed and the object placement distance used is also approximately fixed, this threshold can be set to a value suitable for this structured light three-dimensional reconstruction system. In this embodiment, the threshold is set to 15, indicating that the boundary length of each square in the image of the square target imaged by the camera is about 15 pixels. Because the real corner points are at the junctions of four squares, with the real corner point as the center, a square with a side length of 15 can just cover 1 / 4 of each of the four squares connected to the real corner point. By analogy, all initial corner points can be covered to ensure no omission;

[0092] If there are two or more initial corner points whose Euclidean distance is less than a set threshold, then the two or more initial corner points whose Euclidean distance is less than the set threshold are considered to be in the local region of the same theoretical corner point, and are all approximations of the real corner point. These approximations in the local region of the same theoretical corner point are marked.

[0093] S3.2. Obtain a 7*7 ROI (region of interest) centered on the initial corner points of the same local region, taking the approximate points and their neighborhoods within the same region. i Image region, p i This represents the i-th initial corner point, and then these ROIs(p i The regions are respectively overlapped with the predefined 'crosshair aiming response value' matrix K3. The dot product operation according to Equation 5 is performed to obtain the response matrix R corresponding to the approximate point, as follows:

[0094]

[0095] R = K2 * ROI(p i Formula 5

[0096] S3.3. Summing the response matrix R obtained in step S3.2 according to Equation 6, we obtain the response value of each approximation point; the higher the response value, the higher the similarity between the corresponding approximation point and the true corner point, as detailed below:

[0097] reponse=∑ i ∑ j R(i,j) Equation 6

[0098] In Equation 6, i and j represent the elements in the i-th row and j-th column of the response matrix R;

[0099] Therefore, the approximate point with the largest response value in the current local area is retained as the final correct positioning corner point;

[0100] S3.4 Return to step S3.1 until all initial corner points in the initial corner point set B have been traversed, and the correct corner point set C of the square target image is obtained.

[0101] like Figure 7 As shown, Figure 7 This shows the result of the initial set of corner points B of a chessboard image after the corner point precise localization operation. Figure 7 Within the white rectangular frame, two initial corner points are located in the same local area. These two initial corner points are approximations of the same theoretical chessboard corner point. Figure 7As can be seen from the enlarged image in the upper left corner, one initial corner point has a response value of 4 and the other initial corner point has a response value of 5. Therefore, the initial corner point with a response value of 5 is considered the true corner point, and the initial corner point with a response value of 4 is discarded.

[0102] In summary, after the initial corner location operation and the precise corner location operation described above, the true set of chessboard corner points C of the chessboard image can be obtained.

[0103] Example 2:

[0104] In this embodiment, a typical chessboard calibration board in a square target is used as an example, with the condition that the angle between the plane of the chessboard calibration board and the camera imaging plane is 48°. The results after steps S2 and S3 are given using this chessboard calibration board as an example. When the angle between the plane of the chessboard calibration board and the camera imaging plane is 48°, the result of the chessboard calibration board after step S2 is as follows: Figure 9 As shown. Figure 9 In the example, consistent with Example 1, the × mark represents a false endpoint, the white circle mark represents a correct candidate endpoint, and the lower right corner of the white circle and the white × shows the total number of boundary values ​​for the corresponding candidate endpoint. The result after step S3 is as follows: Figure 10 As shown. Figure 10 The image shows the result of the initial corner point set B after the corner point precise location operation. The area highlighted by the white rectangle shows two initial corner points located in the same local region. These two initial corner points are approximations of the same theoretical chessboard corner point. Figure 10 As can be seen from the enlarged image in the upper right corner, one initial corner point has a response value of 1, and the other initial corner point has a response value of 4. Therefore, the initial corner point with a response value of 4 is considered to be the true corner point. After removing the initial corner point with a response value of 1, the true chessboard corner point set C can be obtained.

[0105] Example 3:

[0106] In this embodiment, a typical chessboard calibration board in a square target is used as an example, with the condition that the angle between the plane of the chessboard calibration board and the camera imaging plane is 20°, and the upper boundary of the chessboard is partially obscured. The results after steps S2 and S3 are given using this chessboard calibration board as an example. When the angle between the plane of the chessboard calibration board and the camera imaging plane is 20°, the result of the chessboard calibration board after step S2 is as follows: Figure 11 As shown. Figure 11 In the example, consistent with Example 1, the × mark represents a false endpoint, the white circle mark represents a correct candidate endpoint, and the lower right corner of the white circle and the white × shows the total number of boundary values ​​for the corresponding candidate endpoint. The result after step S3 is as follows: Figure 12 As shown. Figure 12The image shows the result of the initial corner point set B after the corner point precise location operation. The area highlighted by the white rectangle shows two initial corner points located in the same local region. These two initial corner points are approximations of the same theoretical chessboard corner point. Figure 12 As can be seen from the enlarged image in the upper right corner, one initial corner point has a response value of 1, and the other initial corner point has a response value of 4. Therefore, the initial corner point with a response value of 4 is considered to be the true corner point. After removing the initial corner point with a response value of 1, the true chessboard corner point set C can be obtained.

[0107] The preferred embodiments disclosed above are merely illustrative of the invention and its core concepts. Those skilled in the art will recognize that variations may occur in specific application scenarios and implementation based on the ideas of this invention; therefore, this specification should not be construed as limiting the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for detecting cross-shaped corner points applicable to the calibration of a structured light 3D reconstruction system, characterized in that, Includes the following steps: S1. Obtain images of the cube target by taking pictures; S2. Initially locate similar corner points in the square target image through corner point localization, including the following steps: S2.

1. Extract the edges of the square target image captured by the camera to obtain the edge results of the square target image; S2.

2. Based on the edge results of the square target image, candidate endpoints are detected to obtain a candidate endpoint set A. According to the geometric features of the corner points of the square target, the pixel value of edge pixels is set to 1, and the pixel value of non-edge pixels is set to 0. First, the positional relationship of each pixel in the eight-neighborhood of a pixel is defined. The center pixel in the eight-neighborhood of a pixel is defined as p1, the pixel below the center pixel is defined as p2 and used as the starting point, and the pixels around the center pixel in a clockwise direction are defined as p3 to p9 respectively. The 'approximate cross structure' feature is constructed as follows: when p1 in the eight-neighborhood of a pixel is an edge pixel, at least one of p3, p4 and p5 is an edge pixel, and p6 to p9 and p2 are non-edge pixels. When the pixel p1 and the eight-neighborhood of p1 satisfy the above features, p1 is considered a candidate endpoint. A 'quasi-cross structure' mathematical matrix is ​​defined, as shown in K1 in Equation 1: Formula 1 When the dot product of a pixel in the edge image of the target obtained in step S2.1 and its eight neighbors with the mathematical matrix of the 'quasi-cross structure' reaches the set condition, that is, when the value of the corresponding position of the matrix of the dot product also satisfies the 'quasi-cross structure' feature, the pixel is considered as a candidate endpoint, and then the candidate endpoint set A is obtained. S2.

3. Using a corner and edge feature localization algorithm, false endpoints in the candidate endpoint set A are initially eliminated to obtain an initial corner set B. The corner and edge feature localization algorithm is based on the fact that correct candidate endpoints have the feature of intersecting the boundary lines of the square. In contrast, false endpoints are located on the boundary lines of the square and do not have the 'cross intersection' feature. Therefore, based on the feature that the corners of the square target image are formed by the intersection of two mutually perpendicular straight lines, a 13*13 boundary scanning rectangular window template K2 is defined to determine the 'cross intersection' feature. The boundary scanning rectangular window template K2 is shown in Equation 2. Formula 2 Where, p i The position represents the candidate endpoint. A pixel at position 1 indicates a pixel that needs to be traversed, while a pixel at position 0 does not need to be traversed. Traverse all candidate endpoints in the candidate endpoint set A, aligning the center point of the boundary scanning rectangle template with each candidate endpoint in set A, and identify the features of the boundary region near the candidate endpoint. Specifically, traverse the four boundaries of the region where the candidate endpoint overlaps with the boundary scanning rectangle template in the order of up, down, left, and right. When an edge point (i.e., an edge pixel with a value of 1) is found during the traversal of a boundary, stop the traversal of the current boundary, and set the total edge value of the currently traversed candidate endpoints to [value missing]. Increment the boundary value by 1 and begin traversing the next boundary. After traversing all four boundaries, record the total boundary value of the candidate endpoints traversed. If the total boundary value of the candidate endpoints traversed is 4, it means that there are two intersecting lines near the candidate endpoint, and the candidate endpoint is the correct candidate endpoint. Otherwise, if the total boundary value of the candidate endpoints traversed is less than 4, the candidate endpoint is a false endpoint. Scan all candidate endpoints in the candidate endpoint set A, remove false endpoints, and update the set to obtain the initial corner point set B. S3. By accurately locating corner points and eliminating false corner points, the corner point positions of the target block are located, including the following steps: S3.

1. Traverse each initial corner point in the initial corner point set B and calculate the Euclidean distance between each initial corner point and other initial corner points. If there are two or more initial corner points whose Euclidean distance is less than the set threshold, then the two or more initial corner points whose Euclidean distance is less than the set threshold are considered to be in the local region of the same theoretical corner point, and are all approximate points of the real corner point. Mark these approximate points in the local region of the same theoretical corner point. S3.

2. The approximate points and their neighborhoods within the same region are multiplied by a predefined 'crosshair aiming response value' matrix to obtain the response matrix corresponding to the approximate point. The 'crosshair aiming response value' matrix is ​​a 7*7 matrix K3, as shown in Equation 4. Equation 4; S3.3 Summing the response matrix obtained in step S3.2 yields the response values ​​of each approximation point. The higher the response value, the higher the similarity between the corresponding approximation point and the true corner point. Therefore, the approximation point with the largest response value in the current local area is retained as the final correct positioning corner point. S3.4 Return to step S3.1 until all initial corner points in the initial corner point set B have been traversed, and the correct corner point set C of the square target image is obtained.

Images