Synchronous calibration method for three-dimensional reconstruction of microphotography object

Abstract

The invention relates to the technical field of cameras, and discloses a synchronous calibration method for three-dimensional reconstruction of a microphotography object, which comprises the following steps: image acquisition: acquiring images of the object and a calibration background at different shooting angles; image fusion: respectively fusing the images into a fused image with a clear picture; angular point detection: detecting angular point coordinates of each fused image, dividing the angular point coordinates into true value coordinates and reference value coordinates, and finding out three-dimensional world coordinate points corresponding to the reference value coordinates; angular point matching output: matching reference value coordinates for true value coordinates according to a nearest neighbor principle, and finding a three-dimensional world coordinate point corresponding to each true value coordinate through a corresponding relation between the reference value coordinates and angular point world coordinates; and camera calibration: internal and external parameters of the camera are obtained through calibration, and calibration and acquisition of an object to be reconstructed are synchronously completed in one image. According to the method, the limitation of depth of field of macro photography is overcome, the calibration precision is higher, the robustness is better, the practicability is better, and the improvement of the quality of three-dimensional reconstruction is facilitated.

Description

Technical Field

[0001] The present invention relates to the technical field of cameras, and in particular to a synchronous calibration method for three-dimensional reconstruction of macro photography objects. Background Art

[0002] 3D reconstruction technology has made significant progress in recent years and has demonstrated enormous potential for application. However, improving reconstruction quality remains a key research priority. Camera calibration is an essential prerequisite for 3D reconstruction, and accurate calibration results can help improve 3D reconstruction quality.

[0003] Currently, mainstream camera calibration uses a flat target. However, because the depth of field in macro photography is typically smaller than the object size, the image is easily defocused, resulting in blurred target images. Furthermore, calibration requires capturing images of the target at different angles. For example, when shooting at high angles, existing algorithms often fail to identify corners, making calibration impossible. Furthermore, existing calibration techniques are separate from 3D reconstruction: calibration is performed first, and then the object image is captured separately for reconstruction. This fails to accurately reflect the actual camera parameters used to capture the object. In the prior art, Chinese patent application CN119478055A discloses a camera calibration method and a method for recovering 3D object shape information. This invention is highly robust to camera defocus and extends the camera's original depth of field, but it cannot calibrate images at high angles. Chinese patent application CN118196208A discloses a method for large-field-of-view camera calibration and parameter optimization. This invention stabilizes the calibration results through data optimization, but is only effective in scenes where calibration is possible and does not address the problems encountered in actual calibration.

[0004] In general, the camera calibration technology for macro photography 3D reconstruction still has the following problems:

[0005] (1) Macro photography is prone to defocusing, the target image is partially blurred, and corner identification is difficult, making calibration impossible.

[0006] (2) Existing calibration methods usually require that the entire calibration plate be exposed when acquiring images. In extreme cases such as large-angle shooting and partial target occlusion, the calibration method is prone to failure.

[0007] (3) Even if it can be used for calibration, the number of effectively used corner points is reduced, the regional image information is lost, and only the parameter performance of the camera's local field of view is reflected. The calibration result is not global.

[0008] (4) Currently, camera calibration in 3D reconstruction applications involves first independently photographing a target for camera calibration, and then inputting the calibration results and the image of the independent object into the reconstruction model. During macro photography, the focus states of the target and the object are not completely consistent, and repeated camera shots can also introduce mechanical errors. The calibration results based on the target cannot fully reflect the camera parameters when photographing the object, which has a negative impact on the 3D reconstruction effect. Summary of the Invention

[0009] The purpose of the present invention is to solve the problem that existing camera calibration technology cannot meet the calibration requirements of macro photography images with local blur, incomplete calibration plates or deformed images, thereby improving the quality of three-dimensional reconstruction of objects and providing a synchronous calibration method for three-dimensional reconstruction of macro photography objects.

[0010] To achieve the above object, the technical solution of the present invention is as follows:

[0011] A synchronous calibration method for three-dimensional reconstruction of macro photography objects comprises the following steps:

[0012] S1: Image acquisition: Measure the length, width, and height of the object using measuring tools including but not limited to calipers and laser rangefinders. Take the average of the length, width, and height as the average size. Select or create a black and white checkerboard with a ratio of 1 / 40 to 1 / 20 of the average size of the object, and use a macro camera to capture images of the object and the calibration background at different shooting angles. Keep the lighting constant during shooting. Adjust the focal length of the macro camera for focus each time to obtain a sequence of calibration background images and a sequence of object images at a certain shooting angle. No image processing is performed between image sequences.

[0013] S2: Image fusion, calibrating the background image sequence and the object image sequence, fusing the multiple images in each sequence into a clear and information-rich fused image, thereby obtaining calibrated background images and object images under different shooting angles;

[0014] S3: Corner detection: Detect the corner coordinates of each fused image, that is, calibrate the background image and the object image, and find the 3D world coordinate points that correspond one-to-one with the corner coordinates in the calibrated background image. Specifically, it includes image preprocessing and image corner detection.

[0015] S4: Corner point matching output: The corner point coordinates in the object image are used as the true value coordinates, and the corner point coordinates in the calibration background image are used as the reference value coordinates. The true value coordinates are matched one-to-one with the corresponding reference value coordinates according to the nearest neighbor principle. The 3D world coordinate point corresponding to each true value coordinate is found through the correspondence between the reference value coordinates and the world coordinates of the corner points, and matching corner point pairs are generated between the true value coordinates and the corresponding 3D world coordinate points.

[0016] S5: Camera calibration, using matching corner point pairs to calibrate the camera, obtain the camera's intrinsic parameters and the pose information when the image was taken, and achieve the simultaneous completion of camera calibration and acquisition of the object to be reconstructed in one image.

[0017] Preferably, step S1 image acquisition:

[0018] S11: Measure the length, width, and height of the object using measuring tools including but not limited to calipers and laser rangefinders. Take the average of the length, width, and height as the average size. Select or create a black and white checkerboard with the size of each small square being 1 / 40 to 1 / 20 of the average size of the object. The number of rows and columns of the small squares in the calibration background is set to a combination of odd and even numbers (e.g., even rows and odd columns). The total area covered by the small squares needs to be larger than the area occupied by the object. Set the number of rows (Row) of the corner points on the checkerboard to be the number of rows of small squares minus 1, and the number of columns (Col) to be the number of columns of small squares minus 1.

[0019] S12: Fix the prepared chessboard on a flat background plate as a calibration background;

[0020] S13: Use a common commercially available macro camera. Fix the macro camera on a tripod and adjust the camera height and angle so that both the object and the calibration background appear in the camera's field of view.

[0021] S14: Set camera parameters, including but not limited to aperture, shutter speed, ISO, etc., to ensure image quality;

[0022] S15: Use a constant light source to observe the reflections of the object and the calibration background in the macro camera at different shooting angles to ensure that the light intensity remains constant during the shooting process;

[0023] S16: Collect images of the calibration background at different shooting angles. Manually adjust the focal length of the macro camera to focus at a fixed shooting angle to capture an image sequence. Then, change the shooting angles to capture at least 6 sets of image sequences to form calibration background image sequences blank_1, blank_2, ... blank_n (n≥6).

[0024] S17: The object is then placed in the center of the calibration background and remains stationary. The calibration background and the object need to be observed in the camera field of view at the same time. The object image is collected at the same angle. The focal length of the macro camera is manually adjusted at a fixed shooting angle so that the object and the calibration background are clearly focused. An image sequence is captured. At different shooting angles, no less than 6 sets of image sequences are collected to form the object image sequence obj_1, obj_2, ...obj_n (n ≥ 6). Ensure that the calibration background image sequence and the object image sequence at the same shooting angle correspond to each other.

[0025] Preferably, step S2 image fusion:

[0026] Import the image sequences blank_1, blank_2, ... blank_n (n ≥ 6) and obj_1, obj_2, ... obj_n (n ≥ 6) in sequence, fuse the multiple images in each sequence into a clear and information-rich fused image, and thus obtain the calibrated background image and object image under different shooting angles;

[0027] The image fusion process can effectively solve the problem of calibration failure or low calibration accuracy caused by the inability to identify corner points or only being able to identify local corner points due to image defocus blur;

[0028] The innovation of the image fusion technology in this invention lies in its organic integration with the macro camera calibration process. The image fusion technology is a well-known image fusion technology in the art. Its principle is to generate a single high-quality fused image by integrating complementary information from multiple images. Image fusion technology can be implemented based on various methods, including but not limited to:

[0029] Pixel-level weighted averaging method;

[0030] Feature-level multi-resolution fusion, such as pyramid fusion technology;

[0031] Fusion methods based on transform domain, such as wavelet transform fusion;

[0032] In the present invention, the specific implementation of image fusion is not limited to the above method, and any other technical means that can achieve image clarity processing can also be used. The image fusion method will be optimized according to the actual application scenario and calibration requirements.

[0033] Preferably, step S3 corner point detection:

[0034] S31: Image pre-processing, which can improve the accuracy and robustness of corner detection, includes:

[0035] Grayscale conversion of images: converting three-channel color images into single-channel grayscale images, reducing computational complexity, eliminating color information interference, and speeding up processing.

[0036] Grayscale adjustment: reset pixels with grayscale values ​​greater than 90 to grayscale values ​​of 230 to remove shadows in the image;

[0037] Gaussian blur, with a Gaussian kernel size of 5×5 and a standard deviation of 0, suppresses noise, smoothes image details, and reduces false detections;

[0038] Dilation: dilate the kernel 3×3 to disconnect the small squares in the calibrated background of the image and highlight the corners;

[0039] S32: Image corner detection: Match the calibrated background image and object image with the same shooting angle after image pre-processing in S31. During corner detection, the coordinates of the corner points in the calibrated background image are first detected, and then the coordinates of the corner points in the object image are detected. Furthermore, image corner detection includes five steps: edge detection, subject removal (only for object images), obtaining candidate corner points, corner point verification, and corner point reconstruction. The corner point detection processing of the calibrated background image and the object graphics is slightly different, but the final output corner point coordinates are in the same form.

[0040] S321: Edge detection, using an existing mature contour extraction algorithm to extract contour information in the image. The present invention does not limit the specific contour extraction algorithm;

[0041] S322: Body Removal (Object Image Only): Remove the object body from the object image, calculate the area and maximum linear size of the extracted contours, and filter according to the following thresholds:

[0042] The minimum screening threshold for contour area is 100;

[0043] The minimum screening threshold for contour area is 1000;

[0044] The minimum screening threshold for the maximum linear dimension of the contour is 16;

[0045] The maximum screening threshold for the maximum linear dimension of the contour is 35;

[0046] Contours that do not meet both of the above thresholds will be removed. These removed contours are usually the main structure of the object and some reflective areas formed by light on the surface of the object.

[0047] S323: Obtain candidate corner points and detect corner point information in the image using an existing mature corner point detection algorithm. The present invention does not limit the specific corner point detection algorithm;

[0048] S324: Corner point review. Some falsely detected corner points may come from areas of non-interest, such as unremoved object edges, surface reflections, image noise, etc., and require further screening. The screening is based on whether the corner points are at the intersection of adjacent small squares, and a certain offset of corner points is allowed.

[0049] Specifically, pixel blocks with grayscale values ​​greater than 90 are considered white pixels, and vice versa as black pixels. The distribution ratio of black and white pixels in a 15-pixel circular neighborhood around each candidate corner coordinate is calculated, and the corner coordinates are arranged in order from small to large according to the ratio:

[0050] For the calibration background image: remove the 5% corner points at the beginning and end of the arrangement, and retain the coordinates of the 90% corner points in the middle of the arrangement as the corner point detection result of the calibration background image;

[0051] For object images: remove the first and last 20% of corner points, and retain the coordinates of the 60% corner points in the middle as the corner point detection results of the object image;

[0052] S325: Corner reconstruction. Camera calibration requires the coordinates of the 2D corner points on the image and their corresponding 3D world coordinates. Since the detected corner points do not have a certain arrangement relationship, especially after removing the subject from the object image, the spatial relationship between the corner points is further destroyed. Therefore, the corner points need to be rearranged and the spatial relationship between these corner points is used to find the corresponding 3D world coordinates.

[0053] Furthermore, in step S325, corner point reconstruction:

[0054] S3251 endpoint identification:

[0055] Preselect endpoint coordinates: traverse all corner point coordinates and find the four endpoint coordinates with the maximum and minimum values ​​of x and y coordinates, respectively, and record them as left(x_min, y_min) 1) 、right(x_max,y2)、top(x3,y_min)、bottom(x4,y_max)、calculate the Euclidean distance dis between the left-top、right-top、left-bottom、right-bottom endpoint coordinates left-top 、dis right-top 、dis left-bottom 、dis right-bottom ;

[0056] In order to adapt to the deformation of the square in different viewing angles, the pixel length threshold box of each small square in the background is marked min 15, box max =35, the number of rows and columns of the inner corner points on the calibration background are complete, and the small square sizes distributed in the four groups of Euclidean distances are checked according to formulas (1) to (4) to see whether they are consistent with the preset small square sizes:

[0057]

[0058] Each Euclidean distance dis left-top 、dis right-top 、dis left-bottom 、dis right-bottom Just satisfy one of the corresponding formulas respectively;

[0059] Preselect coordinate orientation judgment: Then find the 6 adjacent coordinates corners_i (1≤i≤6) with the closest Euclidean distance to the four endpoint coordinates, and calculate the offset x of each endpoint coordinate and corners_i (1≤i≤6) in the x and y directions dists-i with y dists-i (1≤i≤6), according to formula (5):

[0060]

[0061] If formula (5) is satisfied, it is considered that the corners_i (1≤i≤6) are distributed on the same side of the endpoint, and the endpoint is an edge corner point, which is retained. Otherwise, corners_i (1≤i≤6) are distributed on both sides of the endpoint, and the endpoint is an internal corner point. Then find the coordinates with the same regularity as the coordinate values ​​in the endpoint. For example, if the endpoint is bottom, find the second largest y coordinate value, that is, the second corner coordinate containing y_max. Repeat the above process until all four endpoints are confirmed and retained.

[0062] Endpoint coordinate color determination: Retain each of the four endpoints and the first three adjacent coordinates in the corresponding corners_i, i.e., corners_i (1≤i≤3). Check the pixel values ​​within the area enclosed by each endpoint and corners_i (1≤i≤3). If the pixel value exceeds 95, the endpoint is considered a white endpoint; otherwise, it is considered a black endpoint. Theoretically, there are two white endpoints and two black endpoints. Select the two endpoints with the smaller x-coordinate values ​​of the same color and record them as "white_start" and "black_start", respectively. The remaining two are recorded as "white_end" and "black_end".

[0063] S3252 corner point sorting:

[0064] Define vector: the vector from white_start to white_end is vector sign , with the vector from white_start to black_start as vector move , rearrange the corner points verified in step S324 in a "Z" shape;

[0065] Slope check: check the slope between white_start and white_end, white_start and black_start, black_start and black_end, black_end and white_end to avoid slope non-existence;

[0066] Initial starting point: white_start is the starting point of the corner point sorting, along the vector move Direction to find the nearest corner points to corner_start. Calculate the Euclidean distance between the two adjacent corner points found as the reference distance dis refer , calculate the distance dis between the two adjacent corner points found recently according to formula (6) new Distance from reference refer The integer quotient is used as the counter count:

[0067]

[0068] If count is 1, it is considered that the newly found corner point is consistent with the corner point on the calibration background and is adjacent;

[0069] If count is greater than 1, it is considered that there is a "discontinuity" between the two adjacent corner points found recently, and count-1 corner points are missed. The missed corner points are solved according to the interpolation method in formula (7);

[0070]

[0071] Where (x1, y1) is the starting coordinate of the discontinuity, (x2, y2) is the ending coordinate of the discontinuity, (x i ,y i ) is the interpolation coordinate, 1≤i≤count-1;

[0072] The detected and interpolated missed corner points are saved in the corner point set corners_col as the "column of corner points close to corner_start" on the corresponding calibration background, which serves as the starting point for reconstructing each "row" of corner points.

[0073] Row sorting: traverse the corner points in corners_col, taking each corner point as the starting point corner col-start , along the vector sign Direction to find distance corner col-start Several recently added corner points;

[0074] According to formulas (8) to (9), check the newly added corner points and corner col-start Is the constructed vector consistent with vector sign parallel:

[0075] vector = corner-corner col-start (8)

[0076]

[0077] Where: corner is the coordinate of the newly added corner point, vector is the distance between the newly added corner point and the starting point corner col-start The vector formed;

[0078] If the cosine value is not less than 0.99, it is considered that the vector and the vector sign The two vectors are essentially parallel;

[0079] Calculate the Euclidean distance between adjacent corner points found as the reference distance dis refer , calculate the distance dis between the two adjacent corner points found recently according to formula (6) new Distance from reference refer The integer quotient is used as the counter count:

[0080] If count is 1, it is considered that the newly found corner point is consistent with the corner point on the calibration background and is adjacent;

[0081] If count is greater than 1, it is considered that there is a "discontinuity" between the two adjacent corner points found recently, and count-1 corner points are missed. The missed corner points are solved according to the interpolation method in formula (7);

[0082] When the latest corner point and corner appear col-start When the constructed vector does not satisfy formula (9), it is considered that all corner points in this direction have been found;

[0083] Continue to traverse the next starting corner point and repeat the above process until you find each row of corner points starting from the corner point in corners_col.

[0084] Each row of corner points is stored in a point set corners_row_N (1≤N≤Row).

[0085] S3253 corner points correspond to:

[0086] For each point set corners_row_N (1≤N≤Row), calculate the number of corner points in the point set one by one, divided into the following cases:

[0087] No missed corner detection: If the number of corner points in each point set is exactly equal to the Col value of the corner points in the chessboard, and N is equal to the number of rows of the corner points in the chessboard, it means that all corner points on the image have been identified and rearranged. At this time, a Row×Col matrix is ​​directly constructed according to the number of corner points in the chessboard. The distribution of the matrix corresponds to the x and y coordinates in the three-dimensional world coordinates, and then multiplied by the actual length len of the small squares in the chessboard. The z coordinates are all set to 0. According to formula (10), the three-dimensional world coordinate points corresponding to all corner points are generated:

[0088]

[0089] Where i, j represents the position of the pixel, 1≤i≤Col, 1≤j≤Row, corner ij Represents the three-dimensional coordinates of the corner point in the i-th row and j-th column of the corresponding image;

[0090] Corner points are missed: the distances between adjacent corner points in the point set corners_row_N need to be calculated separately. The count value is calculated according to formula (6). The count value reflects the distribution of the missed corner points. According to formula (11), the cumulative counter sum accumulates the count value obtained each time. The x-coordinate value in the 3D world coordinate point corresponding to each corner point corresponds to the sum value accumulated at this corner point. The y-coordinate value is unified to the N value corresponding to the current point set corners_row_N, and then multiplied by the actual size len of the small squares in the chessboard. The z-coordinates are all set to 0 to obtain the 3D world coordinate point corresponding to each corner point.

[0091]

[0092] Where i, j represents the position of the pixel, corner ij represents the 3D world coordinate point of the corner point in row i and column j of the corresponding image;

[0093] For the aforementioned interpolated corner point, it is necessary to delete the interpolated corner point and the corresponding 3D world coordinate point. The remaining corner point coordinates and the corresponding 3D world coordinate point are the corner point and the corresponding 3D world coordinate value on the image after the corner point reconstruction.

[0094] Preferably, in step S4, in the corner point matching output:

[0095] In step S1, image acquisition requires consistent shooting angles. However, mechanical errors in adjusting the camera's shooting angle, as well as differences in the focus position of a macro camera when photographing objects of different sizes, such as when the background calibration plate and the object are at different heights, can affect calibration accuracy.

[0096] Since the background pattern remains unchanged and the shooting angle is consistent, it can be ensured that the distribution of corner points in the calibration background image and the object image is basically consistent. Therefore, a one-to-one matching of the corner points in the two images taken at the same shooting angle can be performed.

[0097] Specifically, the matching process is as follows:

[0098] Take the corner point coordinates in the object image as the true value coordinates, and the corner point coordinates in the calibration background image as the reference value coordinates, and traverse the true value coordinates and reference coordinates;

[0099] Using the nearest neighbor principle, find the reference coordinate value closest to each true coordinate, with the threshold being that the Euclidean distance between two coordinates does not exceed 3 pixels. If a reference coordinate value can be found, use this reference coordinate to find the corresponding 3D world coordinate point to which the corner point in step S3253 corresponds, generating a matching corner point pair between the true coordinate and the 3D world coordinate point. Otherwise, discard this true coordinate and continue matching the corresponding 3D world coordinate point for the next true coordinate.

[0100] Preferably, in step S5 synchronous calibration:

[0101] The matching corner point pairs generated in step S4 are input and the Zhang calibration method is used. This method calculates the camera's intrinsic and extrinsic parameters based on matching point pairs from multiple images. However, the present invention is not limited to this method; any other calibration method that accepts similar matching data pairs as input and can output camera intrinsic parameters and pose information is also applicable.

[0102] Compared with the prior art, the present invention has the following beneficial effects:

[0103] (1) Introducing image fusion technology to improve calibration accuracy and reconstruction quality: This invention uses image fusion as a pre-step for camera calibration. By fusing the clearly focused areas in the image sequence, a fully focused image is generated. This process effectively expands the depth of field of the macro camera and solves the problems of image defocus, corner recognition difficulties, and inability to calibrate caused by depth of field limitations in macro photography. At the same time, the fusion of the image of the object to be reconstructed also increases the number of image pixels that can be used for reconstruction, thereby improving the quality of 3D reconstruction.

[0104] (2) Enhance algorithm robustness and break through calibration limitations: Existing calibration technologies usually require that the complete calibration plate be exposed when capturing images, that the field of view be fully covered, and that extreme shooting angles be avoided. However, these requirements are difficult to meet in certain application scenarios. The corner detection method proposed in the present invention fully exploits the image information of the calibration plate, and uses the calibration plate at the same angle as a medium to achieve camera calibration under conditions such as incomplete checkerboard (such as occlusion or damage) or image deformation caused by large shooting angles. Not only is it more practical, but it also makes the calibration results more reflective of the object in the real shooting environment, rather than a substitute for an approximate scene.

[0105] (3) Enhance the globality of corner information and improve calibration accuracy: The fused image is clearer and can reflect the corner information more comprehensively, meeting the camera calibration requirements for corner information in all directions of the image, and improving calibration accuracy by increasing the utilization of corner information.

[0106] (4) Synchronous capture and calibration to eliminate mechanical errors: In existing technologies, camera calibration is usually performed using a calibration plate first, and the data is then applied to subsequent work. This invention achieves the synchronization of object capture and camera calibration, avoiding mechanical errors introduced by asynchrony between the calibration image and the object image, thereby improving calibration accuracy and facilitating subsequent 3D reconstruction.

[0107] (5) Comparative experiments with existing methods show that this method has lower average reprojection error, higher calibration accuracy, better robustness, and better practicality, which helps to improve the quality of 3D reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0108] Figure 1 It is a flow chart of the overall technical solution of an embodiment of the present invention.

[0109] Figure 2 It is a schematic diagram of the overall technical solution of an embodiment of the present invention.

[0110] Figure 3 FIG. 4 is a flowchart of corner detection according to an embodiment of the present invention.

[0111] Figure 4 FIG. 4 is a schematic diagram of corner point identification and naming according to an embodiment of the present invention.

[0112] Figure 5 Schematic diagram of edge corner points and internal corner points according to an embodiment of the present invention.

[0113] Figure 6 Schematic diagram of a Z-shaped arrangement of corner points according to an embodiment of the present invention.

[0114] Figure 7 This is a comparison diagram of corner point distribution on a calibrated background image and an object image according to an embodiment of the present invention.

[0115] Figure 8 is the corner point detection result of the MATLAB camera calibration toolbox in one embodiment of the present invention.

[0116] Figure 9 It is the corner point detection result of the present invention in one embodiment of the present invention. DETAILED DESCRIPTION

[0117] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0118] It should be noted that the drawings are all in a very simplified form and use non-precise proportions, and are only used to conveniently and clearly assist in explaining the embodiments of the present invention. The structures shown in the drawings are part of the actual structure.

[0119] The implementation of macro photography mentioned in the present invention is not limited to a specific shooting distance, lighting conditions or camera settings, and can be adjusted according to specific circumstances.

[0120] The objects mentioned in the present invention have simple or complex geometric structures and surface colors, including but not limited to ring-type jewelry products.

[0121] The various algorithms mentioned in the present invention are not limited to a certain algorithm.

[0122] In this example, the object being photographed is a ring. The ring is small in size and rich in details. Due to the depth of field limitation of macro photography, the image is prone to problems such as partial defocus and reduced clarity. Figure 1 and Figure 2 The present invention provides a synchronous calibration method for three-dimensional reconstruction of macro photography objects, comprising the following steps:

[0123] S1: Image acquisition: Measure the length, width, and height of the ring using measuring tools including but not limited to calipers and laser rangefinders. Take the average of the length, width, and height as the average size. Select or create a black and white checkerboard with a ratio of 1 / 40 to 1 / 20 of the average size of the ring, and use a macro camera to capture images of the ring and the calibration background at different shooting angles. Maintain constant lighting during shooting. Adjust the focal length of the macro camera for focus each time to obtain a sequence of calibration background images and a sequence of ring images at a certain shooting angle. No image processing is performed between image sequences.

[0124] S2: Image fusion, calibrating the background image sequence and the ring image sequence, fusing the multiple images in each sequence into a clear and information-rich fused image, thereby obtaining calibrated background images and ring images under different shooting angles;

[0125] S3: Corner detection: Detect the corner coordinates of each fused image, i.e., the calibrated background image and the ring image, and find the 3D world coordinate points that correspond one-to-one with the corner coordinates in the calibrated background image. Specifically, this includes image preprocessing and image corner detection.

[0126] S4: Corner point matching output: The corner point coordinates in the ring image are used as the true value coordinates, and the corner point coordinates in the calibration background image are used as the reference value coordinates. The true value coordinates are matched with the corresponding reference value coordinates according to the nearest neighbor principle. The world coordinates of each corner point corresponding to the true value coordinates are found through the correspondence between the reference value coordinates and the world coordinates of the corner points, and matching corner point pairs with the true value coordinates and the corresponding world coordinates of the corner points are generated.

[0127] S5: Camera calibration: Use matching corner point pairs to calibrate the camera to obtain the camera's intrinsic parameters and the pose information when the image was taken. This allows the camera calibration and the acquisition of the ring to be reconstructed to be completed simultaneously in one image.

[0128] Preferably, step S1 image acquisition:

[0129] S11: Using measuring tools including but not limited to calipers and laser rangefinders, measure the ring to be 25mm long, 10mm wide, and 25mm high. Take the average of the length, width, and height of 20mm as the average size. Set the size of each small square, len, to 1 / 40 to 1 / 20 of the average size of the ring, and set the value of len to 0.5mm. Select or make a black and white checkerboard. The number of rows and columns of the small squares in the calibration background is set to a combination of odd and even numbers (such as 44 rows and 43 columns). The total area covered by the small squares needs to be larger than the area occupied by the ring. Set the number of rows of the corner points in the checkerboard, Row, to the number of rows of the small squares - 1, i.e., 43, and the number of columns, Col, to the number of columns of the small squares - 1, i.e., 42.

[0130] S12: Fix the prepared chessboard on a flat background plate as a calibration background;

[0131] S13: Use a common commercially available macro camera. Fix the macro camera on a tripod and adjust the camera height and angle so that both the ring and the calibration background appear in the camera's field of view.

[0132] S14: Set camera parameters, including but not limited to aperture, shutter speed, ISO, etc., to ensure image quality;

[0133] S15: Using a constant light source, in this embodiment, an LED light source is used to observe the reflection of the ring and the calibration background in the macro camera at different shooting angles to ensure that the light intensity remains constant during the shooting process;

[0134] S16: Capture images of the calibration background at different shooting angles. Manually adjust the focal length of the macro camera at a fixed shooting angle to focus, capture an image sequence, and capture at least 6 sets of image sequences at different shooting angles to form calibration background image sequences blank_1, blank_2, ..., blank_n (n ≥ 6). In this embodiment, a total of 36 calibration background images are captured, forming calibration background image sequences blank_1, blank_2, ..., blank_36.

[0135] S17: The ring is then placed in the center of the calibration background, keeping it stationary. The calibration background and the ring must be simultaneously visible in the camera's field of view. Images of the ring are captured from the same angle. The macro camera's focal length is manually adjusted at a fixed shooting angle to bring the ring and calibration background into sharp focus. An image sequence is captured. At least six image sequences are captured from different shooting angles to form the ring image sequences obj_1, obj_2, ..., obj_n (n ≥ 6). Ensure that the calibration background image sequences and ring image sequences from the same shooting angle correspond to one another. In this embodiment, a total of 36 ring images are captured, forming the ring image sequences obj_1, obj_2, ..., obj_36.

[0136] Preferably, step S2 image fusion:

[0137] Import the image sequences blank_1, blank_2, ...blank_36 and obj_1, obj_2, ...obj_36 in sequence, and fuse the multiple images in each sequence into a clear and information-rich fused image. This will yield 36 calibration background images and 36 ring images from different shooting angles.

[0138] The image fusion process can effectively solve the problem of calibration failure or low calibration accuracy caused by the inability to identify corner points or only being able to identify local corner points due to image defocus blur;

[0139] The innovation of the image fusion technology in this invention lies in its organic integration with the macro camera calibration process. The image fusion technology is a well-known image fusion technology in the art. Its principle is to generate a single high-quality fused image by integrating complementary information from multiple images. Image fusion technology can be implemented based on various methods, including but not limited to:

[0140] Pixel-level weighted averaging method;

[0141] Feature-level multi-resolution fusion, such as pyramid fusion technology;

[0142] Fusion methods based on transform domain, such as wavelet transform fusion;

[0143] In the present invention, the specific implementation of image fusion is not limited to the above method, and any other technical means that can achieve image clarity processing can also be used. The image fusion method will be optimized according to the actual application scenario and calibration requirements.

[0144] Preferably, step S3 corner point detection:

[0145] S31: Image preprocessing can improve the accuracy and robustness of corner detection, see Figure 3 ,include:

[0146] Grayscale conversion of images: converting three-channel color images into single-channel grayscale images, reducing computational complexity, eliminating color information interference, and speeding up processing.

[0147] Grayscale adjustment: reset pixels with grayscale values ​​greater than 90 to grayscale values ​​of 230 to remove shadows in the image;

[0148] Gaussian blurring, with a Gaussian kernel size of 5×5 and a standard deviation of 0, suppresses noise, smoothes image details, and reduces false detections. In this embodiment, only one Gaussian blur is performed;

[0149] Dilation: dilate the kernel 3×3 to disconnect the small squares in the calibration background of the image and highlight the corners. In this embodiment, only one dilation is performed;

[0150] S32: Image corner detection: Match the calibrated background image and the ring image from the same shooting angle after image pre-processing in S31. During corner detection, the coordinates of the corners in the calibrated background image are first detected, and then the coordinates of the corners in the ring image are detected. Furthermore, image corner detection includes five steps: edge detection, subject removal (only object images), obtaining candidate corners, corner verification, and corner reconstruction. The corner detection process for the calibrated background image and the ring image is slightly different, but the final output corner coordinates are in the same format.

[0151] S321: Edge detection, using an existing mature contour extraction algorithm to extract contour information in the image. The present invention does not limit the specific contour extraction algorithm;

[0152] S322: Body Removal (Object Image Only): Remove the ring body from the ring image, calculate the area and maximum linear size of the extracted contour, and filter based on the following thresholds:

[0153] The minimum screening threshold for contour area is 100;

[0154] The minimum screening threshold for contour area is 1000;

[0155] The minimum screening threshold for the maximum linear dimension of the contour is 16;

[0156] The maximum screening threshold for the maximum linear dimension of the contour is 35;

[0157] Contours that do not meet both of the above thresholds will be removed. These removed contours are usually the main structure of the ring and some reflective areas formed by light shining on the ring.

[0158] S323: Obtain candidate corner points and detect corner point information in the image using an existing mature corner point detection algorithm. The present invention does not limit the specific corner point detection algorithm. In this embodiment, Harris corner point detection is used.

[0159] S324: Corner point review. Some falsely detected corner points may come from areas of non-interest, such as the ring's unremoved edge, surface reflections, image noise, etc., and require further screening. The screening is based on whether the corner points are at the intersection of adjacent small squares, and a certain offset of corner points is allowed.

[0160] Specifically, pixel blocks with grayscale values ​​greater than 90 are considered white pixels, and vice versa as black pixels. The distribution ratio of black and white pixels in a 15-pixel circular neighborhood around each candidate corner coordinate is calculated, and the corner coordinates are arranged in order from small to large according to the ratio:

[0161] For the calibration background image: remove the 5% corner points at the beginning and end of the arrangement, and retain the coordinates of the 90% corner points in the middle of the arrangement as the corner point detection result of the calibration background image;

[0162] For the ring image: remove the first and last 20% of the corner points, and retain the coordinates of the 60% of the corner points in the middle as the corner point detection result of the ring image;

[0163] S325: Corner reconstruction. Camera calibration requires the coordinates of the 2D corner points on the image and their corresponding 3D world coordinates. Since the detected corner points do not have a certain arrangement relationship, especially after removing the main body from the ring image, the spatial relationship between the corner points is further destroyed. Therefore, the corner points need to be rearranged and the spatial relationship between these corner points is used to find the corresponding 3D world coordinates.

[0164] Furthermore, in step S325, corner point reconstruction:

[0165] S3251 endpoint identification:

[0166] Preselect endpoint coordinates: traverse all corner point coordinates and find the four endpoint coordinates with the maximum and minimum x and y coordinates, see Figure 4, recorded as left(x_min, y1), right(x_max, y2), top(x3, y_min), bottom(x4, y_max), calculate the Euclidean distance dis between the left-top, right-top, left-bottom, and right-bottom endpoint coordinates left-top 、dis right-top 、dis left-bottom 、dis right-bottom ;

[0167] In order to adapt to the deformation of the square in different viewing angles, the pixel length threshold box of each small square in the background is marked min is 15, box max =35, the number of rows and columns of the inner corner points on the calibration background are complete, and the small square sizes distributed in the four groups of Euclidean distances are checked according to formulas (1) to (4) to see whether they are consistent with the preset small square sizes:

[0168]

[0169] Each Euclidean distance dis left-top 、dis right-top 、dis left-bottom 、dis right-bottom Just satisfy one of the corresponding formulas respectively;

[0170] Preselect coordinate orientation judgment: Then find the 6 adjacent coordinates corners_i (1≤i≤6) with the closest Euclidean distance to the four endpoint coordinates, and calculate the offset x of each endpoint coordinate and corners_i (1≤i≤6) in the x and y directions dists-i with y dists-i (1≤i≤6), according to formula (5):

[0171]

[0172] If formula (5) is satisfied, it is considered that the corners_i (1≤i≤6) are distributed on the same side of the endpoint. Figure 5 , the endpoint is an edge corner point and is retained. Otherwise, corners_i (1≤i≤6) are distributed on both sides of the endpoint, and the endpoint is an internal corner point. Then find the coordinates with the same pattern as the coordinate values ​​in the endpoint. For example, if the endpoint is bottom, find the second largest y coordinate value, that is, the second corner coordinate containing y_max. Repeat the above process until all four endpoints are confirmed and retained;

[0173] Endpoint coordinate color determination: Retain each of the four endpoints and the first three adjacent coordinates in the corresponding corners_i, i.e., corners_i (1≤i≤3). Check the pixel values ​​within the area enclosed by each endpoint and corners_i (1≤i≤3). If the pixel value exceeds 95, the endpoint is considered a white endpoint; otherwise, it is considered a black endpoint. Theoretically, there are two white endpoints and two black endpoints. Select the two endpoints with the smaller x-coordinate values ​​of the same color and record them as "white_start" and "black_start", respectively. The remaining two are recorded as "white_end" and "black_end".

[0174] S3252 corner point sorting:

[0175] Define vector: the vector from white_start to white_end is vector sign , with the vector from white_start to black_start as vector move , see Figure 6 , rearrange the corner points verified in step S324 in a "Z" shape;

[0176] Slope check: check the slope between white_start and white_end, white_start and black_start, black_start and black_end, black_end and white_end to avoid slope non-existence;

[0177] Initial starting point: white_start is the starting point of the corner point sorting, along the vector move Direction to find the nearest corner points to corner_start. Calculate the Euclidean distance between the two adjacent corner points found as the reference distance dis refer , calculate the distance dis between the two adjacent corner points found recently according to formula (6) new Distance from reference refer The integer quotient is used as the counter count:

[0178]

[0179] If count is 1, it is considered that the newly found corner point is consistent with the corner point on the calibration background and is adjacent;

[0180] If count is greater than 1, it is considered that there is a "discontinuity" between the two adjacent corner points found recently, and count-1 corner points are missed. The missed corner points are solved according to the interpolation method in formula (7);

[0181]

[0182] Where (x1, y1) is the starting coordinate of the discontinuity, (x2, y2) is the ending coordinate of the discontinuity, (x i ,y i ) is the interpolation coordinate, 1≤i≤count-1;

[0183] The detected and interpolated missed corner points are saved in the corner point set corners_col as the "column of corner points close to corner_start" on the corresponding calibration background, which serves as the starting point for reconstructing each "row" of corner points.

[0184] Row sorting: traverse the corner points in corners_col, taking each corner point as the starting point corner col-start , along the vector sign Direction to find distance corner col-start Several recently added corner points;

[0185] According to formulas (8) to (9), check the newly added corner points and corner col-start Is the constructed vector consistent with vector sign parallel:

[0186] vector = corner-corner col-start (8)

[0187]

[0188] Where: corner is the coordinate of the newly added corner point, vector is the distance between the newly added corner point and the starting point corner col-start The vector formed;

[0189] If the cosine value is not less than 0.99, it is considered that the vector and the vector sign The two vectors are essentially parallel;

[0190] Calculate the Euclidean distance between adjacent corner points found as the reference distance dis refer , calculate the distance dis between the two adjacent corner points found recently according to formula (6) new Distance from reference refer The integer quotient is used as the counter count:

[0191] If count is 1, it is considered that the newly found corner point is consistent with the corner point on the calibration background and is adjacent;

[0192] If count is greater than 1, it is considered that there is a "discontinuity" between the two adjacent corner points found recently, and count-1 corner points are missed. The missed corner points are solved according to the interpolation method in formula (7);

[0193] When the latest corner point and corner appear col-start When the constructed vector does not satisfy formula (9), it is considered that all corner points in this direction have been found;

[0194] Continue to traverse the next starting corner point and repeat the above process until you find each row of corner points starting from the corner point in corners_col.

[0195] Each row of corner points is stored in a point set corners_row_N (1≤N≤Row).

[0196] S3253 corner points correspond to:

[0197] For each point set corners_row_N (1≤N≤Row), calculate the number of corner points in the point set one by one, divided into the following cases:

[0198] No missed corner detection: If the number of corner points in each point set is exactly equal to the Col value of the corner points in the chessboard, and N is equal to the number of rows of the corner points in the chessboard, it means that all corner points on the image have been identified and rearranged. At this time, a Row×Col matrix is ​​directly constructed according to the number of corner points in the chessboard. The distribution of the matrix corresponds to the x and y coordinates in the three-dimensional world coordinates, and then multiplied by the actual length len of the small squares in the chessboard. The z coordinates are all set to 0. According to formula (10), the three-dimensional world coordinate points corresponding to all corner points are generated:

[0199]

[0200] Where i, j represents the position of the pixel, 1≤i≤Col, 1≤j≤Row, corner ij Represents the three-dimensional coordinates of the corner point in the i-th row and j-th column of the corresponding image;

[0201] Corner points are missed: the distances between adjacent corner points in the point set corners_row_N need to be calculated separately. The count value is calculated according to formula (6). The count value reflects the distribution of the missed corner points. According to formula (11), the cumulative counter sum accumulates the count value obtained each time. The x-coordinate value in the 3D world coordinate point corresponding to each corner point corresponds to the sum value accumulated at this corner point. The y-coordinate value is unified to the N value corresponding to the current point set corners_row_N, and then multiplied by the actual size len of the small squares in the chessboard. The z-coordinates are all set to 0 to obtain the 3D world coordinate point corresponding to each corner point.

[0202]

[0203] Where i, j represents the position of the pixel, corner ij represents the 3D world coordinate point of the corner point in row i and column j of the corresponding image;

[0204] For the aforementioned interpolated corner point, it is necessary to delete the interpolated corner point and the corresponding 3D world coordinate point. The remaining corner point coordinates and the corresponding 3D world coordinate point are the corner point and the corresponding 3D world coordinate value on the image after the corner point reconstruction.

[0205] Preferably, in step S4, in the corner point matching output:

[0206] In step S1, image acquisition requires consistent shooting angles. However, mechanical errors in adjusting the camera's shooting angle, as well as differences in the focus position of a macro camera when photographing objects of different sizes, such as the height of the background calibration plate and the ring, can affect calibration accuracy.

[0207] Since the background pattern remains constant and the shooting angle is consistent, see Figure 7 , it can be ensured that the distribution of corner points in the calibration background image and the ring image is basically the same. Therefore, a one-to-one matching of the corner points in the two images taken at the same angle can be performed.

[0208] Specifically, the matching process is as follows:

[0209] Take the corner coordinates in the ring image as the true value coordinates and the corner coordinates in the calibration background image as the reference value coordinates, and traverse the true value coordinates and reference coordinates;

[0210] Using the nearest neighbor principle, find the reference coordinate value closest to each true coordinate, with the threshold being that the Euclidean distance between two coordinates does not exceed 3 pixels. If a reference coordinate value can be found, use this reference coordinate to find the corresponding 3D world coordinate point to which the corner point in step S3253 corresponds, generating a matching corner point pair between the true coordinate and the 3D world coordinate point. Otherwise, discard this true coordinate and continue matching the corresponding 3D world coordinate point for the next true coordinate.

[0211] Preferably, in step S5 synchronous calibration:

[0212] The matching corner point pairs generated in step S4 are input and the Zhang calibration method is used. This method calculates the camera's intrinsic and extrinsic parameters based on matching point pairs from multiple images. However, the present invention is not limited to this method; any other calibration method that accepts similar matching data pairs as input and can output camera intrinsic parameters and pose information is also applicable.

[0213] The results of this embodiment are compared with the existing MATLAB camera calibration toolbox, using the average reprojection error as the evaluation indicator of calibration accuracy:

[0214] Reprojection error: refers to the distance error between a point in three-dimensional space projected onto the image plane through a calibrated camera model and the actual observed image point.

[0215] Average reprojection error: The average reprojection error of all feature points is calculated. Obviously, the smaller the average reprojection error, the better the calibration model fits the real scene and the more accurate the calibration result.

[0216] In this embodiment, the number of rows of corner points in the chessboard is 43, and the number of columns is 42. 36 calibration background images and 36 object images are collected respectively. The maximum number of identifiable corner points in each image is Row×Col, that is, 1806. The comparison results are shown in Table 1.

[0217] Table 1 Comparison of implementation results

[0218]

[0219] Table 1 shows that after inputting the image, due to the occlusion of the ring, the MATLAB camera calibration toolbox performs calibration. Figure 8 , 18 images cannot be used for calibration, and the other 18 images that can be used for calibration have fewer corner points detected in each image and higher average reprojection error values; in comparison, see Figure 9 This method can detect unobstructed corner points in the image, mark them in red, and calibrate them. The average reprojection error is also lower, indicating that the method of this embodiment has higher calibration accuracy, better robustness, and better practicality, which helps to improve the quality of 3D reconstruction.

[0220] At this point, the specific implementation steps of this method are described.

[0221] In summary, the above are only preferred embodiments provided by the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the invention shall be included in the scope of protection of the present invention.

[0222] Those skilled in the art will appreciate that the modules in the apparatuses of the embodiments may be distributed in the apparatuses of the embodiments as described in the embodiments, or may be located in one or more apparatuses different from the embodiments with corresponding changes. The modules in the above embodiments may be combined into one module or further divided into multiple sub-modules.

[0223] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A synchronous calibration method for 3D reconstruction of macro photography objects, characterized in that: The method comprises: Image acquisition: Measure the length, width, and height of the object using measuring tools including but not limited to calipers and laser rangefinders, take the average of the length, width, and height as the average size, select or create a black and white checkerboard that meets the ratio of 1 / 40 to 1 / 20 of the average size of the object, and use a macro camera to capture images of the object and the calibration background at different shooting angles. Maintain constant lighting during shooting, adjust the focal length of the macro camera for focus each time, and obtain a sequence of calibration background images and a sequence of object images at a certain shooting angle. No image processing is performed between image sequences. Image fusion: calibrate the background image sequence and object image sequence, and fuse the multiple images in each sequence into a clear and information-rich fused image, thereby obtaining calibrated background images and object images under different shooting angles; Corner detection: Detect the corner coordinates of each fused image, that is, calibrate the background image and the object image, and find the 3D world coordinate points that correspond one-to-one with the corner coordinates in the calibrated background image. Specifically, it includes image preprocessing and image corner detection; Corner point matching output: the corner point coordinates in the object image are used as the true value coordinates, and the corner point coordinates in the calibration background image are used as the reference value coordinates. The true value coordinates are matched one-to-one with the corresponding reference value coordinates according to the nearest neighbor principle. The 3D world coordinate point corresponding to each true value coordinate is found through the correspondence between the reference value coordinates and the world coordinates of the corner points, and matching corner point pairs between the true value coordinates and the corresponding 3D world coordinate points are generated. Camera calibration uses matching corner point pairs to obtain the camera's intrinsic parameters and the pose information when the image was taken, thus achieving the simultaneous completion of camera calibration and acquisition of the object to be reconstructed in one image.

2. The synchronization according to claim 1, characterized in that An image can be used for camera calibration and simultaneously for obtaining object information in 3D reconstruction.

3. The image acquisition according to claim 1, characterized in that The following steps are involved: Measure the length, width, and height of the object using measuring tools including but not limited to calipers and laser rangefinders. Take the average of the length, width, and height as the average size. Select or create a black and white checkerboard with each small square size being 1 / 40 to 1 / 20 of the average size of the object. The number of rows and columns of the small squares in the calibration background is set to a combination of odd and even numbers (e.g., even rows and odd columns). The total area covered by the small squares must be larger than the area occupied by the object. Set the row number (Row) of the corner point on the checkerboard to be the number of small square rows minus 1, and the column number (Col) to be the number of small square columns minus 1. Fix the prepared chessboard on a flat background plate as the calibration background; A common commercially available macro camera is mounted on a tripod, and the camera height and angle are adjusted so that both the object and the calibration background appear in the camera's field of view. Set camera parameters, including but not limited to aperture, shutter speed, ISO, etc., to ensure image quality; Use a constant light source to observe the reflections of the object and the calibration background in the macro camera at different shooting angles to ensure that the light intensity remains constant during the shooting process; Collect images of the calibration background at different shooting angles. Manually adjust the focal length of the macro camera to focus at a fixed shooting angle to capture an image sequence. Then, change the shooting angle to capture at least 6 sets of image sequences to form the calibration background image sequence blank_1, blank_2, ... blank_n (n ≥ 6). Then place the object in the center of the calibration background and keep it still. The calibration background and the object need to be observed in the camera's field of view at the same time. Collect object images at the same angle. Manually adjust the focal length of the macro camera at a fixed shooting angle to make the object and the calibration background clearly focused. Shoot an image sequence. Change the shooting angle to collect at least 6 sets of image sequences to form the object image sequence obj_1, obj_2, ...obj_n (n ≥ 6). Ensure that the calibration background image sequence and the object image sequence at the same shooting angle correspond to each other.

4. The image fusion according to claim 1, characterized in that: Import the image sequences blank_1, blank_2, ... blank_n (n ≥ 6) and obj_1, obj_2, ... obj_n (n ≥ 6) in sequence, and fuse the multiple images in each sequence into a clear and information-rich fused image, thereby obtaining the calibrated background image and object image under different shooting angles.

5. The corner point detection according to claim 1, wherein: The steps include: Image pre-processing can improve the accuracy and robustness of corner detection, including: grayscale conversion, grayscale adjustment, Gaussian blur with a Gaussian kernel size of 5×5 and a standard deviation of 0, and dilation with a dilation kernel of 3×3; Image corner detection matches the calibrated background image and object image with the same shooting angle after image pre-processing. During corner detection, the corner coordinates in the calibrated background image are first detected, and then the corner coordinates in the object image are detected. Image corner detection includes five steps: edge detection, subject removal (object image only), obtaining candidate corner points, corner point verification, and corner point reconstruction. The corner detection processing of the calibrated background image and object graphics is slightly different, and the final output corner coordinates are in the same form.

6. The corner point review according to claim 5, characterized in that: The steps include: Corner point review: Some falsely detected corner points may come from non-interesting areas such as unremoved edges of objects, surface reflections, and image noise. Therefore, further screening is required. The screening criteria are based on whether the corner points are at the intersection of adjacent small squares, and a certain offset of the corner points is allowed. Specifically, pixel blocks with grayscale values ​​greater than 90 are considered white pixels, and vice versa as black pixels. The distribution ratio of black and white pixels in a 15-pixel circular neighborhood around each candidate corner coordinate is calculated, and the corner coordinates are arranged in order from small to large according to the ratio: For the calibration background image: remove the 5% corner points at the beginning and end of the arrangement, and retain the coordinates of the 90% corner points in the middle of the arrangement as the corner point detection result of the calibration background image; For object images: remove the first and last 20% of the corner points, and retain the coordinates of the 60% of the corner points in the middle as the corner point detection results of the object image.

7. The corner point reconstruction according to claim 5, characterized in that: Including endpoint recognition, corner point sorting, and corner point correspondence.

8. The corner point matching output according to claim 1, characterized in that: The steps include: Take the corner point coordinates in the object image as the true value coordinates, and the corner point coordinates in the calibration background image as the reference value coordinates, and traverse the true value coordinates and reference coordinates; Based on the nearest neighbor principle, find the reference coordinate value closest to each true value coordinate, and the judgment threshold is that the Euclidean distance between two coordinates does not exceed 3 pixels; If it can be found, the corresponding three-dimensional world coordinate point corresponding to the corner point is found based on this reference coordinate, and a matching corner point pair of the true value coordinate and the three-dimensional world coordinate point is generated. Otherwise, the true value coordinate is abandoned and the corresponding three-dimensional world coordinate point is matched for the next true value coordinate.

Images