A 3D vision-based splicing calibration method and size detection system

Abstract

The present application relates to a kind of based on 3D vision splicing calibration method and size detection system, it is related to machine vision technical field;Splicing calibration method, comprising: according to different depth camera scanning standard plane calibration piece, obtain corresponding calibration piece depth map;The calibration point of the calibration piece depth map is detected, corresponding image flatness is calculated and judged, and read several image point coordinates;According to the actual point set of different depth camera under the same reference coordinate system, the inter-camera transformation matrix is calculated;According to the inter-camera transformation matrix, complete all depth image splicing of rephotographing, and the workpiece size of the standard plane calibration piece is calculated;According to workpiece size, two or more 3D sensor is calibrated and workpiece image is collected, then multi-view workpiece image is fused, realize fast splicing and engineering size accurate detection, can satisfy the high-precision size detection of various product features.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and in particular to a stitching calibration method and size detection system based on 3D vision. Background Technology

[0002] With the continuous development of intelligent manufacturing and industrial automation, machine vision technology is being used more and more widely in the industrial field, especially in tasks such as automated detection, positioning, guidance and measurement, where it plays a crucial role.

[0003] Traditional 2D vision systems suffer from low measurement accuracy, significant occlusion interference, and lack of spatial information when dealing with tasks such as complex three-dimensional structures, precision measurement, and inspection of large-sized workpieces. They are no longer able to meet the requirements for high-precision, multi-dimensional inspection.

[0004] To obtain complete 3D information with a large field of view or multiple perspectives, it is necessary to effectively stitch and register point cloud data acquired by multiple cameras or the same camera at different locations to construct a complete 3D model of the target workpiece. This typically relies on high-precision multi-view calibration methods and robust point cloud stitching algorithms.

[0005] Application number CN202310757309.4 discloses a calibration method for parallel scanning and stitching of multi-line laser cameras, including: S1, obtaining the installation positions of N cameras and the fixed positions of N calibration blocks, scanning the calibration block data through one of the cameras, and obtaining the calibration block point cloud; S2, extracting corner points of the calibration blocks based on the calibration block point cloud, and generating the corner point coordinates of a single calibration block; S3, integrating multiple calibration blocks into a large calibration block in the same coordinate system through coordinate transformation; S4, calculating the transformation relationship of scanning data from different cameras to the coordinate system of the large calibration block through synchronous scanning of multiple cameras, thereby completing the calibration of multi-camera parallel scanning and stitching; using multiple small calibration blocks for combination and stitching, the position of the calibration blocks can be adjusted according to different products under test, applicable to various measurement ranges, with flexible application scenarios, and high calibration accuracy using three-dimensional calibration blocks.

[0006] The existing technical solutions mentioned above have the following drawbacks: 1. The existing splicing and calibration methods are cumbersome, require a high degree of manual intervention, have strict requirements for on-site installation, and are difficult to achieve rapid deployment and automated integration. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention provides a 3D vision-based splicing calibration method and dimensional inspection system. By fusing multi-view workpiece images, it achieves rapid splicing and accurate inspection, providing a complete and high-precision three-dimensional vision inspection solution for large or complex structure workpieces.

[0008] This invention is achieved through the following technical solutions: A stitching calibration method based on 3D vision includes: By scanning the standard planar calibration component with cameras at different depths, the corresponding depth map of the calibration component is obtained; The calibration points of the depth map of the calibration component are detected, the corresponding image flatness is calculated and determined, and the coordinates of several image points are read; the transformation matrix between cameras is calculated based on the actual point sets of different depth cameras under the same reference coordinate system. The depth images re-captured are stitched together using the camera transformation matrix, and the workpiece dimensions of the standard planar calibration component are calculated.

[0009] By adopting the above technical solution, depth maps are obtained by scanning standard planar calibration parts with multiple depth cameras. Data reliability is ensured based on feature point detection and flatness screening. High-precision transformation matrices between cameras are calculated using actual point sets and registration algorithms such as ICP. Finally, seamless stitching of full-view depth images and accurate measurement of workpiece dimensions are achieved, significantly improving the accuracy and efficiency of 3D reconstruction.

[0010] The present invention is further configured such that the specific steps for obtaining the corresponding calibration depth map based on scanning the standard planar calibration part with cameras at different depths include: m depth cameras are deployed in parallel to the same workpiece, and the workpiece is locked to the servo module; A standard plane calibration piece with known length, width, and height (x, y, z) is fixed to the marble platform; The servo module drives the workpiece to move m depth cameras to scan the standard planar calibration part, respectively, to obtain the corresponding calibration part depth map (I1, I2, I3, ..., I...). m ).

[0011] By adopting the above technical solution, multiple depth cameras are fixed on the workpiece driven by a servo module, and the module is used to drive the cameras to perform high-precision and automated scanning of a standard planar calibration part with known dimensions fixed on a stable marble platform, thereby obtaining a series of depth maps corresponding to precise poses. The servo module and marble platform ensure the accuracy of motion control and environmental stability, realize the automation and high repeatability of data acquisition, and provide accurate prior information on poses, thereby significantly improving the robustness and accuracy of calibration results.

[0012] The present invention is further configured such that: the specific steps of detecting the calibration points of the depth map of the calibration component, calculating and determining the corresponding image flatness, and reading the coordinates of several image points include: Edge detection was performed on the depth maps of all calibration parts to obtain right-angled edges; Based on the right-angled edges of the diagram, construct the corresponding reference coordinate system (G1, G2, ... Gn); The depth map of the calibration component is sampled and fitted according to the reference coordinate system to generate the reference plane BP. (1,2,...,m) ; Based on the X and Y resolutions of the depth camera and the reference plane, the plane coordinates (x, y) of the calibration point are determined; based on the plane coordinates of the calibration point and the preset region radius, a circular region class is constructed, and the z-coordinate value of the calibration point is calculated. Based on the calibration point coordinates and the corresponding z-coordinate value, the image point coordinates P are obtained. (1,2,...,m) ={(x1,y1,z1),(x2,y2,z2),...,(x25,y25,z25)}; The flatness of the calibration part depth map is calculated based on the coordinates of the image points and compared with a preset flatness threshold. If all the flatness values ​​are less than the flatness threshold, the current calibration part depth map is compliant; otherwise, the standard flatness calibration part is rescanned.

[0013] By adopting the above technical solution, right-angled edges are obtained by edge detection of the depth map of the calibration part, and a reference coordinate system is constructed. A reference surface is generated by sampling and fitting based on this coordinate system. The plane coordinates (x, y) of the calibration point are determined by combining the depth camera resolution. Then, a circular area with a preset radius is constructed to calculate the z-coordinate value, and finally a set of coordinates of 25 image points is obtained. By calculating the flatness of these spatial points (such as calculating the root mean square error from the point cloud to the plane after least square fitting the plane) and comparing it with a preset threshold, the flatness compliance judgment is realized automatically. If the error exceeds the tolerance, a rescan is triggered. The reference coordinate system constructed by geometric features ensures the accuracy of coordinate transformation, the circular sampling area effectively suppresses noise interference, the joint evaluation of flatness by multiple spatial points significantly improves the detection robustness, and the closed-loop feedback mechanism ensures the reliability of calibration data.

[0014] The present invention is further configured such that the specific steps of constructing a circular region class based on the plane coordinates of the calibration point and a preset region radius, and calculating the z-coordinate value of the calibration point, include: Scan the circular region to obtain the initial set of scan points, points1; The distance to each scan point in the initial scan point set points1 is calculated based on the reference surface to obtain a first distance, which is then compared with a preset point-to-surface distance threshold. If the first distance is greater than the point-to-surface distance threshold, then the current scan point is directly filtered. Otherwise, aggregate the current scan points to obtain the corrected scan point set points2; Based on the reference surface, the distances to each scan point in the corrected scan point set points2 are calculated to obtain the second distance d = {d1, d2, ..., d...} n}; The mean μ and standard deviation σ are calculated for all the second distances; Where i is the scan point number; Outlier filtering is performed based on the mean and standard deviation combined with the 3x standard deviation rule to obtain the final set of scan points, points3. The z-coordinate value of the calibration point is obtained by averaging the z-coordinate values ​​of all scan points in the final scan point set points3.

[0015] By adopting the above technical solution, a circular region class is first constructed based on the plane coordinates of the calibration points and a preset radius. An initial scan obtains a point set `points1`. The distances between points are calculated using a reference plane, and a pass-through filter (removing points with excessively large distances) or aggregation operation is performed to generate a corrected point set `points2`. Subsequently, the reference plane distances are calculated a second time for `points2`, and outliers are filtered using a 3x standard deviation rule to obtain the final point set `points3`. Finally, the mean of the z-coordinates of the calibration points is taken as the output. Multi-level distance constraints and statistical filtering mechanisms effectively suppress noise interference, and the mean fusion strategy significantly improves the accuracy and robustness of the z-coordinates. Furthermore, the algorithm is efficient and compact, making it suitable for high-precision 3D calibration scenarios.

[0016] The present invention is further configured such that: the specific steps for calculating the transformation matrix between cameras based on the actual point sets of different depth cameras in the same reference coordinate system include: Based on the width and height dimensions of the standard planar calibration component and the coordinates of the image points, the actual point set SP corresponding to all the image point coordinates under the same reference coordinate system is obtained. (1,2,...,m) ; Based on the original image point coordinates and corresponding actual point sets in the current reference coordinate system, calculate the reference camera transformation matrix T. GmCm ; Based on the reference camera transformation matrix, the remaining actual point set SP (1,2,...,m-1) Perform the transformation to obtain the transformation point set SP (1,2,...,m- 1)′; The camera transformation matrix is ​​obtained by calculating the transformation point set and the corresponding image point coordinates.

[0017] By adopting the above technical solution, firstly, based on the image points and actual point sets of the reference camera (calculating the reference camera transformation matrix (T_{GmCm})) (essentially solving a PnP problem), then using (T_{GmCm}) to uniformly transform the actual point sets of non-reference cameras to the reference camera coordinate system, forming a transformed point set. Finally, by matching the transformed point set with the image point coordinates of non-reference cameras (usually using SVD decomposition or least squares method to solve point cloud registration), the transformation matrix of each camera relative to the reference camera is directly calculated. Through the reference camera mediation calibration process, the error accumulation of pairwise calibration of multiple cameras is avoided, significantly improving calibration efficiency and global consistency.

[0018] The present invention is further configured such that: the reference camera transformation matrix T is calculated based on the original image point coordinates in the current reference coordinate system and the corresponding actual point set. GmCm The specific steps include: The original image point coordinates X in the current reference coordinate system m and the corresponding actual point set X m Perform random consistent sampling to obtain s pairs of non-coplanar points, and construct non-coplanar point pairs (X). s ′,X s ); Based on the s pairs of non-coplanar points, construct the affine transformation equation X. s ′=RX s +t, where R is the rotation matrix and t is the translation vector, and calculate the affine transformation matrix H; The reprojection error is calculated for all non-coplanar point pairs based on the affine transformation matrix and compared with a preset standard deviation threshold. If the reprojection error is less than the standard deviation threshold, the current non-coplanar point pair is considered an interior point, and the number of interior points is counted. The total number of interior points is sorted in descending order according to the preset number of samplings, and the affine transformation matrix is ​​recalculated based on the affine transformation equation with the most interior points to generate the optimal transformation matrix.

[0019] By adopting the above technical solution, based on the image points and actual point sets of the reference camera, multiple sets of non-coplanar point pairs are selected through random consistent sampling (RANSAC idea), and the affine transformation equation is constructed to solve the initial transformation matrix (H). Then, interior points are screened and their numbers are counted through reprojection error. Finally, the optimal transformation matrix (T_{GmCm}) is recalculated and generated based on the maximum interior point set using the least squares method (or SVD decomposition). Through iterative sampling and interior point optimization mechanism, noise and outlier interference are effectively resisted, and the robustness and accuracy of the calibration matrix are significantly improved.

[0020] The present invention is further configured such that: the specific steps of stitching together all re-captured depth images according to the inter-camera transformation matrix and calculating the workpiece dimensions of the standard planar calibration component include: The standard planar calibration component is scanned again to obtain the corresponding depth image; Construct the original empty image based on the sum of the field of view of all depth cameras; The depth images are transformed and stitched together according to the camera transformation matrix, and then updated to the original empty image to obtain the final calibration depth map. The depth map of the final calibration part is inspected and calculated to obtain the workpiece size of the standard planar calibration part.

[0021] By adopting the above technical solution, multi-view depth images are obtained by rescanning the standard planar calibration part. An initial blank image is constructed based on the sum of the fields of view of all cameras. Geometric transformation and depth value fusion and stitching are performed using a pre-calibrated inter-camera transformation matrix (such as a homography matrix). The blank image is then updated to generate a complete calibration part depth map. Finally, the workpiece size is detected and calculated by plane fitting or feature extraction algorithms. This achieves high-precision, unobstructed automated measurement and significantly improves the accuracy and efficiency of size calculation.

[0022] Secondly, the present invention also provides a 3D vision-based size inspection system, employing the following technical solution: a 3D vision-based size inspection system for implementing the aforementioned 3D vision-based stitching calibration method, comprising: The image acquisition module is used to acquire depth images of the workpiece using cameras of different depths; The stitching calibration module is used to calculate the transformation matrix between cameras based on the standard plane calibration parts for scanning at different depths, combined with the reference coordinate system. The image stitching module is used to stitch together all the workpiece depth images according to the camera transformation matrix to obtain a complete workpiece depth map; The dimension measurement module is used to detect the depth map of the complete workpiece at the edge and calculate the corresponding workpiece dimensions.

[0023] By adopting the above technical solution, multiple depth cameras are used to simultaneously acquire local depth images of the workpiece. A high-precision transformation matrix between cameras is calculated using standard planar calibration parts combined with feature point matching (such as SIFT / SURF) and least squares optimization algorithms. Based on this matrix, a global depth map is stitched together using point cloud registration technology (such as ICP) and Poisson fusion algorithm. Finally, the workpiece size is automatically measured by Canny edge detection and sub-pixel contour analysis. By breaking through the field of view limitation of a single camera, high-precision and high-efficiency three-dimensional size detection of the entire surface of large workpieces is achieved, with an accuracy of millimeters and full automation of the process.

[0024] Thirdly, the present invention also provides an electronic device, comprising: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement any of the methods described in the above scheme.

[0025] Fourthly, the present invention also provides a storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement a 3D vision-based stitching calibration method as described above.

[0026] In summary, the beneficial technical effects of the present invention are as follows: 1. By scanning standard planar calibration parts with multiple depth cameras to obtain depth maps, and ensuring data reliability based on feature point detection and flatness screening, a high-precision transformation matrix between cameras is calculated using actual point sets and registration algorithms such as ICP. Finally, seamless stitching of full-view depth images and accurate measurement of workpiece dimensions are achieved, significantly improving the accuracy and efficiency of 3D reconstruction. 2. The reference coordinate system constructed through geometric features ensures the accuracy of coordinate transformation, the circular sampling area effectively suppresses noise interference, and the joint evaluation of flatness by multiple spatial points significantly improves the robustness of detection. At the same time, the closed-loop feedback mechanism ensures the reliability of calibration data. 3. The multi-level distance constraint and statistical filtering mechanism (direct filtering, aggregation and 3σ anomaly removal) effectively suppresses noise interference. Combined with the mean fusion strategy, the accuracy and robustness of z coordinate are significantly improved. At the same time, the algorithm process is efficient and compact, and it is suitable for high-precision 3D calibration scenarios. Attached Figure Description

[0027] Figure 1 This is a flowchart of a splicing calibration method according to one embodiment of the present invention.

[0028] Figure 2 This is a flowchart of a splicing calibration method according to one embodiment of the present invention.

[0029] Figure 3 This is a flowchart of a splicing calibration method according to one embodiment of the present invention.

[0030] Figure 4 This is a structural diagram of a dimension detection system according to one embodiment of the present invention. Detailed Implementation

[0031] The present invention will be further described in detail below with reference to the accompanying drawings.

[0032] Example 1: Reference Figure 1 The present invention discloses a 3D vision-based stitching calibration method, comprising: S1: Obtain the corresponding depth map of the calibration component by scanning the standard planar calibration component with different depth cameras; S2: Detect the calibration points of the depth map of the calibration component, calculate and determine the corresponding image flatness, and read the coordinates of several image points; S3: Calculate the transformation matrix between cameras based on the actual point sets of cameras at different depths under the same reference coordinate system; S4: Complete the stitching of all re-captured depth images according to the camera transformation matrix, and calculate the workpiece size of the standard planar calibration part.

[0033] In this embodiment, images of a standard known length, width and height planar calibration component on 3D camera 1 and camera 2 are scanned and acquired respectively, and are denoted as image1 and image2; The planness 1 and planness 2 of the obtained images 1 and 2 are detected respectively; If both planness1 and planness2 are less than the set threshold p, proceed to the next step; otherwise, check the environment and repeat the scan. Obtain the actual xyz coordinates of the 25 known points in image1 and image2; Data processing was performed on 25 points of camera 1 and camera 2 respectively, and the relationship matrix between the two cameras was obtained by calibration. The workpiece is rescanned, and the resulting relation matrix is ​​used to stitch the images together to calculate the 3D related dimensions of the entire workpiece.

[0034] The implementation principle of this embodiment is as follows: a standard planar calibration part of known size is scanned by dual depth cameras to obtain depth maps (Image1 and Image2) respectively. After the flatness of both images is detected to be lower than the threshold p, the actual spatial coordinates of 25 calibration points are extracted. Based on these sets of points with the same name, the spatial transformation relationship matrix between the cameras is calculated. Finally, the depth map of the re-scanned workpiece is stitched together in real time using this matrix to achieve high-precision three-dimensional dimension measurement.

[0035] Example 2: Step S1 includes: m depth cameras are deployed in parallel to the same workpiece, and the workpiece is locked to the servo module; A standard plane calibration piece with known length, width, and height (x, y, z) is fixed to the marble platform; The servo module drives the workpiece to move m depth cameras to scan the standard planar calibration part, respectively, to obtain the corresponding calibration part depth map (I1, I2, I3, ..., I...).m ).

[0036] In this embodiment, two 3D laser sensors are mounted side by side on the same workpiece, and the workpiece is locked on the servo module. A pre-calibrated planar calibration piece with known length, width, and height (x, y, z) is placed on a marble platform. The servo drive simultaneously drives the two 3D laser sensors to complete the scanning, obtaining the corresponding 3D depth images image1 and image2.

[0037] The implementation principle of this embodiment is as follows: m depth cameras are deployed in parallel on the same workpiece and locked to the servo module. The workpiece is moved as a whole by the servo drive so that it scans the known-sized standard plane calibration part fixed on the stable marble platform, thereby obtaining a corresponding depth map for each camera and establishing their spatial relationship in a unified coordinate system.

[0038] Example 3: Step S2 includes: S21: Perform edge detection on the depth map of all calibration parts to obtain the right-angled edges of the map; S22: Based on the right-angled edges of the diagram, construct the corresponding reference coordinate system (G1, G2, ... Gn); S23: Sample and fit the depth map of the calibration component according to the reference coordinate system to generate the reference plane BP. (1,2,...,m) ; S24: Determine the plane coordinates (x,y) of the calibration point based on the X-direction resolution and Y-direction resolution of the depth camera and the reference plane. S25: Based on the plane coordinates of the calibration point and the preset area radius, construct a circular area class and calculate the z-coordinate value of the calibration point; S26: Based on the calibration point coordinates and the corresponding z-coordinate value, obtain the image point coordinates P. (1,2,...,m) ={(x1,y1,z1),(x2,y2,z2),...,(x25,y25,z25)}; S27: Calculate the flatness of the depth map of the calibration component based on the coordinates of the image points, and compare it with a preset flatness threshold; If all of the stated flatness is less than the stated flatness threshold, then the current calibration part depth map is compliant; otherwise, the standard flat calibration part is rescanned.

[0039] In this embodiment, right-angled edges of images image1 and image2 are detected respectively, and then coordinate references G1 and G2 are constructed. Under this coordinate system, 25 points P1 = {(x1,y1,z1),(x2,y2,z2),...,(x25,y25,z25)} and P2 = {(x1,y1,z1),(x2,y2,z2),...,(x25,y25,z25)} are obtained in the positive column distribution of the image respectively. The planness1 and planness2 representing image1 and image2 are calculated for P1 and P2 respectively; The planness1 and planness2 are evaluated. If they meet the requirements, the next calibration step is performed; otherwise, the scan is repeated.

[0040] The implementation principle of this embodiment is as follows: Multiple depth cameras are deployed in parallel on the same workpiece and locked to a servo module. The servo drive synchronously scans a standard planar calibration part fixed on a marble platform to acquire depth images from each camera. Then, edge detection is performed on the depth images to extract right-angle edge features, and a unified reference coordinate system (such as G1, G2) is constructed based on this. A reference surface is generated by sampling and fitting. The planar coordinates (x, y) of the calibration points are determined by combining the resolution parameters of the depth cameras, and the depth coordinates z are calculated based on the preset area radius. Finally, the planar and depth information are integrated to generate an image point coordinate set containing 25 points (such as P1, P2). The planarity (planness1, planness2) of the depth map is calculated and compared with a threshold. If all planarity values ​​meet the standard, the depth map is confirmed to be compliant; otherwise, a rescanning process is triggered, thereby ensuring the high accuracy and stability of the calibration data and providing a reliable foundation for subsequent calculation of the transformation matrix between cameras and measurement of workpiece dimensions.

[0041] Example 4: Reference Figure 2 Step S25 includes: Scan the circular region to obtain the initial set of scan points, points1; The distance to each scan point in the initial scan point set points1 is calculated based on the reference surface to obtain a first distance, which is then compared with a preset point-to-surface distance threshold. If the first distance is greater than the point-to-surface distance threshold, then the current scan point is directly filtered. Otherwise, aggregate the current scan points to obtain the corrected scan point set points2; Based on the reference surface, the distances to each scan point in the corrected scan point set points2 are calculated to obtain the second distance d = {d1, d2, ..., d...} n}; The mean μ and standard deviation σ are calculated for all the second distances; Where i is the scan point number; Outlier filtering is performed based on the mean and standard deviation combined with the 3x standard deviation rule to obtain the final set of scan points, points3. The z-coordinate value of the calibration point is obtained by averaging the z-coordinate values ​​of all scan points in the final scan point set points3.

[0042] In this embodiment, BP1 and BP2 are obtained by fitting a reference plane to images image1 and image2. The X-coordinate of each point P1 and P2 depends on the X-direction resolution of the 3D sensor, and the Y-coordinate of each point depends on the Y-direction resolution of the 3D sensor. The value of the Z coordinate depends on the filtered average of all points in a circular region centered at (x,y) with radius r1; Get the collection of all 3D scan points of the circular region class, points1; Set a threshold d for the distance from a point to a surface. For points1, use a pass-through filter to filter out points that are greater than the threshold d, and obtain a set of points2 that meet the conditions. Continue to calculate the distance from the point set points2 to the reference plane. Using the 3σ principle, assuming that the data follows a normal distribution, data points that deviate from the mean by more than 3 times the standard deviation are regarded as outliers and removed, resulting in points3. The final z-coordinate value is obtained by calculating the average of the Z values ​​of the point set points3.

[0043] The implementation principle of this embodiment is as follows: For the initial set of scanning points in the calibration area, outliers that are more than a preset threshold away from the reference plane are first filtered out by direct filtering to obtain a corrected set of points; then, based on the distance distribution of the point set to the reference plane, outliers are further removed by applying the statistical principle of three times the standard deviation; finally, the average value of the Z coordinates of the filtered high-quality point set is calculated to accurately calculate the Z coordinates of the calibration points.

[0044] Example 5: Reference Figure 3 Step S3 includes: S31: Based on the width and height dimensions of the standard planar calibration component and the coordinates of the image points, obtain the actual point set SP corresponding to all the image point coordinates under the same reference coordinate system. (1,2,...,m) ; S32: Calculate the reference camera transformation matrix T based on the original image point coordinates and the corresponding actual point set in the current reference coordinate system. GmCm; S33: Apply the remaining actual point set SP according to the aforementioned reference camera transformation matrix. (1,2,...,m-1) Perform the transformation to obtain the transformation point set SP (1,2,...,m-1) ′; S34: Calculate the camera transformation matrix based on the set of transformation points and the corresponding image point coordinates.

[0045] In this embodiment, given the width and height dimensions w and h of the standard planar workpiece, and the 36 points P1 and P2 in the positive column distribution of the image under coordinate references G1 and G2, the actual point set SP1 corresponding to the P1 point set and the actual point set SP2 corresponding to the P2 point set can be obtained under the coordinate system of calibration plate G1. Using P1 and SP1, the transformation matrix T from the calibration plate coordinate system G1 to the camera 1 coordinate system C1 is calculated. G1C1 ; Through T G1C1 The relation can transform the point set SP2 in the G1 coordinate system to the C1 coordinate system of camera 1, resulting in SP2'; The matrix transformation relationship T between camera 1 and camera 2 can be obtained by calculating SP2' and P2. C1C2 .

[0046] The implementation principle of this embodiment is as follows: Based on a standard planar calibration component with known width and height dimensions, the coordinates of image points obtained by multiple depth cameras are converted into actual point sets SP(1,2,…,m) through a reference coordinate system. The transformation matrix TGmCm (e.g., TG1C1) between the reference camera and the calibration plate coordinate system is constructed using SVD decomposition and plane fitting techniques. The actual point sets of other cameras (e.g., SP2) are mapped to the coordinate system of the reference camera (e.g., C1) through this matrix to obtain the transformed point set SP'. Combining the transformed point set with the corresponding image point coordinates, the transformation matrix between cameras (e.g., TC1C2) is solved using the least squares method or matrix operations. This achieves accurate alignment of the multi-camera coordinate systems and global consistency of the calibration data, ensuring high accuracy and stability for subsequent workpiece depth map stitching and dimensional measurement.

[0047] Example 6: Step S32 includes: The original image point coordinates X in the current reference coordinate system m and the corresponding actual point set X m Perform random consistent sampling to obtain s pairs of non-coplanar points, and construct non-coplanar point pairs (X). s ′,X s ); Based on the s pairs of non-coplanar points, construct the affine transformation equation X. s ′=RX s+t, where R is the rotation matrix and t is the translation vector, and calculate the affine transformation matrix H; The reprojection error is calculated for all non-coplanar point pairs based on the affine transformation matrix and compared with a preset standard deviation threshold. If the reprojection error is less than the standard deviation threshold, then the current non-coplanar point pair is considered an interior point, and the number of interior points is counted. The number of all interior points is sorted in descending order according to a preset number of samplings, and the affine transformation matrix is ​​recalculated based on the affine transformation equation with the most interior points to generate the optimal transformation matrix.

[0048] In this embodiment, a three-dimensional affine transformation is calculated by randomly selecting three pairs of non-coplanar points (minimum subsets) from the source point set P1 and the target point set SP1 using the random consistency sampling algorithm. Based on the selected 3 pairs of points, calculate the affine transformation matrix H using the least squares method; Points with errors less than the threshold (default 3 times the standard deviation) are considered interior points, and the number of interior points is counted. Repeat the above steps multiple times (usually 3 times) and select the model with the most interior points as the optimal solution. The transformation matrix of the interior points of the optimal model is recalculated (least squares optimization) to improve accuracy.

[0049] Steps S33 and S34 use the same calculation method as step S32, and will not be described again here.

[0050] The implementation principle of this embodiment is as follows: the calculation of the affine transformation matrix is ​​optimized by using the Random Sample Consensus (RANSAC) algorithm. First, s pairs of non-coplanar points (e.g., 3 pairs of points forming the smallest subset) are randomly selected from the image point set (X_m) and the actual point set (X_m'). The affine transformation equation (X_s' = RX_s + t) is constructed based on the least squares method to solve for the initial transformation matrix (H). Then, the reprojection error of all point pairs is calculated and compared with a preset standard deviation threshold (e.g., 3 times). Points below the threshold are identified as interior points and their number is counted. The above sampling-calculation-verification process is repeated multiple times (e.g., a fixed number of times), and the model with the most interior points is selected as the optimal solution. Finally, the transformation matrix is ​​recalculated based on all interior points of the model (e.g., least squares optimization) to generate a high-precision, noise-resistant optimal transformation matrix, which significantly improves the robustness and accuracy of camera calibration or 3D reconstruction.

[0051] Example 7: Step S4 includes: The standard planar calibration component is scanned again to obtain the corresponding depth image; Construct the original empty image based on the sum of the field of view of all depth cameras; The depth images are transformed and stitched together according to the camera transformation matrix, and then updated to the original empty image to obtain the final calibration depth map. The depth map of the final calibration part is inspected and calculated to obtain the workpiece size of the standard planar calibration part.

[0052] In this embodiment, the workpiece is rescanned to obtain two images of the workpiece under camera 1 and camera 2, which are denoted as src1 and src2 respectively; Create a new original image dst, whose size should be able to encompass the field of view of both camera 1 and camera 2; Using the transformation matrix TC1C2 obtained from the above calibration of camera 1 to camera 2, the transformed images are calculated and updated in the final image dst; Perform the relevant size calculations in the new image dst.

[0053] The implementation principle of this embodiment is as follows: a depth image is obtained by rescanning the standard planar calibration part, and an original empty image (such as dst) is constructed based on the field of view of all depth cameras. The coordinate transformation and stitching of the depth images of each camera are performed using the calibrated inter-camera transformation matrix (such as TC1C2), and the depth information from different perspectives is fused into a unified coordinate system to form the final calibration part depth map. Then, by detecting the geometric features (such as plane edges or point set distribution) in the depth map, and combining the known dimensions of the calibration plate with the coordinate mapping relationship, the actual workpiece size of the standard planar calibration part is calculated and verified, thereby realizing the integration of multi-camera depth data and high-precision measurement.

[0054] Example 8: Reference Figure 4 This is a 3D vision-based size inspection system applied to the aforementioned splicing calibration method, comprising: The image acquisition module is used to acquire depth images of the workpiece using cameras of different depths; The stitching calibration module is used to calculate the transformation matrix between cameras based on the standard plane calibration parts for scanning at different depths, combined with the reference coordinate system. The image stitching module is used to stitch together all the workpiece depth images according to the camera transformation matrix to obtain a complete workpiece depth map; The dimension measurement module is used to detect the depth map of the complete workpiece at the edge and calculate the corresponding workpiece dimensions.

[0055] The implementation principle of this embodiment is as follows: multi-view depth images of the workpiece are acquired through the image acquisition module; the transformation matrix between each depth camera is calculated by scanning a standard plane calibration part and establishing a reference coordinate system using the stitching calibration module; then, the coordinate transformation and fusion of all depth images are performed by the image stitching module using the transformation matrix to generate a complete workpiece depth map under a unified coordinate system; finally, the dimension measurement module performs edge detection and calculation on the depth map to accurately determine the size of the workpiece.

[0056] Example 9: An electronic device, comprising: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement any of the methods described in the above scheme.

[0057] Example 10: A storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement a 3D vision-based stitching calibration method as described above.

[0058] The embodiments described herein are preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape, and principle of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A stitching calibration method based on 3D vision, characterized in that, include: Based on the scanning of the standard planar calibration component by m depth cameras, the corresponding calibration component depth map is obtained; The calibration points of the depth map of the calibration component are detected, the corresponding image flatness is calculated and determined, and the coordinates of several image points are read, including: Edge detection was performed on the depth maps of all calibration parts to obtain right-angled edges; Based on the right-angled edges of the diagram, construct the corresponding reference coordinate system (G1, G2, ... Gn); Based on the reference coordinate system, the depth map of the calibration component is sampled and fitted to generate reference surfaces BP1, BP2, ..., BP3. m ; Based on the X-axis and Y-axis resolutions of the depth camera and the reference surface, the plane coordinates (x, y) of the calibration point are determined. Based on the plane coordinates of the calibration point and the preset area radius, a circular area class is constructed, and the z-coordinate value of the calibration point is calculated; Based on the plane coordinates of the calibration points and the corresponding z-coordinate values, the image point coordinate set P1, P2, ..., P is obtained. m Each image point coordinate set includes 25 calibration point three-dimensional coordinates; The flatness of the calibration component depth map is calculated based on the coordinates of the image points and compared with a preset flatness threshold. If all of the stated flatness is less than the stated flatness threshold, then the current calibration part depth map is compliant; otherwise, the standard flatness calibration part is rescanned. Based on the actual point sets of different depth cameras in the same reference coordinate system, calculate the transformation matrix between cameras, including: Based on the width and height dimensions of the standard planar calibration component and the coordinates of the image points, the actual point set SP1, SP2, ..., SP1 corresponding to all the image point coordinates under the same reference coordinate system is obtained. m ; Based on the original image point coordinates and the corresponding actual point set in the current reference coordinate system, calculate the reference camera transformation matrix T. GmCm ,include: For the original image point coordinate set P in the current reference coordinate system m and the corresponding actual point set SP m Perform random consistent sampling to obtain s pairs of non-coplanar points, and construct non-coplanar point pairs (X). s ′,X s ), where X s ′ represents the actual point set SP m The actual coordinates of the point, X s P is the set of coordinates of image points m Image point coordinates; Based on the s pairs of non-coplanar points, construct the affine transformation equation X. s =RX s +t, where R is the rotation matrix and t is the translation vector, and calculate the affine transformation matrix H; ； The reprojection error is calculated for all non-coplanar point pairs based on the affine transformation matrix and compared with a preset standard deviation threshold. If the reprojection error is less than the standard deviation threshold, then the current non-coplanar point pair is considered an interior point, and the number of interior points is counted. The number of all interior points is sorted in descending order according to a preset number of samplings, and the affine transformation matrix is ​​recalculated based on the affine transformation equation with the most interior points to generate the optimal transformation matrix. Based on the reference camera transformation matrix, the remaining actual point sets SP1 and SP2 are... ... SP m-1 Perform the transformation to obtain the transformation point set SP1′, SP2′. ... SP m-1 ′; The camera transformation matrix is ​​calculated based on the set of transformation points and the corresponding image point coordinates. The depth images re-captured are stitched together using the camera transformation matrix, and the workpiece dimensions of the standard planar calibration component are calculated.

2. The stitching calibration method based on 3D vision according to claim 1, characterized in that, The specific steps for obtaining the corresponding calibration depth map based on scanning the standard planar calibration component using m depth cameras include: m depth cameras are deployed in parallel to the same workpiece, and the workpiece is locked to the servo module; A standard plane calibration piece with known length, width, and height (x, y, z) is fixed to the marble platform; The servo module drives the workpiece to move m depth cameras to scan the standard planar calibration part, respectively, to obtain the corresponding calibration part depth map (I1, I2, I3, ..., I...). m ).

3. The stitching calibration method based on 3D vision according to claim 1, characterized in that, The specific steps for constructing a circular region class based on the plane coordinates of the calibration point and a preset region radius, and calculating the z-coordinate value of the calibration point, include: Scan the circular region to obtain the initial set of scan points, points1; The distance to each scan point in the initial scan point set points1 is calculated based on the reference surface to obtain a first distance, which is then compared with a preset point-to-surface distance threshold. If the first distance is greater than the point-to-surface distance threshold, then the current scan point is directly filtered. Otherwise, aggregate the current scan points to obtain the corrected scan point set points2; Based on the reference surface, the distance to each scan point in the corrected scan point set points2 is calculated to obtain the second distance d = {d1, d2, ..., d...}. n }; Calculate the mean μ and standard deviation for all the second distances. calculate; ； Where i is the scan point number; Outlier filtering is performed based on the mean and standard deviation combined with the 3x standard deviation rule to obtain the final set of scan points, points3. The z-coordinate value of the calibration point is obtained by averaging the z-coordinate values ​​of all scan points in the final scan point set points3.

4. The stitching calibration method based on 3D vision according to claim 1, characterized in that, The specific steps for stitching together all the re-captured depth images according to the camera transformation matrix and calculating the workpiece dimensions of the standard planar calibration part include: The standard planar calibration component is scanned again to obtain the corresponding depth image; Construct the original empty image based on the sum of the field of view of all depth cameras; The depth images are transformed and stitched together according to the camera transformation matrix, and then updated to the original empty image to obtain the final calibration depth map. The depth map of the final calibration part is inspected and calculated to obtain the workpiece size of the standard planar calibration part.

5. A 3D vision-based size detection system for implementing the method as described in any one of claims 1-4, characterized in that, include: The image acquisition module is used to acquire depth images of the workpiece using cameras of different depths; The stitching calibration module is used to calculate the transformation matrix between cameras based on the standard plane calibration parts for scanning at different depths, combined with the reference coordinate system. The image stitching module is used to stitch together all the workpiece depth images according to the camera transformation matrix to obtain a complete workpiece depth map; The dimension measurement module is used to detect the depth map of the complete workpiece at the edge and calculate the corresponding workpiece dimensions.

6. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-4.

7. A storage medium storing at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement a 3D vision-based stitching calibration method as described in any one of claims 1 to 4.

Images