Online full dimensional inspection method and system based on multi-view vision

Abstract

An online full dimensional inspection method and system based on multi-view vision. The method comprises: first determining correspondences between cameras in a camera array and holes to be measured, and carrying out three-dimensional reconstruction on all said holes by means of a triangulation method on the basis of hole center coordinates and the correspondences, to obtain three-dimensional coordinates of said holes in a workpiece coordinate system; and further converting three-dimensional coordinates to be measured from the workpiece coordinate system to a reconstructed measurement coordinate system, and determining dimension information of said holes in the measurement coordinate system. In this way, joint calibration is carried out by means of the camera array to determine the three-dimensional coordinates of said holes, dimension information of said holes is determined on the basis of the measurement coordinate system and the three-dimensional coordinates of said holes, the purpose of non-contact inspection can be achieved, the method is suitable for high-precision dimensional inspection of online inspection requirements on an assembly line, and the method has the characteristics of high efficiency, high precision and high stability.

Description

An online full-size detection method and system based on multi-vision

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on August 1, 2023, with application number 202310958248.8, and entitled “An online full-size detection method and system based on multi-vision”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the technical field of multi-vision dimension measurement, and in particular to an online full-size detection method and system based on multi-vision. Background Art

[0003] As an indispensable part of industrial manufacturing, dimension detection can complete the detection of many items such as position, flatness, contour, linear distance, aperture, etc. according to different detection requirements. At present, enterprises often use traditional measuring tools such as micrometers, vernier calipers, hand pieces, gauge blocks, etc. for dimension detection, or use three-coordinate measuring machines, flexible arms and other detection equipment for measurement of some large precision parts. During the measurement, the structural information of the hole is measured to determine whether the part to be tested is qualified. Among them, the former is mostly manual operation, which requires a lot of manpower and material resources for detection, and the detection accuracy is generally low, and it is impossible to perform high-precision measurements; although the latter has good detection accuracy and repeatability, it requires contact measurement of each measuring point, and the detection time is generally calculated in hours, which is difficult to meet the production rhythm of the production line, and it is impossible to perform online measurement.

[0004] It can be seen that in the existing technology, it is difficult to achieve the goal of achieving high-efficiency detection while meeting high-precision measurement.

[0005] Summary of the Invention

[0006] The present application provides an online full-size detection method and system based on multi-vision.

[0007] In order to achieve the above objectives, this application is implemented through the following technical solutions:

[0008] In a first aspect, the present application provides an online full-size detection method based on multi-viewing, comprising:

[0009] S1: Capture the image of the hole to be measured on the workpiece through the camera array;

[0010] S2: Detecting the coordinates of the hole center of the hole to be measured, wherein the coordinates of the hole center are the two-dimensional coordinates of the hole to be measured in each image;

[0011] S3: grouping and saving the hole center coordinates, and determining the corresponding relationship between each camera in the camera array and the hole to be measured according to the hole center coordinates;

[0012] S4: performing three-dimensional reconstruction on all the holes to be measured according to the correspondence and the triangulation method, and determining the three-dimensional coordinates of all the holes to be measured in the workpiece coordinate system according to the three-dimensional reconstruction results and the hole center coordinates;

[0013] S5: Convert each three-dimensional coordinate to be measured from the workpiece coordinate system to the reconstructed measurement coordinate system, and determine the size information of each hole to be measured in the measurement coordinate system.

[0014] Optionally, the S1 includes:

[0015] The camera array is controlled to collect images of the workpiece to be measured in a time-sharing manner under the set normal lighting and local lighting.

[0016] Optionally, the S2 includes:

[0017] According to the internal and external parameters of the cameras in the camera array and the CAD three-dimensional coordinates of the hole, a 2D template image of the hole to be measured under the camera array and the template hole center coordinates corresponding to the hole to be measured are generated by projection;

[0018] Obtain edge feature point information in the template image;

[0019] The ROI image of the hole to be measured in the image of the workpiece to be measured is cropped, and the edge extraction is performed on the ROI image to obtain an edge image. The gradient information of each pixel in the edge image is extracted, and the gradient information is subjected to sliding window search and matching with the edge feature point information in the template image to finally obtain the coordinates of the hole center in the ROI image.

[0020] Optionally, the correspondence between the cameras and the holes to be measured in S3 refers to which holes can be captured by each camera and which cameras can capture each hole.

[0021] Optionally, the S4 includes:

[0022] Determine optimized camera intrinsic parameters and camera extrinsic parameters;

[0023] According to the camera intrinsic parameters, the camera extrinsic parameters and the corresponding relationship, a triangulation method is used to calculate the three-dimensional coordinates of the hole to be measured, wherein the calculation formula satisfies the following relationship: X=K[R|T]x;

[0024] Where X represents the coordinate value of the hole center reconstructed after triangulation, K represents the camera intrinsic parameter, [R|T] represents the camera extrinsic parameter, and x represents the coordinate value of the hole center.

[0025] Optionally, before S5, the method further includes:

[0026] A reference plane and two reference holes of the workpiece to be measured are determined, and the reference plane and the two reference holes are aligned with the theoretical coordinates according to the drawing requirements of the actual workpiece to be measured to reconstruct the coordinate system and obtain a measurement coordinate system.

[0027] Optionally, the S5 includes:

[0028] Determine the position of the hole to be measured; the position satisfies the following relationship: position=2*sqrt((x1-x0) 2 +(y1-y0) 2 +(z1-z0) 2 );

[0029] Where, position represents the position, the reconstructed measured coordinates are (x1, y1, z1), the theoretical coordinates are (x0, y0, z0), and sqrt represents the square root expression;

[0030] The size information of the hole to be measured is determined according to the three-dimensional coordinates of the hole to be measured in the workpiece coordinate system and the position accuracy.

[0031] In a second aspect, the present application further provides an online full-scale inspection system based on multi-view vision, comprising: a darkroom, a camera array, and a main control unit, wherein the camera array is communicatively connected to the main control unit, the camera array comprising a plurality of first cameras, a plurality of second cameras, and a plurality of third cameras, wherein the plurality of first cameras are spaced apart from each other and arranged at the top of the darkroom, the plurality of second cameras are spaced apart from each other and arranged at the bottom of the darkroom, and the plurality of third cameras are spaced apart from each other and arranged on the side walls of the darkroom along the circumference of the darkroom:

[0032] The camera array is used to capture images of the holes to be measured on the workpiece to be measured;

[0033] The main control unit is used to detect the coordinates of the hole center of the hole to be measured, wherein the coordinates of the hole center are the coordinates of the hole center of the hole to be measured under the camera;

[0034] The hole center coordinates are grouped and saved, and the corresponding relationship between each camera in the camera array and the hole to be measured is determined according to the hole center coordinates;

[0035] Performing three-dimensional reconstruction on all holes to be measured according to the correspondence and the triangulation method, and determining the three-dimensional coordinates of all holes to be measured in the workpiece coordinate system according to the three-dimensional reconstruction results and the hole center coordinates;

[0036] The three-dimensional coordinates of the hole to be measured in the workpiece coordinate system are converted into dimension information in the reconstructed measurement coordinate system.

[0037] Optionally, a plurality of light sources are further included, and the plurality of light sources are arranged at intervals on the inner wall of the darkroom.

[0038] The details of one or more embodiments of the present application are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the present application will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0040] FIG1 is a flow chart of an online full-size detection method based on multi-viewing according to a preferred embodiment of the present application;

[0041] FIG2 is a schematic structural diagram of a workpiece to be tested according to a preferred embodiment of the present application;

[0042] FIG3 is a schematic diagram of the specific coordinates of the hole under the camera in a preferred embodiment of the present application;

[0043] FIG4 is a schematic diagram of a template image according to a preferred embodiment of the present application;

[0044] FIG5 is a schematic diagram of the coordinates of the template hole center in a preferred embodiment of the present application;

[0045] FIG6 is a schematic diagram of a calibration plate according to a preferred embodiment of the present application;

[0046] FIG7 is a schematic diagram of a calibration workpiece after a calibration plate is attached according to a preferred embodiment of the present application;

[0047] FIG8 is a triangulated schematic diagram of a preferred embodiment of the present application. DETAILED DESCRIPTION

[0048] The following is a clear and complete description of the technical solution of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0049] Unless otherwise defined, the technical or scientific terms used in this application shall have the usual meanings understood by persons of ordinary skill in the field to which this application belongs. The "first", "second" and similar words used in this application do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, "one" or "an" and similar words do not indicate a quantity limitation, but rather indicate the existence of at least one. "Connected" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the object being described changes, the relative positional relationship also changes accordingly.

[0050] Referring to FIG1 , this application provides an online full-size detection method based on multi-viewing, including:

[0051] S1: Capture the image of the hole to be measured on the workpiece through the camera array;

[0052] S2: Detecting the coordinates of the center of the hole to be measured, where the coordinates of the center of the hole to be measured are the two-dimensional coordinates of the hole to be measured in each image;

[0053] S3: Save the hole center coordinates in groups, and determine the corresponding relationship between each camera in the camera array and the hole to be measured according to the hole center coordinates;

[0054] S4: Performing three-dimensional reconstruction on all holes to be measured using a triangulation method according to the hole center coordinates and the corresponding relationship to obtain the three-dimensional coordinates of the holes to be measured in the workpiece coordinate system;

[0055] S5: Convert each three-dimensional coordinate to be measured from the workpiece coordinate system to the reconstructed measurement coordinate system, and determine the size information of each hole to be measured in the measurement coordinate system.

[0056] It should be understood that the above-mentioned online full-size detection method based on multi-vision can be applied to an online full-size detection system based on multi-vision, which includes: a darkroom, a camera array, and a main control unit. The camera array is communicatively connected to the main control unit. The camera array includes multiple first cameras, multiple second cameras and multiple third cameras. The multiple first cameras are arranged at the top of the darkroom at intervals, the multiple second cameras are arranged at the bottom of the darkroom at intervals, and the multiple third cameras are arranged on the side walls of the darkroom at intervals along the circumference of the darkroom.

[0057] A camera array is used to capture images of the holes to be measured on the workpiece. The camera array can be composed of CCD industrial cameras. The camera layout can be customized according to the workpiece shape and the measurement hole position.

[0058] The correspondence between cameras and holes to be measured in S3 refers to which holes each camera can capture and which cameras can capture each hole. For example, camera A can capture holes 1, 2, and 3, and hole 1 can be captured by both camera A and camera B. Therefore, there is a correspondence between camera A and holes 1, 2, and 3, and hole 1 has a correspondence with both camera A and camera B. Given this cross-correlation, the relationship between all cameras and holes can form a network structure. As shown in Figures 2-3, Figure 2 is a schematic diagram of the structure of a workpiece to be measured, and Figure 3 shows the specific coordinates of the holes as viewed by the cameras.

[0059] The above-mentioned online full-size detection method based on multi-vision determines the three-dimensional coordinates of the hole to be measured through joint calibration of the camera array, and determines the size information of the hole to be measured based on the measurement coordinate system and the three-dimensional coordinates of the hole to be measured. It can achieve the purpose of non-contact detection and is suitable for high-precision dimensional detection required for online detection on the assembly line. It has the characteristics of high efficiency, high precision and high stability.

[0060] Optionally, S1 includes:

[0061] The camera array is controlled to collect images of the workpiece to be measured in a time-sharing manner under the set normal lighting and local lighting.

[0062] In this optional embodiment, conventional lighting refers to the lighting from light sources set at intervals on the inner wall of the darkroom, and local lighting refers to the lighting generated by a light source set at a specific position on the inner wall of the darkroom according to actual needs. By performing time-sharing shooting under conventional lighting and local lighting, optimal imaging of each hole can be ensured.

[0063] Furthermore, after acquiring the image of the workpiece to be tested, each hole is marked with a ROI (Region of Interest) to facilitate narrowing the detection range during subsequent inspections and also to achieve the purpose of determining the hole name under the current camera.

[0064] Optionally, S2 includes:

[0065] According to the internal and external parameters of the cameras in the camera array and the CAD three-dimensional coordinates of the hole, a 2D template image of the hole to be measured under the camera array and the template hole center coordinates corresponding to the hole to be measured are generated by projection;

[0066] Obtain edge feature point information in the template image;

[0067] The ROI image of the hole to be measured in the image of the workpiece to be measured is cropped, and the edge of the ROI image is extracted to obtain an edge image. The gradient information of each pixel in the edge image is extracted, and the gradient information is searched and matched with the edge feature point information in the template image through a sliding window, and finally the coordinates of the hole center in the ROI image are obtained.

[0068] In this optional embodiment, the hole center coordinates are detected using a template matching method. As an alternative embodiment, in other feasible embodiments, circle center detection methods such as circle fitting and ellipse fitting can also be used to detect the hole center coordinates. This is only an example and not a limitation.

[0069] Specifically, the template matching process includes the following steps: template generation, template training, and template matching:

[0070] Template Generation: Based on the camera's internal and external parameters and the hole's CAD 3D coordinates, a 2D template image of the hole is generated through projection. In one achievable embodiment, the template image is a black background with white fill, as shown in Figures 4-5. In addition to the template image shown in Figure 4, each hole also has corresponding template hole center coordinates, as shown in Figure 5. In Figure 5, the template center coordinates are x: 2669.436279, y: 1018.707703. The CAD 3D coordinates of the hole can be determined based on the workpiece drawing.

[0071] Template training: After obtaining the template image, begin training the template. The purpose of template training is to obtain edge feature point information from the template image. This information includes information such as the coordinates of the feature points on the template image and the gradient direction. After training the template image, save the feature point information to a YAML file to reduce template matching time.

[0072] In actual size measurement, the saved template feature point information can be directly read, and then the correlation between the template and the real hole image can be calculated.

[0073] Template matching: Hole center coordinates are obtained through template matching. An ROI image of the hole to be measured is cropped from the captured image of the workpiece to be measured. This cropped ROI image is then smoothed and contrast-enhanced. To improve the effectiveness of feature point extraction, edge extraction is first performed on the ROI image of the hole to be measured. The gradient information of each pixel in the edge image is extracted, and finally, a sliding window search and matching is performed with the template features to obtain the hole center coordinates in the ROI image of the hole to be measured. After template matching is performed on all holes to be measured, their hole center coordinates are obtained and stored grouped by camera number for the reconstruction of the hole center's 3D coordinates in subsequent steps.

[0074] Optionally, S4 includes:

[0075] Determine optimized camera intrinsic parameters and camera extrinsic parameters;

[0076] According to the camera intrinsic parameters, camera extrinsic parameters and corresponding relationships, the three-dimensional coordinates of the hole to be measured are calculated using the triangulation method, where the calculation formula satisfies the following relationship: X = K[R|T]x;

[0077] Where X represents the coordinate value of the hole center reconstructed after triangulation, K represents the camera intrinsic parameter, [R|T] represents the camera extrinsic parameter, and x represents the coordinate value of the hole center.

[0078] In this optional implementation, the coordinate system of the optimized camera intrinsic parameters and camera extrinsic parameters will be in the same coordinate system as the workpiece CAD, which can facilitate the later reconstruction and measurement of the coordinate system. The steps for optimizing camera intrinsic parameters and camera extrinsic parameters are as follows:

[0079] Initial camera calibration: In a multi-camera vision system, to calibrate multiple cameras to the same coordinate system, sufficient calibration plates (AprilTags) are attached to the workpiece to be measured as calibration artifacts. The calibration plates are shown in Figure 6. Each calibration plate has its own ID: "Tag36h11" in Figure 6(a), "TagStandard41h12" in Figure 6(b), and "TagStandard52h13" in Figure 6(c). This ensures that each calibration plate detected by each camera has a unique ID. The calibration plates are attached, as shown in Figure 7. After attaching the plates, the calibration artifact is placed in a darkroom fixture for image acquisition. Images can be captured in a single set or in multiple sets after movement. Capturing multiple sets effectively increases the number of feature points and improves calibration accuracy. After image acquisition, all cameras are jointly calibrated using colmap. This allows all cameras to be calibrated to a single camera coordinate system, completing the calibration of all cameras. After the initial calibration of the camera, the camera intrinsic parameters, distortion coefficients and camera extrinsic parameters in the camera coordinate system can be obtained.

[0080] COLMAP is an open-source software package for multi-view stereo reconstruction, which can generate the geometry and texture of a 3D scene from multiple images. COLMAP can be used to estimate the camera pose and calibrate the camera parameters in a multi-view vision system.

[0081] The camera coordinate system is converted to the workpiece coordinate system: Before the conversion, it is necessary to determine the CAD model of the workpiece to be converted. More specifically, it is necessary to determine the coordinate value of each hole in the CAD. It is best if this coordinate value is measured using a three-coordinate measuring instrument and used as the true value (GroundTruth) of this application. After obtaining the initial camera parameters of each camera in the multi-camera system, the 2D coordinates of each hole under each camera are detected, and the 3D coordinates of the hole in the camera coordinate system are reconstructed by triangulation. Then, matrix operations are used to calculate the conversion relationship between the 3D coordinates in the camera coordinate system and the coordinates of the workpiece hole obtained by the three coordinates, and the rotation matrix (R), translation vector (T), and scaling factor (S) between the two coordinate systems are obtained. Finally, these parameters are applied to each camera to complete the conversion from the camera coordinate system to the workpiece coordinate system, which is convenient for measurement in subsequent steps.

[0082] Use holes to optimize camera parameters: In the previous step, the 3D observation coordinates and three-coordinate measurement values ​​of each hole can be obtained, as shown in Figure 8. Through the bundle adjustment method (BA), the three-coordinate measurement values ​​can be used to constrain the 3D observation coordinates of each hole, so that the observation value of each hole is closer to the true value (three-coordinate measurement value). Because this constraint actually acts on the camera's internal and external parameters, more accurate camera internal and external parameters can be obtained in this way, thus completing the hole-based camera parameter optimization.

[0083] Furthermore, the use of triangulation methods presupposes the knowledge of the intrinsic and extrinsic parameters (also known as the pose) of each camera. The coordinates of the center of each hole (measurement point) as viewed by each camera are known. Since the intrinsic and extrinsic parameters of the cameras are already known, and the 2D coordinates of each hole as viewed by each camera are also obtained, triangulation can be used to perform a 3D reconstruction of each hole, obtaining the coordinates of the hole to be measured in the workpiece coordinate system.

[0084] Specifically, assume the coordinate value of the hole center reconstructed after triangulation is X, its image coordinate in the camera imaging is x, the camera's intrinsic parameters are K, the rotation matrix is ​​R, and the translation vector is T, where the combination of R and T [R|T] represents the camera's extrinsic parameters. The following formula can be used to determine the relationship between the hole's image coordinates and the three-dimensional coordinates, and the three-dimensional coordinates of the hole can be obtained based on at least two point pairs, where the at least two point pairs are the image coordinates of the same hole under two cameras. X = K[R|T]x;

[0085] This formula can be used to reconstruct the 3D coordinates of each hole based on the hole center coordinates under multiple cameras, which can be used for system establishment and measurement in subsequent steps.

[0086] In the preceding steps, the 3D coordinates of each hole to be measured on the workpiece are obtained, and these coordinates are in the workpiece coordinate system. However, these coordinates are not aligned with the reference holes as required by the measurement. Therefore, the coordinates need to be reconstructed and the system needs to be established according to the actual drawing requirements of the workpiece to be inspected. In addition, in actual workpieces, different parts of the same workpiece may be measured under different references depending on the assembly relationship. This re-establishment operation can meet the measurement requirements of multiple references.

[0087] Optionally, before S5, the above method further includes:

[0088] Determine a reference plane and two reference holes of the workpiece to be measured, and align the reference plane and two reference holes with the theoretical coordinates according to the drawing requirements of the actual workpiece to be measured to reconstruct the coordinate system and obtain the measurement coordinate system.

[0089] In this optional embodiment, the steps of establishing the system are as follows:

[0090] Since the camera parameters are in the workpiece coordinate system, reconstructing the 3D coordinate system is a rigid body transformation process. The scaling factor (S), rotation matrix (R), and translation vector (T) required for the rigid body transformation are obtained by aligning the reconstructed coordinates with the theoretical coordinates.

[0091] Determine a datum plane and two datum holes on the workpiece to be measured. When establishing the system, the reconstructed datum coordinates must be strictly aligned with the nominal CAD datum coordinates. Assuming a datum plane A, a first datum hole B, and a second datum hole C, the reconstructed coordinates must maintain the datum plane normal, the xy coordinates of the first datum hole B, and the x (or y, depending on the workpiece) coordinate of the second datum hole C after alignment. Both the x and y coordinates are theoretical coordinates.

[0092] By building a system based on the actual drawing requirements of the workpiece to be inspected, the reconstructed 3D coordinates of the holes to be measured can be converted to different reference coordinate systems according to actual measurement requirements, thus meeting the multiple reference measurement requirements in actual inspection needs. By obtaining the dimensional information of all holes under multiple references, the measurement results are more consistent with the actual measurement requirements and the measurement accuracy is improved.

[0093] Optionally, S5 includes:

[0094] Determine the position of the hole to be measured; the position satisfies the following relationship: position=2*sqrt((x1-x0) 2 +(y1-y0) 2 +(z1-z0) 2 );

[0095] Where, position represents the position, the reconstructed measured coordinates are (x1, y1, z1), the theoretical coordinates are (x0, y0, z0), and sqrt represents the square root expression;

[0096] The size information of the hole to be measured is determined according to the three-dimensional coordinates and position of the hole to be measured in the workpiece coordinate system.

[0097] As an alternative implementation, in other feasible implementations, the above method can also be used to measure flatness, contour, linear distance, etc.

[0098] The flatness is determined as follows:

[0099] Assume that the plane feature points are a1, a2, ..., an, the plane fitted by the feature points is Ax + By + Cz + D = 0, and the distances from the feature points to the plane are d1, d2, ..., dn. Then the flatness can be expressed as: flatness = max(d1, d2, ..., dn) - min(d1, d2, ..., dn);

[0100] Among them, max and min represent the maximum and minimum values ​​of the distance from the feature point to the plane, respectively. A, B, and C represent the components of the plane normal vector on the X-axis, Y-axis, and Z-axis, respectively. D is a constant term, which represents the distance between the plane and the origin.

[0101] The method for determining the contour degree is as follows:

[0102] Assume that the plane feature points are a1, a2, ..., an, the theoretical coordinates of the plane are X, the theoretical normal is (1, 0, 0), and the plane fitted by the feature points is Ax + By + Cz + D = 0. Since the theoretical plane normal is (1, 0, 0), the contour of the plane only needs to calculate the deviation in the x direction. It can be set as: d1x = a1 x -X x ,d2x=a2 x -X x ,...,dnx=an x -X x ;

[0103] Where d1x, d2x, and dnx represent the deviations in the X direction from each plane feature point to the theoretical feature point, respectively. The profile can be expressed as: profile = 2*max(d1x, d2x, ..., dnx);

[0104] The linear distance is determined as follows:

[0105] Assume the coordinates of the two existing holes are (x1, y1, z1) and (x2, y2, z2). The linear distance can be calculated by the Euclidean distance between the two points. distance = sqrt((x2-x1) 2 +(y2-y1) 2 +(z2-z1) 2 );

[0106] In this embodiment, the linear distance refers to the distance between two holes.

[0107] In one example, the above-mentioned online full-size inspection method based on multi-vision can complete the inspection of 1,000 or more measuring points within 150 seconds, and the accuracy of each inspection can reach advanced levels. For example, the performance of position accuracy 0.3 can reach more than 98%, flatness 0.15 can reach more than 98%, and contour accuracy 0.3 can reach more than 95%.

[0108] The present application also provides an online full-scale inspection system based on multi-vision, comprising: a darkroom, a camera array, and a main control unit, wherein the camera array is communicatively connected to the main control unit, the camera array comprising a plurality of first cameras, a plurality of second cameras, and a plurality of third cameras, wherein the plurality of first cameras are spaced apart from each other and arranged at the top of the darkroom, the plurality of second cameras are spaced apart from each other and arranged at the bottom of the darkroom, and the plurality of third cameras are spaced apart from each other and arranged on the side walls of the darkroom along the circumference of the darkroom:

[0109] The camera array is used to obtain an image of the hole to be measured on the workpiece to be measured;

[0110] The main control unit is used to detect the coordinates of the hole center of the hole to be measured, wherein the coordinates of the hole center are the coordinates of the hole center of the hole to be measured under the camera;

[0111] The hole center coordinates are grouped and saved, and the corresponding relationship between each camera in the camera array and the hole to be measured is determined according to the hole center coordinates;

[0112] Perform three-dimensional reconstruction of all holes to be measured based on the corresponding relationship and triangulation method, and determine the three-dimensional coordinates of all holes to be measured in the workpiece coordinate system based on the three-dimensional reconstruction results and the coordinates of the hole centers;

[0113] Convert the three-dimensional coordinates of the hole to be measured in the workpiece coordinate system into the dimensional information in the measurement coordinate system.

[0114] Optionally, a plurality of light sources are further included, and the plurality of light sources are arranged at intervals on the inner wall of the darkroom.

[0115] In this optional embodiment, specifically, the multiple light sources may include multiple first light sources and multiple second light sources, the multiple first light sources are all arranged in the upper part of the darkroom and facing the top wall of the darkroom, the multiple first light sources are arranged at intervals along the circumference of the darkroom, the multiple second light sources are all arranged in the lower part of the darkroom and facing the bottom wall of the darkroom, and the multiple second light sources are arranged at intervals along the circumference of the darkroom.

[0116] Specifically, the main control unit can be used to control different light sources to illuminate at different times, thereby achieving different lighting and photography for each hole, ensuring optimal imaging of each hole.

[0117] The above-mentioned online full-size detection system based on multi-vision can implement various embodiments of the above-mentioned online full-size detection method based on multi-vision and achieve the same beneficial effects, which will not be described in detail here.

[0118] The above describes in detail the preferred embodiments of the present application. It should be understood that those skilled in the art can make numerous modifications and variations based on the concepts of the present application without inventive effort. Therefore, any technical solutions that can be derived by those skilled in the art based on the concepts of the present application through logical analysis, reasoning, or limited experimentation on the basis of the prior art should be within the scope of protection defined by the claims.

Claims

1. An online full-size detection method based on multi-eye vision, characterized in that: include: S1: Capture the image of the hole to be measured on the workpiece to be measured through the camera array; S2: Detecting the coordinates of the hole center of the hole to be measured, wherein the coordinates of the hole center are the two-dimensional coordinates of the hole to be measured in each image; S3: saving the hole center coordinates in groups, and determining the corresponding relationship between each camera in the camera array and the hole to be measured according to the hole center coordinates; S4: Performing three-dimensional reconstruction on all holes to be measured using a triangulation method according to the hole center coordinates and the corresponding relationship to obtain the three-dimensional coordinates of the holes to be measured in the workpiece coordinate system; S5: converting each three-dimensional coordinate to be measured from the workpiece coordinate system to the reconstructed measurement coordinate system, and determining the size information of each hole to be measured in the measurement coordinate system.

2. The online full-size detection method based on multi-eye vision according to claim 1 is characterized in that: The S1 includes: The camera array is controlled to collect images of the workpiece to be measured in time-sharing under the set normal illumination and local illumination.

3. The online full-size detection method based on multi-eye vision according to claim 2 is characterized in that: The S2 includes: According to the internal and external parameters of the cameras in the camera array and the CAD three-dimensional coordinates of the hole, a 2D template image of the hole to be measured under the camera array and the template hole center coordinates corresponding to the hole to be measured are generated by projection; Obtain edge feature point information in the template image; The ROI image of the hole to be measured in the image of the workpiece to be measured is cropped, and the edge of the ROI image is extracted to obtain an edge image. The gradient information of each pixel is extracted from the edge image, and the gradient information is searched and matched with the edge feature point information in the template image through a sliding window, and finally the coordinates of the hole center in the ROI image are obtained.

4. The online full-size detection method based on multi-eye vision according to claim 1 is characterized in that: The correspondence between the cameras and the holes to be measured in S3 refers to which holes can be captured by each camera and which cameras can capture each hole.

5. The online full-size detection method based on multi-eye vision according to claim 1 is characterized in that: The S4 includes: Determine optimized camera intrinsic parameters and camera extrinsic parameters; According to the camera intrinsic parameters, the camera extrinsic parameters and the corresponding relationship, the three-dimensional coordinates of the hole to be measured are calculated by using a triangulation method, wherein the calculation formula satisfies the following relationship: X = K[R|T]x; Where X represents the coordinate value of the hole center reconstructed after triangulation, K represents the camera intrinsic parameter, [R|T] represents the camera extrinsic parameter, and x represents the coordinate value of the hole center.

6. The online full-size detection method based on multi-eye vision according to claim 1 is characterized in that: Before S5, the method further includes: A reference plane and two reference holes of the workpiece to be measured are determined, and the reference plane and the two reference holes are aligned with the theoretical coordinates according to the drawing requirements of the actual workpiece to be measured to reconstruct the coordinate system and obtain the measurement coordinate system.

7. The online full-size detection method based on multi-eye vision according to claim 6 is characterized in that: The S5 includes: Determine the position of the hole to be measured; wherein the position satisfies the following relationship: position=2*sqrt((x1-x0) 2 +(y1-y0) 2 +(z1-z0) 2 ); Where, position represents the position, the reconstructed measured coordinates are (x1, y1, z1), the theoretical coordinates are (x0, y0, z0), and sqrt represents the square root expression; The size information of the hole to be measured is determined according to the three-dimensional coordinates of the hole to be measured in the workpiece coordinate system and the position degree.

8. An online full-size inspection system based on multi-eye vision, characterized in that: include: darkroom, A camera array and a main control unit, wherein the camera array is communicatively connected to the main control unit, the camera array comprises a plurality of first cameras, a plurality of second cameras and a plurality of third cameras, wherein the plurality of first cameras are arranged at a top of the darkroom at intervals from each other, the plurality of second cameras are arranged at a bottom of the darkroom at intervals from each other, and the plurality of third cameras are arranged on a side wall of the darkroom at intervals from each other along a circumferential direction of the darkroom: The camera array is used to collect images of the holes to be measured on the workpiece to be measured; The main control unit is used to detect the hole center coordinates of the hole to be measured, wherein the hole center coordinates are the hole center coordinates of the hole to be measured under the camera; The hole center coordinates are grouped and saved, and the corresponding relationship between each camera in the camera array and the hole to be measured is determined according to the hole center coordinates; Performing three-dimensional reconstruction on all holes to be measured according to the corresponding relationship and the triangulation method, and determining the three-dimensional coordinates of all holes to be measured in the workpiece coordinate system according to the three-dimensional reconstruction results and the hole center coordinates; The three-dimensional coordinates of the hole to be measured in the workpiece coordinate system are converted into dimension information in the reconstructed measurement coordinate system.

9. The multi-vision-based online full-size inspection system according to claim 8, characterized in that: It also includes a plurality of light sources, which are arranged at intervals on the inner wall of the darkroom.

Images