Workpiece hole and point position degree flexible detection equipment based on multi-view stereoscopic vision

Abstract

The invention discloses workpiece hole and point position degree flexible detection equipment based on multi-view stereoscopic vision, belongs to the technical field of computer vision and intelligent detection, and particularly relates to a workpiece position degree flexible detection method and equipment technology based on multi-view stereoscopic vision. The problems that in the prior art, a detection system cannot achieve flexible detection of workpiece hole and point position degrees, compatibility is poor, the function is single, and the requirements for high precision and high efficiency cannot be met at the same time are solved. The apparatus comprises: a measurement frame; at least one robot; a plurality of cameras; a plurality of light source modules; a laser; a plurality of correction targets; and a calculation control unit. The workpiece hole and point position degree flexible detection equipment based on multi-view stereoscopic vision is suitable for part quality detection scenes in the automobile manufacturing field, and is especially suitable for hole and point position degree online detection application requiring high precision, high flexibility and high efficiency.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent inspection technology, and in particular to a flexible detection method and equipment technology for workpiece position based on multi-view stereo vision. Background Technology

[0002] In the quality control system of manufacturing, the positional accuracy of holes and point features on a workpiece directly determines the assembly quality and production efficiency of the final product. Especially in fields such as automobile manufacturing, there are extremely strict requirements for the dimensional deviations of key components. In such cases, high-precision equipment is needed to detect deviations that are undetectable by the human eye.

[0003] Traditional manual inspection methods can no longer meet the demands for high-precision and high-efficiency inspection. To overcome the limitations of manual inspection, the industry has gradually developed various automated inspection technologies, but these traditional methods still have significant drawbacks.

[0004] Technical bottlenecks of traditional detection methods: Currently, widely used automated inspection methods mainly include fixture inspection, coordinate measuring machine (CMM) inspection, and binocular structured light inspection, but each has its own drawbacks: Inspection of gauges: Although it can achieve full inspection, it can only make qualitative judgments (such as inspection of go and no-go gauges), cannot quantify specific deviation values, cannot provide quantitative data for in-depth analysis and continuous improvement, cannot perform SPC (process capability statistics) analysis, and has poor compatibility due to the large differences among operators. Coordinate measuring machine (CMM) inspection: It has high measurement accuracy, but the inspection speed is long, the equipment cost is high, it can only be carried out by sampling inspection, and there is a risk of missed inspection. Binocular structured light inspection: This method performs non-contact measurement through point cloud reconstruction, but it is susceptible to the reflective properties of the workpiece surface; missing point clouds directly lead to a decrease in measurement accuracy. Furthermore, in binocular structured light inspection, the system is easily affected by the accuracy of the robot and moving parts, requiring high repeatability (typically 0.2mm). This increases the complexity of equipment debugging and maintenance, resulting in higher subsequent repair and maintenance costs. Simultaneously, as the number of workpiece inspection items increases, the number of points that need to be traversed also increases, significantly extending cycle time and impacting inspection efficiency.

[0005] These traditional methods cannot simultaneously meet the comprehensive requirements of high precision, full inspection, rapid response, and multi-model adaptation. Summary of the Invention

[0006] This invention proposes a flexible detection device for workpiece hole and point position accuracy based on multi-view stereo vision, which solves the problems of existing detection systems that cannot achieve flexible detection of workpiece hole and point position accuracy, have poor compatibility, limited functionality, and cannot simultaneously meet the requirements of high precision and high efficiency.

[0007] The workpiece hole and point position flexibility detection device based on multi-view stereo vision described in this invention includes: A measuring frame is used to fix and mount other components and define a detection area; the detection area includes a detection position and a detection surface, wherein the detection position is a given position where the workpiece is placed for detection, and the detection surface is a plane on the workpiece surface where the point to be detected is located; At least one robot is used to grip the workpiece and position it at the detection position; Multiple cameras are mounted on the measuring frame at different angles to form a multi-view vision system for acquiring workpiece images from different angles. Multiple light source modules are arranged in sections on the measuring frame to provide illumination for different areas of the hole to be measured on the workpiece; by controlling the light source modules, the light source zoning control is realized to avoid the hole to be measured being affected by diffuse reflection from other areas of the light source; A laser, mounted above the detection surface, is used to project a laser speckle pattern onto the workpiece surface for point position accuracy detection. Multiple correction targets are fixedly mounted on the measurement frame and located within the camera's field of view; The computing control unit is electrically connected to the plurality of cameras, the plurality of light source modules, at least one robot, and the laser; the computing control unit is configured to: The system stores the reference coordinates, camera parameters, feature templates, and the nominal coordinates of the hole positions to be measured associated with the feature templates in the workpiece coordinate system, which are established through the system initialization method. Store and execute the steps of the workpiece hole and point position measurement method, and / or the steps of the system initialization method. Furthermore, a preferred embodiment is provided in which the plurality of cameras are divided into a plurality of camera groups, and the same camera can be assigned to different camera groups; the light source module is an LED light source, and each light source module is assigned to one camera group.

[0008] Furthermore, a preferred embodiment is provided in which the correction target is fixedly mounted on a non-movable part of the measurement frame for dynamic error correction; the correction target is composed of a high-contrast pattern to ensure that there are at least two correction targets in each camera's field of view.

[0009] This invention also proposes a method for detecting the positional accuracy of workpiece holes and points based on multi-view stereo vision. Using any of the above embodiments, the method employs a flexible detection device for the positional accuracy of workpiece holes and points based on multi-view stereo vision. The method is based on reference coordinates, camera parameters, feature templates, and the nominal coordinates of the hole to be measured in the workpiece coordinate system, all established through a system initialization method. The method includes the following online detection steps: S21: Workpiece hole image acquisition and two-dimensional coordinate positioning: Acquire the acquired images of the workpiece holes to be measured; For the acquired images of the workpiece holes to be measured, use feature templates for matching and positioning to obtain the two-dimensional center coordinates of each hole on a single image; S22: Calculation of three-dimensional coordinates of workpiece hole: For each hole to be measured, using camera parameters, the two-dimensional center coordinates of the hole to be measured obtained by at least two cameras are triangulated to calculate the three-dimensional coordinates of the hole to be measured in the equipment coordinate system. S23: Workpiece Hole Coordinate System Mapping and Position Evaluation: Taking the three-dimensional coordinates of multiple holes to be tested in the equipment coordinate system and their nominal coordinates in the workpiece coordinate system as input, the transformation parameters from the equipment coordinate system to the workpiece coordinate system are calculated through an optimization algorithm. Then, the three-dimensional coordinates of all holes to be tested are transformed to the workpiece coordinate system to obtain the actual coordinates. By comparing the actual coordinates of each hole to be tested with its nominal coordinates, it is determined whether its position deviation is within the preset tolerance range, and the position detection result of the hole to be tested is output. S24: Workpiece point position measurement: Acquire speckle images of the workpiece points to be measured; perform digital image correlation analysis on the acquired speckle images of the workpiece points to be measured, calculate the speckle displacement field, and convert the displacement field into a height field through a calibration formula to obtain the position measurement results of the points to be measured.

[0010] Furthermore, in a preferred embodiment, step S21, the matching and localization using feature templates, includes: Load the feature template, which is a mask image with gradient field information; By calculating the correlation between the feature template and the gradient field of the target image, and combining the particle swarm optimization algorithm or the feature pyramid algorithm for optimization search, the two-dimensional center coordinates of the hole to be tested on the image are determined.

[0011] Furthermore, a preferred embodiment is provided, in step S21: For the same camera under the same light source module, the same hole to be measured is photographed repeatedly N times, where N≥2; the arithmetic mean of the two-dimensional center coordinates obtained from the N calculations is taken as the final two-dimensional center coordinates of the hole to be measured. Furthermore, in a preferred embodiment, after step S21 and before step S22, a dynamic error correction step is also included: Acquire images of the corrected target, identify the corrected target in the image, and locate its two-dimensional center coordinates; The camera pose transformation parameters are calculated by comparing the deviation between the two-dimensional center coordinates of the corrected target in the current image and the reference coordinates established in the system initialization method. The two-dimensional center coordinates of the hole position to be measured obtained in step S21 are corrected using the pose transformation parameters.

[0012] Furthermore, a preferred embodiment is provided, wherein in step S22, the triangulation is implemented using Newton's iteration method, including: Randomly initialize a three-dimensional point coordinate system to represent the hole position to be measured in the device coordinate system; The coordinates of the 3D point are projected onto the image coordinate system of each camera using the camera parameters to obtain the projected 2D point. The coordinates of the three-dimensional point are iteratively optimized to minimize the reprojection error between its projected two-dimensional point and the two-dimensional center coordinates of the corresponding hole position obtained in step S21. Finally, the optimized three-dimensional point coordinates are used as the three-dimensional coordinates of the hole to be tested in the device coordinate system. Furthermore, a preferred embodiment is provided, wherein in step S23, the optimization algorithm is Newton's method, and the transformation parameters include a rotation matrix and a translation vector. The present invention also proposes a system initialization method for implementing the workpiece hole and point position measurement method based on multi-view stereo vision as described in any of the above embodiments, the method comprising the following steps: S11: Camera configuration and installation: Arrange multiple cameras on the measuring frame to ensure that each hole to be measured on the workpiece can be observed by at least two cameras; S12: Allocation of light source and camera group: Configure different light source modules for different areas of the workpiece to be measured, and allocate a camera group to each light source module. The same camera can be assigned to different camera groups. S13: Laser installation: Install the laser vertically above the detection position, ensuring that the beam is perpendicular to the surface of the workpiece being measured; S14: Correction target fabrication and installation: At least two correction targets are fixedly installed on the immovable parts of the measurement frame, and at least two correction targets are observed within the field of view of each camera; S15: Feature template creation: For each hole to be tested and each correction target under each camera field of view, draw its edge contour to generate the corresponding feature template, associate the feature template of the hole to be tested with its nominal coordinates in the workpiece coordinate system, and record the position of the correction target in the image as the reference coordinates. S16: Camera Calibration: Using a coded dot calibration board, the camera parameters of each camera are calculated by having all cameras capture multiple images of the calibration board from different angles. The camera parameters include the intrinsic parameters of each camera and the extrinsic parameters of all cameras relative to a common device coordinate system, thus completing the camera calibration.

[0013] The present invention has the following beneficial effects: 1. The workpiece hole and point positional flexibility detection device based on multi-view stereo vision described in this invention uses a robot to hold the workpiece instead of the traditional fixed pallet, and combines visual guidance and path planning technology to achieve rapid workpiece positioning and changeover, so that the detection process time is controlled within 10 to 30 seconds, which greatly improves production efficiency and reduces equipment compatibility costs.

[0014] 2. The flexible workpiece hole and point position measurement device based on multi-view stereo vision of the present invention integrates hole and point position measurement functions, and utilizes multi-source time-division control and DIC analysis technology to achieve comprehensive measurement of hole and point features of automotive workpieces, thus solving the problem of single function in traditional systems.

[0015] 3. The workpiece hole and point position accuracy detection method based on multi-view stereo vision of the present invention, through the precise calibration and dynamic error correction technology of multi-view camera array, combined with gradient field feature template matching and laser speckle analysis algorithm, ensures that the hole position accuracy detection accuracy reaches ±0.1 mm and the repeatability detection accuracy reaches ±0.02 mm, effectively improving the measurement stability and reliability.

[0016] 4. The workpiece hole and point position accuracy detection method based on multi-view stereo vision of the present invention significantly reduces the repetitive positioning requirements (compared to binocular structured light detection, this method only requires within 2mm) by using a multi-view vision system and zoned light source control, reducing the dependence on the accuracy of robots and moving parts, thereby reducing equipment costs and maintenance complexity; at the same time, by using a light source group and camera synchronous image triggering method, it can cover all detection items on a surface, regardless of the number of detection items, avoiding the point traversal and cycle time extension caused by the increase of detection items, and realizing highly efficient flexible detection.

[0017] The workpiece hole and point positional flexibility detection device based on multi-view stereo vision described in this invention is suitable for parts quality inspection scenarios in the automotive manufacturing field, and is especially suitable for online detection applications of hole and point positional accuracy that require high precision, high flexibility and high efficiency. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1In one embodiment of the present invention, a three-dimensional isometric view of the three-dimensional structure of a workpiece hole and point positional flexibility detection device based on multi-view stereo vision is provided. Figure 2 This is a top view of the three-dimensional structure of a flexible detection device for workpiece hole and point position based on multi-view stereo vision, as described in one embodiment of the present invention. Detailed Implementation

[0020] To make the technical solutions and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail and completely below with reference to the accompanying drawings. The various embodiments described below are only some preferred embodiments of the present invention, and not all of them; the various embodiments described below are intended to explain the present invention and should not be construed as limiting the present invention; reasonable combinations of the technical features defined in the various embodiments of the present invention, as well as all other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort, are all within the scope of protection of the present invention.

[0021] Implementation Method 1: A flexible detection device for workpiece hole and point position based on multi-view stereo vision, the device comprising: A measuring frame is used to fix and mount other components and define a detection area; the detection area includes a detection position and a detection surface, wherein the detection position is a given position where the workpiece is placed for detection, and the detection surface is a plane on the workpiece surface where the point to be detected is located; At least one robot is used to grip the workpiece and position it at the detection position; Multiple cameras are mounted on the measuring frame at different angles to form a multi-view vision system for acquiring workpiece images from different angles. Multiple light source modules are arranged in sections on the measuring frame to provide illumination for different areas of the hole to be measured on the workpiece; by controlling the light source modules, the light source zoning control is realized to avoid the hole to be measured being affected by diffuse reflection from other areas of the light source; A laser, mounted above the detection surface, is used to project a laser speckle pattern onto the workpiece surface for point position accuracy detection. Multiple correction targets are fixedly mounted on the measurement frame and located within the camera's field of view; The computing control unit is electrically connected to the plurality of cameras, the plurality of light source modules, at least one robot, and the laser; the computing control unit is configured to: The system stores the reference coordinates, camera parameters, feature templates, and the nominal coordinates of the hole positions to be measured associated with the feature templates in the workpiece coordinate system, which are established through the system initialization method. Store and execute the steps of the workpiece hole and point position measurement method, and / or the steps of the system initialization method. In this embodiment, the device achieves (high) flexibility inspection by using a robot to clamp the workpiece, which is applicable to different types of workpieces (simplifies model switching), replaces the traditional fixed pallet, and improves compatibility.

[0022] Meanwhile, using robots to hold workpieces for measurement reduces costs.

[0023] In this embodiment, the measuring frame (or equipment frame) provides rigid support, and the camera and light source modules are divided into zones to ensure that each workpiece's measured hole and measured point are covered by at least two cameras.

[0024] In this embodiment, the measuring frame is the base structure of the equipment, made of steel, and is in the form of a gantry or bracket, including a crossbeam, columns and a base.

[0025] The frame size is adjustable (width 2-5 meters, height 1-3 meters) to ensure stability and adaptability (suitable for workpieces of different sizes).

[0026] The frame includes beams, columns, and a base, ensuring that it will not deform due to vibration or external forces during the testing process.

[0027] The frame has mounting holes for fixing the camera, light source and correction target.

[0028] The detection area is a physical space defined by the measurement frame, wherein: The detection position is located at the center point of the frame, and its coordinates are determined through calibration. The workpiece is placed here by the robot system for detection. The detection position is a fixed point on the measuring frame, aligned with the frame coordinate system to ensure detection consistency. The detection position maintains a preset distance from the camera and light source module (500-1000mm for the camera, 500-800mm for the light source) to ensure field of view coverage and uniform illumination; The detection surface is the plane on the workpiece surface where the point to be detected is located. When the workpiece is in the detection position, its upper surface is the detection surface. The design of the detection area ensures that the camera and light source can fully cover the workpiece without blind spots.

[0029] In this embodiment, multiple robots can work together (e.g., robot 1 is responsible for picking up the workpiece and moving it to the inspection position, while robot 2 is responsible for positioning the workpiece at the inspection position).

[0030] In this embodiment, the robot adopts a six-axis articulated robotic arm, which achieves vision-guided grasping through hand-eye calibration, and has a high-precision gripper that can adapt to workpieces of different shapes and sizes.

[0031] The robot includes a first robot and a second robot.

[0032] The first robot is equipped with a 3D camera and a gripper for visual guidance to grasp the workpiece to a given input position (or detection position); the first robot scans the workpiece with the 3D camera to generate a point cloud, and grasps it after deep learning localization; wherein, after the first robot acquires images and builds the point cloud with the 3D camera, it performs preprocessing operations on the point cloud data, including but not limited to point cloud denoising, filtering and registration, to improve the accuracy and robustness of subsequent recognition and localization; The second robot is used for precision handling, moving the workpiece from the input position to the detection position; the second robot performs path planning to avoid collisions and ensure that the workpiece is stably placed at the detection position.

[0033] This dual-robot design enables flexible loading and unloading, adapting to workpiece changes.

[0034] The robot's repeatability is ±0.05mm.

[0035] In this embodiment, the cameras are arranged in a surrounding array on the beams and columns of the measurement frame, for example, in a 4x4 grid layout.

[0036] The camera is a high-resolution industrial camera (e.g., 20 megapixels), with a frame rate of 30fps, and supports synchronous triggering; ensuring that each hole and point to be measured is observed simultaneously by at least two cameras.

[0037] The camera is pointed at the detection position, covering a 360-degree field of view with no blind spots.

[0038] In this embodiment, the light source module is of the LED type, and the brightness is adjusted by PWM and controlled by the calculation and control unit in a time-sharing manner to avoid interference.

[0039] In this implementation, the partitioning logic is a physical spatial division, with each partition corresponding to a set of cameras and light sources. The computing control unit controls (or triggers) the light source modules in a time-division manner to avoid lighting interference (such as avoiding cross-illumination). For example, the top side hole of the workpiece is illuminated by the top light source, triggering the corresponding camera group to take pictures simultaneously.

[0040] In this embodiment, the laser is a semiconductor laser with a wavelength of 650nm and an output power of 100mW.

[0041] The laser is mounted vertically above the inspection surface at a height of 50-1000cm (adjustable range), ensuring that the emitted beam is perpendicular to the workpiece surface being measured (i.e., the inspection surface). The inspection surface must be flat or have a known curvature to facilitate point position accuracy detection and measurement.

[0042] The laser generates a speckle pattern through diffractive optical elements. The speckle pattern is used for DIC analysis to achieve point position accuracy detection.

[0043] In this embodiment, the computing control unit integrates all algorithms to achieve fully automatic detection.

[0044] In this embodiment, the computing control unit includes an industrial computer and a GPU accelerator card (such as NVIDIA RTX3080), communicates with the robot system via the EtherCAT protocol, controls the light source module and laser via the GPIO interface, and connects to the camera via the USB3.0 interface.

[0045] The computing control unit is used for real-time processing of images and data. Unit storage system parameters, such as camera intrinsic parameters, extrinsic parameters, feature templates, nominal coordinates, and reference coordinates.

[0046] The method for detecting the positional accuracy of workpiece holes and points specifically includes the following steps: image acquisition and two-dimensional coordinate positioning, three-dimensional coordinate calculation, coordinate system mapping and hole positional accuracy evaluation, and point positional accuracy detection. This method utilizes the principle of multi-view vision to achieve automatic detection of holes and points with an accuracy of ±0.1mm.

[0047] In this embodiment, the detection device is used to detect the position of the holes and points to be tested on the workpiece.

[0048] In this embodiment, the hole to be measured (or the hole to be measured) refers to the hole features on the workpiece, such as circular holes, square holes, or threaded holes, used for mechanical assembly. These holes typically have standard dimensions (e.g., diameter 2-10mm) and high positional accuracy requirements (tolerance ±0.1mm).

[0049] In this embodiment, the point to be measured (or the test point) refers to a point-like feature on the surface of the workpiece, such as a protrusion, a concave point, or a marker point, used for positional reference, and is relatively small in size (e.g., 1-5 mm in diameter). The testing equipment needs to process the positional accuracy of both holes and points simultaneously.

[0050] In this embodiment, the aforementioned testing equipment enables efficient automated testing. The workpiece hole and point position accuracy testing method implemented using this equipment takes between 10 and 30 seconds from the start of the measurement process to the end of the test. The testing accuracy can reach ±0.1 mm, and the repeatability accuracy can reach ±0.02 mm, fully meeting the high-standard testing requirements for hole and point position accuracy of precision parts such as automotive workpieces.

[0051] In this embodiment, compared with traditional binocular structured light detection, the use of multi-view stereo vision and zoned light source design overcomes the problems of susceptibility to the accuracy of robots and moving parts, high repeatability requirements (0.2mm), and high maintenance costs in existing technologies. The detection device only needs to repeat within 2mm to operate stably, and it covers the entire detection surface synchronously with a light source group and camera, regardless of the number of detection items. This avoids the increased cycle time caused by point traversal when the number of detection items increases, thus achieving high-precision, high-flexibility, and high-efficiency detection.

[0052] Implementation Method 2: The multiple cameras are divided into multiple camera groups, and the same camera can be assigned to different camera groups; the light source module is an LED light source, and each light source module is assigned to one camera group.

[0053] In this embodiment, the allocation of camera clusters and light source modules is optimized through software settings to improve resource utilization.

[0054] In this embodiment, the light source module is divided based on the workpiece geometry (e.g., the workpiece geometry is predefined based on the workpiece CAD model), that is, the light source module is dynamically allocated according to different test hole areas on the workpiece: For example, the top area uses a top light module (LED surface light source), and the side area uses a side light module (LED strip light source). Each zone corresponds to a specific part of the workpiece (such as a densely packed area of ​​holes).

[0055] In this embodiment, each light source module corresponds to a subset of cameras (camera cluster), and is controlled in a time-division manner by a computational control unit to avoid light interference. The computational control unit switches the light source modules (and camera cluster) via a relay array to ensure that only one set of light source modules illuminates the workpiece at each detection moment.

[0056] Implementation method 3: The correction target is fixedly installed on the immovable part of the measurement frame for dynamic error correction; the correction target is composed of high-contrast patterns to ensure that there are at least two correction targets in the field of view of each camera.

[0057] In this embodiment, the correction target uses a black and white concentric circle pattern, printed on a ceramic substrate and mounted on the frame column. The pattern design meets the sub-pixel positioning requirements and is used for coordinate correction after calculating camera displacement.

[0058] Implementation Method 4: A workpiece hole and point position measurement method based on multi-view stereo vision, using the flexible workpiece hole and point position measurement device based on multi-view stereo vision as described in any of the above implementation methods. The method is based on the reference coordinates, camera parameters, feature template, and the nominal coordinates of the hole position to be measured associated with the feature template in the workpiece coordinate system, established through a system initialization method. The method includes the following online detection steps: S21: Workpiece hole image acquisition and two-dimensional coordinate positioning: Acquire the acquired images of the workpiece holes to be measured; For the acquired images of the workpiece holes to be measured, use feature templates for matching and positioning to obtain the two-dimensional center coordinates of each hole on a single image; S22: Calculation of three-dimensional coordinates of workpiece hole: For each hole to be measured, using camera parameters, the two-dimensional center coordinates of the hole to be measured obtained by at least two cameras are triangulated to calculate the three-dimensional coordinates of the hole to be measured in the equipment coordinate system. S23: Workpiece Hole Coordinate System Mapping and Position Evaluation: Taking the three-dimensional coordinates of multiple holes to be tested in the equipment coordinate system and their nominal coordinates in the workpiece coordinate system as input, the transformation parameters from the equipment coordinate system to the workpiece coordinate system are calculated through an optimization algorithm. Then, the three-dimensional coordinates of all holes to be tested are transformed to the workpiece coordinate system to obtain the actual coordinates. By comparing the actual coordinates of each hole to be tested with its nominal coordinates, it is determined whether its position deviation is within the preset tolerance range, and the position detection result of the hole to be tested is output. S24: Workpiece point position measurement: Acquire speckle images of the workpiece points to be measured; perform digital image correlation analysis on the acquired speckle images of the workpiece points to be measured, calculate the speckle displacement field, and convert the displacement field into a height field through a calibration formula to obtain the position measurement results of the points to be measured.

[0059] In this embodiment, the method achieves flexible positioning through a robot. The method is fully automated, with a detection time of 10-30 seconds and an accuracy of ±0.1mm.

[0060] In this embodiment, the image of the hole to be measured on the workpiece is acquired using the following method: The robot picks up the workpiece and moves it to the detection position to ensure automatic workpiece positioning; The system controls the light source module corresponding to the current detection area to light up and triggers the corresponding camera group to capture images of the workpiece's test hole.

[0061] In this embodiment, the speckle image of the measured point on the workpiece is acquired using the following method: The laser is controlled to project a speckle pattern onto the detection surface, and the camera is triggered to capture the speckle image.

[0062] In this embodiment, the positional accuracy detection results include an assessment of the deviation between the hole position and the point position to be measured.

[0063] In this embodiment, digital image correlation analysis, also known as DIC analysis, is used.

[0064] In this embodiment, the speckle displacement field is calculated. And through calibration formula Convert the displacement field into a height field Based on the positional deviation of the height field calculation point, the positional detection result of the point to be measured is obtained, where This is the magnification factor. The laser incident angle is denoted as .

[0065] laser incident angle It is usually 90°.

[0066] In this embodiment, the DIC algorithm is used to calculate the full-field displacement field. .

[0067] DIC analysis uses the Ncorr open-source library with a window size of 21×21 pixels and a step size of 5 pixels.

[0068] In this embodiment, the step of "calculating the transformation parameters from the device coordinate system to the workpiece coordinate system through an optimization algorithm" is specifically implemented in the following way: The objective function of the optimization algorithm is to minimize the overall transformation error between the three-dimensional coordinates of all test holes involved in the fitting in the device coordinate system and their nominal coordinates in the workpiece coordinate system.

[0069] Specifically, let the three-dimensional coordinates of the hole to be measured in the device coordinate system be P_device, and the nominal coordinates in the workpiece coordinate system be P_nominal. The goal is to find an optimal rigid transformation, consisting of a rotation matrix R and a translation vector T, such that the difference between the transformed coordinates P_workpiece = R·P_device + T and the nominal coordinates P_nominal is minimized. In other words, the optimization problem is:

[0070] Where i is the hole index participating in the coordinate system fitting.

[0071] Implementation Method 5: In step S21, the matching and localization using feature templates includes: Load the feature template, which is a mask image with gradient field information; By calculating the correlation between the feature template and the gradient field of the target image, and combining the particle swarm optimization algorithm or the feature pyramid algorithm for optimization search, the two-dimensional center coordinates of the hole to be tested on the image are determined.

[0072] In this embodiment, the "mask image with gradient field information" is generated using the following specific method: the gradient field of the workpiece hole template image is calculated using the Sobel operator; the feature template image is the result of multiplying the non-zero pixel region of the workpiece hole binary mask image with the gradient values ​​of the corresponding hole image in the x and y directions. This template has better robustness to changes in illumination and contrast.

[0073] In this embodiment, the calculation of gradient field correlation includes the following similarity measurement calculation process: the pixels of the feature template image are geometrically transformed using the remap interpolation method to make them precisely aligned with the pixels of the target image (i.e., the image captured by the current camera); the target image is traversed by a sliding window to calculate the sum of the gradient field dot products of the feature template and the target image at corresponding positions, thereby quantifying the similarity between the two images and providing a benchmark for subsequent optimization search (particle swarm optimization or feature pyramid algorithm).

[0074] Implementation method 6: In step S21: For the same camera under the same light source module, the same hole to be measured is photographed repeatedly N times, where N≥2; the arithmetic mean of the two-dimensional center coordinates obtained from the N calculations is taken as the final two-dimensional center coordinates of the hole to be measured. In this embodiment, N=5, meaning that for the same camera under the same light source module, the same hole to be measured is photographed repeatedly at least 5 times to improve accuracy.

[0075] Implementation Method 7: After step S21 and before step S22, a dynamic error correction step is also included: Acquire images of the corrected target, identify the corrected target in the image, and locate its two-dimensional center coordinates; The camera pose transformation parameters are calculated by comparing the deviation between the two-dimensional center coordinates of the corrected target in the current image and the reference coordinates established in the system initialization method. The two-dimensional center coordinates of the hole position to be measured obtained in step S21 are corrected using the pose transformation parameters.

[0076] In this embodiment, a dynamic error correction step is used to eliminate observation errors caused by camera displacement.

[0077] In this embodiment, the purpose of the dynamic error correction step is to transform the coordinates in the image captured after camera displacement back to the original camera coordinate system established during the camera calibration process, where the camera positional relationships remained unchanged. Specifically, the pose transformation parameters calculated by correcting the target are essentially the inverse transformation of the camera's movement from its original position to its current position. Using these parameters to correct the aperture coordinates is equivalent to "moving" the observation data back to the original, accurate camera coordinate system, thereby ensuring the accuracy of subsequent triangulation and 3D reconstruction.

[0078] Implementation Method 8: In step S22, the triangulation is implemented using Newton's iteration method, including: Randomly initialize a three-dimensional point coordinate system to represent the hole position to be measured in the device coordinate system; The coordinates of the 3D point are projected onto the image coordinate system of each camera using the camera parameters to obtain the projected 2D point. The coordinates of the three-dimensional point are iteratively optimized to minimize the reprojection error between its projected two-dimensional point and the two-dimensional center coordinates of the corresponding hole position obtained in step S21. Finally, the optimized three-dimensional point coordinates are used as the three-dimensional coordinates of the hole to be tested in the device coordinate system. In this embodiment, the triangulation process targets multiple two-dimensional center coordinates of the same hole location obtained from different cameras (possibly belonging to different light source modules) observing from different perspectives. Through iterative optimization, the final solution is the unique three-dimensional coordinates of the hole location in a common device coordinate system.

[0079] In this embodiment, the step of "projecting the coordinates of the three-dimensional point to the image coordinate system of each camera using camera parameters" is specifically achieved through the following coordinate transformation chain: First, using the extrinsic parameters obtained from camera calibration (i.e., the rotation matrix R and translation vector T), the randomly initialized 3D point coordinates P_w in the device coordinate system are transformed to the camera coordinate system, resulting in coordinates P_c. The mathematical relationship is: P_c = R·P_w + T.

[0080] Then, using the intrinsic parameter matrix K obtained from camera calibration, the 3D point P_c in the camera coordinate system is projected onto the 2D image coordinate system to obtain the theoretical projected 2D point coordinates p. The mathematical relationship is: p = K·P_c (here, a simplified representation of homogeneous coordinate transformation).

[0081] The "reprojection error" refers to the Euclidean distance between the theoretical projection point p calculated above and the actual coordinates of the two-dimensional center point obtained through image matching in step S21.

[0082] The "iterative optimization" process specifically employs Newton's method, which continuously adjusts the coordinate values ​​of the initial 3D point P_w to minimize the sum of reprojection errors on all camera views that can observe the hole, thereby ultimately solving for the most accurate 3D coordinates of the hole in the device coordinate system.

[0083] Implementation method 9: In step S23, the optimization algorithm is Newton's method, and the transformation parameters include rotation matrix and translation vector. In this embodiment, the iterative process of the optimization algorithm solves for the optimal rotation matrix and translation vector by continuously reducing the overall transformation error between the three-dimensional coordinates of the hole to be measured in the device coordinate system and its nominal coordinates in the workpiece coordinate system.

[0084] In this embodiment, Newton's method is used as the optimization algorithm for iterative solution to obtain the optimal rotation matrix R and translation vector T. After obtaining the optimal R and T, the three-dimensional coordinates of all the holes to be measured in the device coordinate system can be uniformly transformed to the workpiece coordinate system using the formula P_final = R·P_device + T to obtain the final actual coordinates that can be used for comparison.

[0085] Implementation Method 10: A system initialization method for implementing the workpiece hole and point position detection method based on multi-view stereo vision as described in any of the above embodiments, the method comprising the following steps: S11: Camera configuration and installation: Arrange multiple cameras on the measuring frame to ensure that each hole to be measured on the workpiece can be observed by at least two cameras; S12: Allocation of light source and camera group: Configure different light source modules for different areas of the workpiece to be measured, and allocate a camera group to each light source module. The same camera can be assigned to different camera groups. S13: Laser installation: Install the laser vertically above the detection position, ensuring that the beam is perpendicular to the surface of the workpiece being measured; S14: Correction target fabrication and installation: At least two correction targets are fixedly installed on the immovable parts of the measurement frame, and at least two correction targets are observed within the field of view of each camera; S15: Feature template creation: For each hole to be tested and each correction target under each camera field of view, draw its edge contour to generate the corresponding feature template, associate the feature template of the hole to be tested with its nominal coordinates in the workpiece coordinate system, and record the position of the correction target in the image as the reference coordinates. S16: Camera Calibration: Using a coded dot calibration board, the camera parameters of each camera are calculated by having all cameras capture multiple images of the calibration board from different angles. The camera parameters include the intrinsic parameters of each camera and the extrinsic parameters of all cameras relative to a common device coordinate system, thus completing the camera calibration.

[0086] In this embodiment, the system initialization method is used to initialize (or build) a flexible detection device for workpiece hole and point position based on multi-view stereo vision, and to perform camera calibration.

[0087] In this embodiment, the correction target consists of a square iron sheet and a circular label pasted on it. In this embodiment, the specific manufacturing and installation method of the modified target is as follows: The correction target is made using a combination of a square iron sheet and a circular label. Specifically, a high-contrast circular label (e.g., a solid black circle on a white background) is pasted into the center of the square iron sheet. The square iron sheet base provides sufficient structural strength and stability.

[0088] During installation, the prepared target is securely installed on structural columns or other components inside the measuring device that will not move due to equipment operation or environmental vibration, using methods such as bolts or welding.

[0089] The installation layout must ensure that for each camera, at least two or more such fixed targets can be clearly observed in its field of view, so as to provide a reliable reference point for subsequent dynamic error correction.

[0090] In this embodiment, the "arranging multiple cameras on the measurement frame" is specifically achieved in the following way: First, the overall spatial range is determined based on the final positioning and placement of the workpiece in the inspection station. Then, the position and angle of each camera are finely adjusted according to the specific distribution of the holes in the workpiece to be measured.

[0091] The core principle of camera placement is to meet the observation requirements of "multi-view vision," meaning that each hole to be measured on the workpiece must be simultaneously observed by at least two cameras in different positions for subsequent 3D reconstruction. Simultaneously, the planning will fully utilize the field of view of each camera, enabling one camera to cover and capture multiple hole features, thereby optimizing the number of cameras and reducing costs while ensuring measurement accuracy.

[0092] In this embodiment, after the cameras are installed and positioned, on-site adjustment of optical parameters is required. Specifically, this includes: starting the cameras, adjusting the aperture size of each camera to a suitable value and fixing it; then, observing the dynamic images captured in real time by the cameras, selecting the center aperture of the image to be observed, and finely rotating the camera lens focal length until the aperture appears as a clear outline in the image, thereby ensuring the best image acquisition quality.

[0093] In this embodiment, the "camera cluster" refers to the multiple cameras being divided into multiple camera clusters, and the same camera can be assigned to different camera clusters.

[0094] In this embodiment, the zoning of the light source modules and the allocation of the camera group are specifically targeted. For example, for the hole to be measured on the upper side of the workpiece, the corresponding light source module located at the top layer of the measurement frame and the camera group belonging to that module are configured and triggered to perform illumination and image acquisition.

[0095] In this embodiment, the "allocating a camera group to each light source module" is achieved through the following specific control strategy: In the system software, a corresponding list of enableable camera serial numbers is set for each preset light source module. When it is necessary to detect a specific aperture area, the calculation and control unit performs the following operations: Light up the corresponding light source module in that area.

[0096] Based on the preset list (camera number list), all cameras in the camera group under this module are started (or triggered) to shoot synchronously, while ensuring that cameras not in this list do not work, so as to avoid light interference between different light source modules.

[0097] After the area is detected, the current light source module will be turned off.

[0098] This allocation method allows the same physical camera to be repeatedly triggered and used at different detection times (i.e., when different light source modules are lit) based on its configuration in different lists, thereby efficiently covering the detection tasks of multiple areas of the workpiece and maximizing the utilization of hardware resources.

[0099] In this embodiment, the creation and quality verification of the feature template are achieved through the following specific methods: The "drawing its edge contour" process involves, during system initialization, using a mouse or stylus in the software interface of the computational control unit to precisely trace a closed contour along the visible edges of each test hole and each correction target on the reference image captured by the camera. The software then automatically generates the corresponding binarized feature template based on this contour.

[0100] Immediately after template generation, quality verification is performed. The system uses the newly generated template to conduct multiple small offset matching tests on the original reference image used to create the template (i.e., "matching and positioning on the reference image"). The software calculates the coordinates of the center point of each match and observes the fluctuation range of these coordinate values. When the fluctuation range is no greater than 0.3 pixels (a preset threshold), the template is deemed to be of acceptable quality and can be used for subsequent high-precision online detection; if the fluctuation exceeds the tolerance, it needs to be redrawn until the requirements are met.

[0101] Meanwhile, in the software interface, for each successfully created hole feature template, its nominal coordinates in the workpiece coordinate system can be manually entered or associated from the database to complete the data binding.

[0102] In this embodiment, the quality verification after template generation is as follows: In the system initialization method described above, in step S15, when generating the corresponding feature template, the quality of the feature template is confirmed to be qualified by verifying that the fluctuation of its center position is not greater than a preset threshold when the feature template is matched and located on the reference image. In this embodiment, the preset threshold is 0.3 pixels.

[0103] In this embodiment, the reference image refers to one or more initial, high-quality reference images used during the system initialization phase (i.e., in the system initialization method) when drawing edge contours for each test hole and correction target to generate a feature template.

[0104] Specifically: Timing of shooting: This image was taken when the camera and light source were all adjusted and the workpiece was in a standard inspection position. At this time, the lighting conditions and the position of the workpiece were both ideal.

[0105] Core function: This image serves as the "base map" for creating feature templates. Operators precisely trace the outlines on this image. Therefore, it naturally becomes the benchmark or reference standard for subsequent template quality verification.

[0106] Verification process: After generating the template, the system immediately uses this newly generated template to match the "baseline image" that was just used to create it. Since the template was created based on this image, the matching result should be perfect. The system will perform multiple small offset matching tests to calculate the fluctuation range of the matching center. If this fluctuation range (i.e., the matching error) is less than a preset threshold (e.g., 0.3 pixels), it proves that the template is of high quality and has good stability.

[0107] In summary, the "baseline image" specifically refers to the original, standard reference image used in the initialization phase for template creation and template quality verification. Its existence ensures that the comparison benchmark for all subsequent online detections is consistent and accurate.

[0108] In this embodiment, the specific operations and requirements of the camera calibration process are as follows: During calibration, the operator moves the coded dot calibration plate within the common field of view of the measurement frame. When taking pictures at each placement position, it must be ensured that at least two cameras in any group of cameras with a common field of view can simultaneously capture the same area of ​​the calibration plate. This is a prerequisite for calculating the relative positions (extrinsic parameters) between the cameras.

[0109] Meanwhile, for each camera, more than twenty valid calibration board images need to be captured from different angles and positions to ensure that the calibration algorithm can fully fit the camera's internal parameters (such as focal length and distortion) and external parameters.

[0110] Because it uses a coded dot calibration board, its surface contains unique coded patterns. This feature allows the system to uniquely identify the absolute position of a part of the calibration board in a single image (i.e., incomplete), based on the local pattern, thus enabling high-precision parameter calculation. This significantly reduces the difficulty of capturing images and improves the robustness and efficiency of the calibration process.

[0111] In this embodiment, the camera parameter calculation process includes specific calculation procedures and optimization measures.

[0112] During parameter calculation, the intrinsic parameters of each camera (such as focal length, distortion coefficients, etc.) are calculated first, and then the extrinsic parameters (i.e. rotation matrix and translation vector) between cameras are calculated based on the intrinsic parameter results.

[0113] After obtaining the preliminary extrinsic parameter results, the system will sort the acquired calibration board images according to the magnitude of the reprojection error and manually delete the images with the highest errors in the top 10.

[0114] During the deletion process, special attention should be paid to the correlation between cameras. For example, if deleting an image for a certain camera (such as cam0) results in insufficient correlation with other cameras (such as fewer than 2 calibration board images in the common field of view), this deletion operation may cause calibration failure, and recalibration or adjustment of the deletion strategy is required.

[0115] Repeat this parameter calculation and image optimization process until the average reprojection error of all cameras drops below the preset convergence threshold. At this point, the intrinsic and extrinsic parameters are considered to have been successfully calculated.

[0116] In this embodiment, the dot array on the coded dot calibration board contains a specific coding pattern, so that even if the calibration board image is incomplete, it can still be uniquely identified and solved.

[0117] In this embodiment, the specific coding pattern is composed of missing or different sizes of dots at specific locations, which allows the computer to quickly identify the orientation of a local image without relying on the physical boundaries of the calibration board, thereby improving the robustness and efficiency of the calibration process.

[0118] In this embodiment, the online detection phase is initiated according to the following process upon first run: After system initialization is completed, before the first workpiece hole position measurement, the camera calibration data (camera parameters), generated feature template data, recorded correction target reference coordinates, and hole position nominal coordinate data package obtained during the initialization phase must be uploaded or configured into the online inspection system (to execute the online inspection step) in the software interface of the measurement system as the reference dataset for the inspection task.

[0119] Subsequently, based on the specific requirements of this inspection, in the software interface, the key workpiece holes that need to be used to calculate the transformation parameters from the device coordinate system to the workpiece coordinate system are selected and set to "Participate in Coordinate System Fitting". Typically, these holes should be characteristic holes with high nominal coordinate accuracy and uniform distribution in the workpiece coordinate system. Other holes that only require final inspection but do not participate in coordinate system fitting do not need to have this option selected. When executing step S23, the system will only use the hole data set to "Participate in Fitting" to calculate the optimal transformation parameters, but will use these parameters to transform the coordinates of all holes to be tested.

[0120] After completing the above settings, the automatic detection process can be started. The system will execute steps S21 to S23 of the online detection process and finally output the precise coordinates and position evaluation results of all test holes.

[0121] Implementation Method 11: The present invention will now be described in detail using a typical workpiece hole and point position measurement scenario as an example: I. Hardware System Setup (Flexible Testing Equipment): This system employs a flexible design for gripping workpieces with a robot, replacing the traditional fixed measuring table, to achieve high-precision and rapid inspection of automotive parts of different models. The system includes the arrangement of the measuring frame, robot system, camera, light source module, and laser; all components work in coordination through a computational control unit.

[0122] Measurement Frames and Robots: A rigid measuring frame (width adjustable from 2-5 meters, height adjustable from 1-3 meters) is used, with pre-set mounting holes for the camera / light source. The center of the frame defines the detection position, whose coordinates are determined through calibration and aligned with the frame coordinate system.

[0123] Robotic systems: Robot 1: Equipped with a 3D camera and gripper, located in the material area, it uses hand-eye calibration to achieve vision-guided grasping and places the workpiece in the inspection position (accuracy ±0.05mm). Robot 2: Located in the inspection area, it picks up the workpiece from the inspection position, precisely transports it to the inspection position after path planning, and clamps and fixes it in place. Layout features: The camera and light source on the frame are all aligned with the detection position held by the robot 2, forming a surrounding observation layout centered on the workpiece.

[0124] Visual acquisition system deployment: Camera array: 16 20-megapixel industrial cameras (4×4 grid layout), fixed in a surrounding array on the frame beams / columns. Each camera ensures a frame rate of 30fps, supports synchronous triggering, and guarantees that each hole / point under test is observed by at least 2 cameras simultaneously.

[0125] Lighting system: Zoned controllable LED light source modules (top light / side light modules), each module corresponding to a camera group. Brightness is adjusted via PWM, triggered in a time-sharing manner by a computational control unit to avoid light interference.

[0126] Laser: A semiconductor laser (wavelength 650nm, power 100mW) is vertically mounted 50-1000cm above the detection surface to project a speckle pattern for point position measurement.

[0127] Correction targets: Ceramic-based black and white concentric circle targets, fixed to the immovable parts of the frame, with at least 2 targets in each camera's field of view, used for dynamic error correction.

[0128] Computational control unit integration: The computing control unit includes an industrial computer (such as the Advantech IPC-610H, equipped with an Intel i7 processor and 32GB of RAM) and a GPU accelerator card (such as an NVIDIA RTX 3080). The unit communicates with the robot system via the EtherCAT protocol, controls the light source module and laser via GPIO interfaces, and connects to the camera via a USB 3.0 interface. The unit stores all system parameters, such as camera intrinsic and extrinsic parameters, feature templates, nominal coordinates, and reference coordinates.

[0129] II. System Initialization (Calibration and Template Preparation): System initialization is the preparatory stage before inspection, ensuring all parameters are accurate and providing a foundation for online inspection. This stage is performed when the equipment is used for the first time or when the workpiece model is changed, and includes camera calibration, feature template generation, and baseline recording.

[0130] Camera calibration: Camera calibration is performed using a coded dot calibration board (such as an ArUco marker board, 500mm x 500mm). The calibration board is placed at the detection position, and all cameras are driven to capture at least 20 images from different angles. The calibration board is moved at multiple angles within the detection position space, ensuring that at least two cameras in each group with a shared field of view capture the same area of ​​the calibration board. The camera intrinsic parameters (focal length, distortion coefficients) and extrinsic parameters (rotation matrix, translation vector) are calculated by the computation control unit. The calibration accuracy requires a reprojection error of less than 0.1 pixels. Optimization: Based on the extrinsic parameter results, the top 10 images with the highest error are manually deleted, and this process is repeated iteratively until the reprojection error is below the convergence threshold. Output: The intrinsic and extrinsic parameter sets for all cameras, establishing an accurate device coordinate system. After calibration, the parameters are stored in the computation control unit.

[0131] Feature template generation: Acquire a reference image of the workpiece (e.g., a high-quality image of a car door), and calculate the gradient field using the Sobel operator. Generate a binary mask: extract the hole contour using Canny edge detection to form a black and white image (white represents the hole area). Multiply the non-zero regions of the mask with the gradient values ​​to form a robust template. Template quality verification: when matching on the reference image, the center position fluctuation is no greater than 0.3 pixels. The template is stored in the computation control unit.

[0132] Benchmark Record: Record the reference coordinates of the correction target (mounted on the equipment frame column). The correction target is a black and white concentric circle pattern, and its center coordinates are located by matching a feature template. Simultaneously, import the nominal coordinates (ideal positions of holes and points) from the workpiece CAD file. All reference data is stored in the calculation and control unit.

[0133] III. Online Testing Process: The online inspection process is fully automated and coordinated by a computing control unit. The following steps are performed sequentially and take 20-30 seconds, taking an automobile door workpiece as an example.

[0134] Workpiece loading and positioning (robot collaboration): Robot 1 scans the material area with a 3D camera, generates a point cloud, and identifies and locates the workpiece through deep learning. After hand-eye calibration and coordinate transformation, it grabs the workpiece to the inspection position. Robot 2 picks up the workpiece from the inspection position and precisely positions it to the inspection position after collision avoidance path planning.

[0135] Regional image acquisition (multi-light source collaboration): Light source and camera group allocation: The computational control unit controls the light source modules according to the workpiece area partitioning. For example, the top light source module is lit first, triggering the top camera subset (cameras 1-4) to capture images synchronously; after completion, the top light source is turned off, the side light source modules are lit, and the side camera subset (cameras 5-8) is triggered to capture images. Time-division control avoids light interference, with each area capturing images for approximately 100ms. Five images are captured for each hole under the same light source, and the average value is taken.

[0136] Hole position measurement (3D reconstruction algorithm): Two-dimensional coordinate extraction: The acquired image is transmitted to the computing control unit via a USB 3.0 interface. Matching and localization using a pre-generated feature template: A normalized cross-correlation algorithm is applied, combined with particle swarm optimization search, to calculate the gradient dot product of the template and the target image. The position of the maximum value is the two-dimensional center coordinate of the hole. Matching accuracy reaches 0.1 pixels.

[0137] Dynamic error correction: Identify the target in the image and locate its two-dimensional center coordinates. Compare the deviation between the current coordinates and the reference coordinates, calculate the camera pose transformation parameters, and apply inverse transformation to correct all aperture coordinates, eliminating camera displacement or vibration errors.

[0138] 3D coordinate calculation: For each aperture, the 3D coordinates are calculated using triangulation based on the 2D coordinates and camera parameters from at least two cameras. Newton's iterative method is used to minimize the reprojection error, and the optimization is iteratively performed until convergence (error less than 0.01 mm). The calculation accuracy is ±0.1 mm. If multiple cameras observe the same aperture, a weighted average coordinate fusion is used.

[0139] Coordinate system mapping and hole position evaluation: Key hole positions (high precision, uniform distribution) are selected for fitting. Using the 3D coordinates of multiple hole positions in the equipment coordinate system and their nominal coordinates in the workpiece coordinate system as input, an optimization algorithm calculates the transformation parameters (rotation matrix R and translation vector T) from the equipment coordinate system to the workpiece coordinate system. The 3D coordinates of all hole positions are transformed to the workpiece coordinate system to obtain the actual coordinates. The actual coordinates and nominal coordinates of each hole position are compared, and the Euclidean distance deviation is calculated. If the deviation exceeds a preset tolerance (e.g., ±0.1mm), it is marked as unqualified.

[0140] Point position measurement (laser speckle analysis): The laser is controlled to project a laser speckle pattern onto the workpiece inspection surface, and the speckle image is captured synchronously by a camera.

[0141] The speckle image was analyzed using a digital image correlation (DIC) algorithm (such as the Ncorr open-source library). The full-field displacement field was calculated by comparing the speckle image with a reference template image. . The DIC window size is 21×21 pixels, and the step size is 5 pixels.

[0142] Apply the calibration formula Convert the displacement field to the height field , where = 10 (system magnification), = 30° (laser incident angle). Obtain the actual height coordinates of the points.

[0143] Compare the actual height with the nominal height to judge the position tolerance deviation of the points. The tolerance is set to ±0.05 mm.

[0144] Result output and report generation: The calculation control unit integrates all data to generate an inspection report. The report includes: the actual coordinates, nominal coordinates, deviation values, and qualified status of each hole position; the actual height, nominal height, deviation values, and qualified status of each point; visualization charts such as deviation heat maps.

[0145] The report is output in PDF and CSV formats and sent to the MES through the network interface or displayed on the HMI. If the deviation exceeds the tolerance, the system triggers an alarm or automatically classifies the workpieces.

[0146] Robot 2 moves the workpiece out of the inspection position. The entire inspection cycle is 10 - 30 seconds, and the repeatability accuracy is ±0.02 mm.

[0147] IV. Typical scenario example: Automobile door detection: 1. Workpiece description: Left connecting floor, size 293 mm x 236 mm x 4.5 mm, with 11 mounting holes (diameter 14.5 mm) and 46 surface bumps (diameter 2 mm).

[0148] Inspection process: After the hardware system is built, the system is initialized: camera calibration, template generation, and reference recording.

[0149] Online inspection: The robot visually guides and grasps the left connecting floor and transports it to the inspection position; image acquisition and two-dimensional coordinate positioning; three-dimensional coordinate calculation; coordinate system mapping and hole position tolerance evaluation; point position tolerance detection.

[0150] Result: The deviation of all hole positions is within ±0.1 mm, the point deviation is within ±0.025 mm, and the workpiece is qualified. The inspection takes 13 seconds.

[0151] Effect: Fully automated process, high precision, good repeatability, suitable for the automobile parts manufacturing assembly line.

[0152] 2. Workpiece description: Energy absorption box, with dimensions of 289mm x 162mm x 100mm, having 9 mounting holes (diameter 9.2mm) and 64 surface bumps (diameter 2mm).

[0153] Detection process: After the hardware system is built, the system is initialized: camera calibration, template generation, and reference recording.

[0154] Online detection: The robot visually guides and grabs the energy absorption box and transports it to the detection position; image acquisition and two-dimensional coordinate positioning; three-dimensional coordinate calculation; coordinate system mapping and hole position degree evaluation; point position degree detection.

[0155] Result: The deviation of all hole positions is within ±0.1mm, and the point deviation is within ±0.03mm. The workpiece is qualified. The detection takes 15 seconds.

[0156] Effect: Fully automated process, high precision, good repeatability, suitable for the automotive parts production line.

[0157] It should be noted that the rise and limitations of the existing multi-camera vision measurement system: To break through the limitations of traditional methods, multi-camera stereo vision technology has gradually become an important development direction in industrial inspection. This system synchronously acquires images from different angles through multiple cameras, and uses three-dimensional reconstruction algorithms to obtain the spatial coordinate information of the workpiece, achieving non-contact high-precision measurement. Compared with traditional methods, the multi-camera vision system has obvious advantages in terms of automation level and detection efficiency. <00​​​​​​​​​​​​​​​This invention replaces the traditional fixed pallet with a robot-held workpiece, achieving greater flexibility in the inspection system. This design allows the equipment to adapt to different workpiece models without requiring redesigning and processing the pallet, significantly improving system compatibility.

[0161] This invention features an adjustable measuring frame (width 2-5 meters, height 1-3 meters), ensuring the device can adapt to workpieces of different sizes. The frame has mounting holes for securing the camera, light source, and correction target; this modular design further enhances the device's flexibility.

[0162] This invention solves the problem of limited functionality: This invention achieves integrated detection of hole and point position accuracy through a specific detection method, overcoming the problem of limited functionality in existing technologies.

[0163] The detection device of this invention includes multiple cameras and lasers. The cameras are used for hole position measurement, and the lasers are used for point position measurement. This hardware configuration enables the device to perform two different types of detection tasks simultaneously.

[0164] This invention solves the problem of insufficient system stability: This invention achieves dynamic error correction by modifying the target. The specific method involves acquiring an image of the modified target, identifying the target center coordinates, calculating camera pose transformation parameters, and correcting the measurement results. This effectively eliminates observation errors caused by camera displacement. (Without target correction, the detection result differs from the coordinate measuring machine by ±0.2-±0.3mm; with target correction, the detection result differs from the coordinate measuring machine by ±0.1mm).

[0165] The present invention includes a complete camera calibration process in the system initialization method, and ensures measurement accuracy through a coded dot calibration plate and an optimized algorithm.

[0166] The technical solution of the present invention achieves significant technical progress through the following means: Improve inspection efficiency: By using robots for automatic loading and unloading and automated inspection processes, the inspection time can be reduced to 10-30 seconds; Guarantee detection accuracy: Hole position accuracy reaches ±0.1mm, and repeatability accuracy is ±0.02mm; Achieving flexible production: One set of equipment can adapt to the testing needs of different types of workpieces, greatly improving equipment utilization and production efficiency; Ensuring system stability: Multiple safeguards (rigid framework, dynamic correction, optimization algorithms, etc.) ensure long-term reliability.

[0167] The various technical solutions of this invention work together to form a complete technical system, effectively solving the various technical problems pointed out in the background art, and have significant technical progress and industrial application value.

[0168] The above description of several specific embodiments further details the technical solution provided by the present invention in order to highlight the advantages and benefits of the technical solution provided by the present invention. However, the above-described specific embodiments are not intended to limit the present invention. Any reasonable modifications and improvements to the present invention, reasonable combinations of embodiments, and equivalent substitutions based on the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A flexible inspection apparatus for workpiece hole and point location based on multi-view stereo vision, characterized in that, The device includes: A measuring frame is used to fix and mount other components and define a detection area; the detection area includes a detection position and a detection surface, wherein the detection position is a given position where the workpiece is placed for detection, and the detection surface is a plane on the workpiece surface where the point to be detected is located; At least one robot is used to grip the workpiece and position it at the detection position; Multiple cameras are mounted on the measuring frame at different angles to form a multi-view vision system for acquiring workpiece images from different angles. Multiple light source modules are arranged in sections on the measuring frame to provide illumination for different areas of the hole to be measured on the workpiece. A laser, mounted above the detection surface, is used to project a laser speckle pattern onto the workpiece surface for point position accuracy detection. Multiple correction targets are fixedly mounted on the measurement frame and located within the camera's field of view; The computing control unit is electrically connected to the plurality of cameras, the plurality of light source modules, at least one robot, and the laser; the computing control unit is configured to: The system stores the reference coordinates, camera parameters, feature templates, and the nominal coordinates of the hole positions to be measured associated with the feature templates in the workpiece coordinate system, which are established through the system initialization method. Store and execute the steps of the workpiece hole and point position measurement method, and / or the steps of the system initialization method.

2. The multi-view stereo vision based workpiece hole and dot location degree flexibility inspection apparatus of claim 1, wherein, The multiple cameras are divided into multiple camera groups, and the same camera can be assigned to different camera groups; the light source module is an LED light source, and each light source module is assigned to one camera group.

3. The multi-view stereo vision based workpiece hole and dot location degree of freedom inspection apparatus of claim 1, wherein, The correction target is fixedly installed on a non-movable part of the measurement frame and is used for dynamic error correction; the correction target is composed of a high-contrast pattern to ensure that there are at least two correction targets in the field of view of each camera.

4. A method for detecting the position degree of a workpiece hole and point based on multi-view stereo vision, using the flexible detection equipment for detecting the position degree of a workpiece hole and point based on multi-view stereo vision according to any one of claims 1 to 3, characterized in that, The method is based on reference coordinates, camera parameters, feature templates, and the nominal coordinates of the hole to be measured associated with the feature templates in the workpiece coordinate system, established through a system initialization method; the method includes the following online detection steps: S21: Workpiece hole image acquisition and two-dimensional coordinate positioning: Acquire the image of the workpiece hole to be measured; For the acquired image of the workpiece hole to be measured, use feature templates for matching and positioning to obtain the two-dimensional center coordinates of each hole to be measured on a single image; S22: Calculation of three-dimensional coordinates of workpiece hole: For each hole to be measured, using camera parameters, the two-dimensional center coordinates of the hole to be measured obtained by at least two cameras are triangulated to calculate the three-dimensional coordinates of the hole to be measured in the equipment coordinate system. S23: Workpiece Hole Coordinate System Mapping and Position Evaluation: Taking the three-dimensional coordinates of multiple holes to be tested in the equipment coordinate system and their nominal coordinates in the workpiece coordinate system as input, the transformation parameters from the equipment coordinate system to the workpiece coordinate system are calculated through an optimization algorithm. Then, the three-dimensional coordinates of all holes to be tested are transformed to the workpiece coordinate system to obtain the actual coordinates. By comparing the actual coordinates of each hole to be tested with its nominal coordinates, it is determined whether its position deviation is within the preset tolerance range, and the position detection result of the hole to be tested is output. S24: Workpiece point position measurement: Acquire speckle images of the workpiece points to be measured; perform digital image correlation analysis on the acquired speckle images of the workpiece points to be measured, calculate the speckle displacement field, and convert the displacement field into a height field through a calibration formula to obtain the position measurement results of the points to be measured.

5. The method for detecting the positional accuracy of workpiece holes and points based on multi-view stereo vision according to claim 4, characterized in that, In step S21, the matching and localization using feature templates includes: Load the feature template, which is a mask image with gradient field information; By calculating the correlation between the feature template and the gradient field of the target image, and combining the particle swarm optimization algorithm or the feature pyramid algorithm for optimization search, the two-dimensional center coordinates of the hole to be tested on the image are determined.

6. The method for detecting the positional accuracy of workpiece holes and points based on multi-view stereo vision according to claim 4, characterized in that, In step S21: For the same camera under the same light source module, the same hole to be measured is photographed repeatedly N times, where N≥2; the arithmetic mean of the two-dimensional center coordinates calculated N times is taken as the final two-dimensional center coordinates of the hole to be measured.

7. The method for detecting the positional accuracy of workpiece holes and points based on multi-view stereo vision according to claim 4, characterized in that, After step S21 and before step S22, a dynamic error correction step is also included: Acquire images of the corrected target, identify the corrected target in the image, and locate its two-dimensional center coordinates; The camera pose transformation parameters are calculated by comparing the deviation between the two-dimensional center coordinates of the corrected target in the current image and the reference coordinates established in the system initialization method. The two-dimensional center coordinates of the hole position to be measured obtained in step S21 are corrected using the pose transformation parameters.

8. The method for detecting the positional accuracy of workpiece holes and points based on multi-view stereo vision according to claim 4, characterized in that, In step S22, the triangulation is implemented using Newton's iteration method, including: Randomly initialize a three-dimensional point coordinate system to represent the hole position to be measured in the device coordinate system; The coordinates of the 3D point are projected onto the image coordinate system of each camera using the camera parameters to obtain the projected 2D point. The coordinates of the three-dimensional point are iteratively optimized to minimize the reprojection error between its projected two-dimensional point and the two-dimensional center coordinates of the corresponding hole position obtained in step S21. Finally, the optimized three-dimensional point coordinates are used as the three-dimensional coordinates of the hole to be tested in the device coordinate system.

9. The method for detecting the positional accuracy of workpiece holes and points based on multi-view stereo vision according to claim 4, characterized in that, In step S23, the optimization algorithm is Newton's method, and the transformation parameters include the rotation matrix and the translation vector.

10. A system initialization method for implementing the workpiece hole and point position measurement method based on multi-view stereo vision as described in any one of claims 4 to 9, characterized in that, The method includes the following steps: S11: Camera configuration and installation: Arrange multiple cameras on the measuring frame to ensure that each hole to be measured on the workpiece can be observed by at least two cameras; S12: Allocation of light source and camera group: Configure different light source modules for different areas of the workpiece to be measured, and allocate a camera group to each light source module. The same camera can be assigned to different camera groups. S13: Laser installation: Install the laser vertically above the detection position, ensuring that the beam is perpendicular to the surface of the workpiece being measured; S14: Correction target fabrication and installation: At least two correction targets are fixedly installed on the immovable parts of the measurement frame, and at least two correction targets are observed within the field of view of each camera; S15: Feature template creation: For each hole to be tested and each correction target under each camera field of view, draw its edge contour to generate the corresponding feature template, associate the feature template of the hole to be tested with its nominal coordinates in the workpiece coordinate system, and record the position of the correction target in the image as the reference coordinates. S16: Camera Calibration: Using a coded dot calibration board, the camera parameters of each camera are calculated by having all cameras capture multiple images of the calibration board from different angles. The camera parameters include the intrinsic parameters of each camera and the extrinsic parameters of all cameras relative to a common device coordinate system, thus completing the camera calibration.

Images