Multi-vision hole site measurement method and device for structural member

Abstract

The invention discloses a multi-view vision hole site measuring method and device for a structural part, and relates to the technical field of multi-view vision. In order to solve the technical problems that an existing hole site detection method is insufficient in detection precision, low in detection efficiency, incapable of achieving comprehensive detection and unstable, the technical scheme provided by the invention includes that the multi-view vision hole site measurement method for the structural component comprises the following steps: loading a feature template image of a workpiece hole site, calculating similarity by using an image gradient field, and positioning the central position of a hole site from the workpiece image; a step of calibrating the position offset of the camera by using the correction target; according to the camera position, the light source condition and the image data of the hole site, the accurate coordinate position of the hole site of the workpiece in the three-dimensional space is calculated through a three-dimensional reconstruction method; and converting the three-dimensional coordinate data in the equipment coordinate system into a workpiece coordinate system. The method is suitable for being applied to the work of multi-view vision hole site measurement of the structural member.

Description

Technical Field

[0001] The invention relates to the field of multi-eye vision technology, and in particular to multi-eye vision hole position measurement of structural parts. Background Art

[0002] In the field of machine vision and 3D measurement, multi-camera vision technology, as an important non-contact measurement method, has been widely used in various industries, especially in precision manufacturing and assembly processes. By capturing images of the same object from different angles with multiple cameras and combining them with computer vision algorithms, high-precision 3D reconstruction and measurement can be achieved. In recent years, with the improvement of hardware equipment performance and the optimization of computing algorithms, multi-camera vision systems have been widely used in many fields such as industrial inspection, robot guidance, and virtual reality.

[0003] The basic concept of multi-camera vision technology comes from the principle of stereoscopic perception of the human visual system. At present, three-dimensional measurement technology based on multi-camera vision has matured and has achieved remarkable results in many fields. For example, in the aerospace field, multi-camera vision systems are used to automatically detect aircraft structural parts; in the robotics field, multi-camera vision is used for environmental perception and path planning; multi-camera vision systems are also widely used in parts inspection and assembly positioning in automated manufacturing. The research on related technologies has entered the stage of precision improvement and real-time optimization, especially in the application of large-scale production lines, which is gradually replacing traditional manual inspection methods.

[0004] In the automotive industry, with the development of intelligent manufacturing and industrial automation, 3D visual measurement technology has begun to be widely used in vehicle production lines, especially in the hole detection and assembly accuracy verification of workpieces. Traditional hole detection methods mostly rely on manual inspection or the cooperation of robotic arms and laser ranging equipment. These methods usually require a long operation time and rely on manual experience to judge the qualification of the hole position, which is prone to deviations. For example, although the commonly used robotic arm visual inspection system can provide higher accuracy, due to the complexity of its movement mode, it is easily affected by the movement error of the robotic arm, resulting in instability of the detection results. In addition, the robotic arm usually requires complex coordination and control during operation, which increases the complexity of the equipment and the downtime of the production line.

[0005] Traditional hole detection methods usually use two methods: random inspection or selective inspection. Random inspection is to randomly select a part of a large number of workpieces for inspection to infer the quality of the entire batch. However, this method cannot fully reflect the quality of the entire batch of workpieces, and the randomly selected workpieces may have deviations, resulting in the inability to timely discover potential quality problems. Selective inspection is to detect several points on a certain workpiece, but due to the uncertainty of the distribution of holes on the workpiece, it is often impossible to cover all measurement points of the entire workpiece. Therefore, this method also has the risk of incomplete detection and misjudgment.

[0006] At present, most high-precision detection devices use a robot arm and a sensor to work together. Although they can provide high detection accuracy, the complexity and high cost of the robot arm movement make the detection process often unstable and the production line production rate is limited. Therefore, how to achieve high-precision, comprehensive and stable hole position detection has become a difficulty in current technology.

[0007] Although existing technologies have achieved relatively successful results in many application scenarios, in fields with high precision requirements such as automobile manufacturing, existing multi-view vision measurement systems still face some technical challenges, which are mainly reflected in the following aspects:

[0008] Insufficient detection accuracy: The multi-eye vision system in the existing technology is limited by factors such as camera position offset and ambient light changes, which can easily lead to fluctuations in measurement accuracy.

[0009] Low detection efficiency: The traditional robot arm collaborative operation method is complicated to operate, resulting in a long detection process and failing to meet the needs of efficient production line operation.

[0010] Unable to conduct comprehensive inspection: Traditional sampling and selective inspection methods cannot fully cover all holes of the workpiece, which can easily lead to missed inspections or misjudgments and cannot meet the requirements of high-quality production.

[0011] Instability: Existing high-precision detection equipment usually relies on complex mechanical structures, such as robotic arms, and unstable errors may occur during movement, affecting the overall detection performance. Summary of the invention

[0012] In order to solve the technical problems of insufficient detection accuracy, low detection efficiency, inability to conduct comprehensive detection and instability in the existing hole position detection methods, the technical solution provided by the present invention includes:

[0013] A multi-eye vision hole position measurement method for a structural part, comprising:

[0014] The step of locating the center position of the hole from the workpiece image by loading a feature template image of the workpiece hole;

[0015] Steps to calibrate the camera's position offset;

[0016] The step of calculating the accurate coordinate position of the workpiece hole in three-dimensional space according to the camera position, light source conditions and image data of the hole;

[0017] The steps to convert the 3D coordinate data in the device coordinate system to the workpiece coordinate system.

[0018] Furthermore, a preferred embodiment is provided, in which the similarity is calculated using the image gradient field to locate the center position of the hole from the workpiece image.

[0019] Furthermore, a preferred embodiment is provided, in which a correction target is used to calibrate the position offset of the camera.

[0020] Furthermore, a preferred embodiment is provided, which uses a three-dimensional reconstruction method to calculate the accurate coordinate position of the workpiece hole in three-dimensional space.

[0021] Based on the same inventive concept, the present invention also provides a multi-eye vision hole position measurement device for structural parts, comprising:

[0022] A module for locating the center position of the hole from the workpiece image by loading the feature template image of the workpiece hole;

[0023] A module for calibrating the position offset of the camera;

[0024] A module that calculates the exact coordinate position of the workpiece hole in three-dimensional space based on the camera position, light source conditions, and image data of the hole;

[0025] A module that converts the 3D coordinate data in the device coordinate system to the workpiece coordinate system.

[0026] Based on the same inventive concept, the present invention also provides a multi-eye vision hole position measurement device for structural parts, which is used to implement the method described, including:

[0027] The task model definition unit is used to create a logical relationship model of the task, which includes the input, output, dependency and execution conditions of the task;

[0028] An automated scheduling unit, used to automatically generate a scheduling order for task execution according to the dependency relationship in the task model, and determine the priority of task execution according to the scheduling result;

[0029] An execution unit, which is used to execute tasks in the order indicated by the scheduling unit and dynamically adjust the task execution strategy according to real-time execution data;

[0030] The result feedback unit is used to feed back the task results to the system after the task is completed, for verification, summary and subsequent processing, and automatically generate a report based on the execution results;

[0031] The adaptive optimization unit is connected to the execution unit and is used to adjust the task execution strategy according to the execution feedback data during the task execution process to optimize the execution efficiency of subsequent tasks.

[0032] Based on the same inventive concept, the present invention also provides a multi-eye vision hole position measurement system for structural parts, which is implemented based on the device and further includes:

[0033] The display and control unit is used to display the task execution status, scheduling information and execution results, and allow the user to intervene and adjust the task scheduling process.

[0034] Based on the same inventive concept, the present invention also provides a computer storage medium, a display and control unit, which are used to display the task execution status, scheduling information and execution results, and allow the user to intervene and adjust the task scheduling process.

[0035] Based on the same inventive concept, the present invention also provides a computer, including a processor and a storage medium. When the processor reads a computer program stored in the storage medium, the computer executes the method described.

[0036] Based on the same inventive concept, the present invention also provides a computer program product, which is a computer program. When the computer program is executed, the method described above is implemented.

[0037] Compared with the prior art, the technical solution provided by the present invention is beneficial in that:

[0038] By adopting a multi-eye vision system and a fixed structure design, the problems of incomplete hole position detection, low efficiency and instability in the traditional method are solved. In particular, the full coverage and multi-point measurement method ensures that each hole position of each workpiece can be observed by multiple cameras. Compared with the traditional random inspection and selective inspection methods, this avoids missed inspections and misjudgments caused by limited detection range. Compared with the traditional method of only inspecting some hole positions, the system of the present invention can fully detect all hole positions, greatly improving the reliability of detection and the guarantee of workpiece quality.

[0039] The fixed structure design avoids the motion error of the traditional robotic arm system and improves the stability of the detection. Although the traditional detection method of the collaborative operation of the robotic arm has advantages in accuracy, due to the unstable factors in the movement of the robotic arm, the detection results may be affected by the mechanical structure. For example, during the movement of the existing robotic arm, positioning errors may be caused by factors such as temperature and vibration, resulting in unstable detection accuracy. The present invention adopts a fixed camera structure to avoid errors caused by movement and ensure the consistency and stability of detection accuracy.

[0040] Multiple shooting and image processing technology significantly improve the detection speed. Traditional high-precision detection methods, such as robotic arms with laser scanning or tactile sensors, often take a long time to complete the detection of workpieces. In contrast, the system of the present invention uses multi-cameras to shoot from different angles at the same time, combined with image processing algorithms, to complete the precise measurement of workpiece hole positions in a short time, and the measurement time is shortened from the traditional 30-60 minutes to 10-30 seconds, which greatly improves the work efficiency of the production line and meets the needs of modern automobile manufacturing for fast and efficient detection.

[0041] High-precision measurement results come from accurate feature template matching and iterative optimization algorithms. By applying feature template matching and gradient field calculation in the image processing stage, the present invention can accurately locate the center of the workpiece hole, and use an iterative optimization algorithm to solve the mapping problem between the device coordinate system and the workpiece coordinate system. Compared with the simple geometric algorithm or single-shot measurement method in the prior art, this has higher accuracy. Traditional methods usually rely on relatively simple geometric calculations, while the present invention uses more complex image matching and optimization technology to ensure that the detection accuracy can reach ±0.1mm, and the repeated measurement accuracy is ±0.02mm, which significantly improves the accuracy of the measurement results.

[0042] The calibration process of the correction target and the camera improves the accuracy and consistency of the system. In traditional multi-eye vision systems, camera calibration is often a complex and error-prone process, especially during long-term use, where a slight offset in the camera position will affect the measurement accuracy of the entire system. The present invention, by introducing a correction target, can correct the offset of the camera in real time, ensuring that the accuracy of the positional relationship between the cameras is maintained during each measurement. This correction mechanism effectively avoids measurement errors caused by camera offset, ensures that each measurement is performed in the same coordinate system, and improves the stability of the system and the consistency of the measurement results.

[0043] Compared with the traditional reliance on complex collaboration of robotic arms, the present invention achieves an efficient and stable measurement method by simplifying the design of the mechanical structure. In traditional detection systems, the movement of the robotic arm requires complex control algorithms, and in practical applications, the stability of the robotic arm is often affected by environmental factors such as vibration and temperature changes, resulting in unstable accuracy. The present invention eliminates the collaborative operation of the robotic arm through a fully fixed structure design, making the camera positioning and measurement process simpler and more stable, and greatly reducing production line downtime. Compared with the existing robotic arm + visual system solution, the high efficiency and stability of this solution have obvious advantages in reducing equipment maintenance and operational complexity.

[0044] It is suitable for multi-eye vision hole position measurement of structural parts. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a three-dimensional schematic diagram of a multi-eye vision hole position measurement device for structural parts;

[0046] Figure 2 for Figure 1 A top view of

[0047] Figure 3 A schematic diagram of the test results. DETAILED DESCRIPTION

[0048] In order to make the advantages and benefits of the technical solution provided by the present invention more clearly reflected, the technical solution provided by the present invention is now further described in detail in conjunction with the accompanying drawings, specifically:

[0049] Embodiment 1: This embodiment provides a multi-vision hole position measurement method for a structural part, comprising:

[0050] The step of locating the center position of the hole from the workpiece image by loading a feature template image of the workpiece hole;

[0051] Steps to calibrate the camera's position offset;

[0052] The step of calculating the accurate coordinate position of the workpiece hole in three-dimensional space according to the camera position, light source conditions and image data of the hole;

[0053] The steps to convert the 3D coordinate data in the device coordinate system to the workpiece coordinate system.

[0054] Implementation method 2: This implementation method further limits the multi-vision hole position measurement method for structural parts provided in implementation method 1, and uses the image gradient field to calculate the similarity and locate the center position of the hole position from the workpiece image.

[0055] Implementation method three: This implementation method further limits the multi-vision hole position measurement method for structural parts provided in implementation method one, and adopts a correction target to calibrate the position offset of the camera.

[0056] Embodiment 4: This embodiment further limits the multi-eye vision hole position measurement method for structural parts provided in embodiment 1. The accurate coordinate position of the workpiece hole position in three-dimensional space is solved by a three-dimensional reconstruction method.

[0057] Embodiment 5: This embodiment provides a multi-eye vision hole position measurement device for structural parts, comprising:

[0058] A module for locating the center position of the hole from the workpiece image by loading the feature template image of the workpiece hole;

[0059] A module for calibrating the position offset of the camera;

[0060] A module that calculates the exact coordinate position of the workpiece hole in three-dimensional space based on the camera position, light source conditions, and image data of the hole;

[0061] A module that converts the 3D coordinate data in the device coordinate system to the workpiece coordinate system.

[0062] Embodiment 6: This embodiment provides a multi-eye vision hole position measurement device for a structural part, which is used to implement the method provided in embodiment 1 and includes:

[0063] The task model definition unit is used to create a logical relationship model of the task, which includes the input, output, dependency and execution conditions of the task;

[0064] An automated scheduling unit, used to automatically generate a scheduling order for task execution according to the dependency relationship in the task model, and determine the priority of task execution according to the scheduling result;

[0065] An execution unit, which is used to execute tasks in the order indicated by the scheduling unit and dynamically adjust the task execution strategy according to real-time execution data;

[0066] The result feedback unit is used to feed back the task results to the system after the task is completed, for verification, summary and subsequent processing, and automatically generate a report based on the execution results;

[0067] The adaptive optimization unit is connected to the execution unit and is used to adjust the task execution strategy according to the execution feedback data during the task execution process to optimize the execution efficiency of subsequent tasks.

[0068] Embodiment 7: This embodiment provides a multi-eye vision hole position measurement system for structural parts, which is implemented based on the device provided in Embodiment 6 and further includes:

[0069] The display and control unit is used to display the task execution status, scheduling information and execution results, and allow the user to intervene and adjust the task scheduling process.

[0070] Specifically, the system workflow is:

[0071] 1. Camera configuration and installation

[0072] First, adjust the camera installation position according to the installation position of the workpiece and the workpiece hole position to be measured. Make sure that each hole position is observed by at least two or more cameras to ensure the comprehensiveness of the measurement.

[0073] Detailed description:

[0074] A fixed camera mounting bracket is set above the workpiece installation position and the hole to be measured to ensure that the camera can shoot the hole from different angles. The camera's field of view must cover the entire workpiece to avoid blind spots.

[0075] Each camera should have a high resolution and appropriate focal length to ensure that the image captured can clearly present the hole characteristics. On this basis, the clarity of the hole image can be ensured by adjusting the aperture and focal length of the camera.

[0076] The focal length of each camera should be adjusted according to the distance between the workpiece and the hole to ensure the accurate position of the hole image in the camera's field of view when shooting.

[0077] Each camera configuration should take into account the distribution of workpiece holes so that each hole can be captured by at least two cameras to ensure the reliability of the measurement data. Adjust the camera's aperture and focal length to the optimal configuration to make the image clear, and observe the dynamic picture taken by the camera after the camera is started to select the image center of the hole.

[0078] 2. Correction target production and installation

[0079] Make and install correction targets, and use them to calibrate the camera position in real time to avoid measurement errors caused by camera offset.

[0080] Detailed description:

[0081] Production of correction target: Use a combination of square iron sheet and round marking paper to make the correction target. The size of the target should be adapted to the size ratio of the workpiece and the camera to ensure that the camera can accurately identify the target.

[0082] Installation of correction targets: The correction targets should be installed on the internal structural columns of the measurement device. Each camera should be able to observe at least two correction targets to ensure the accuracy of the calibration process.

[0083] During installation, it is necessary to ensure that the correction target and the hole position of the workpiece are not blocked to avoid the problem that the correction target cannot be observed due to blockage or movement of the object. During installation, it is also necessary to ensure that the distance of the correction target is not too close to avoid errors in the calculation of the correction rotation vector.

[0084] The installation position of the correction target must ensure that it can cover all the hole areas of the workpiece, ensure that the camera position can be corrected in real time during each inspection, and avoid measurement errors caused by slight offsets of the camera.

[0085] 3. Allocation of light sources and camera groups

[0086] According to the measurement requirements of the workpiece holes, the light source module and camera combination are reasonably configured to ensure that each hole of the workpiece can obtain sufficient light and ensure image quality.

[0087] Detailed description:

[0088] Different light source modules are configured for each hole area according to the hole area of ​​the workpiece. For example, the top hole area of ​​the workpiece uses the top light source, and the bottom hole area of ​​the workpiece uses the bottom light source.

[0089] Multiple cameras can be configured under each light source module to ensure that each hole position has at least two cameras for observation. By setting the camera sequence number, you can start or shut down a specific camera to measure a specific area.

[0090] The design of different camera and light source combinations aims to ensure that each hole position can be captured by multiple cameras regardless of the lighting conditions, thereby improving the accuracy and stability of hole position measurement.

[0091] Under each light source module, all cameras should maintain the same aperture setting to avoid the difference in illumination between different cameras affecting the measurement results.

[0092] 4. Correction of target and target hole template production

[0093] Make hole template and correct target template, draw the edge contour of hole and target manually in the initial configuration stage, and generate corresponding template file. This template will be used in the subsequent measurement process.

[0094] Detailed description:

[0095] Hole template creation: Manually draw the outline of each workpiece hole observed by the camera and mark the nominal coordinates of the hole in the template. This template is not only used for hole detection, but also helps the system identify the position of each hole in the image.

[0096] Correction target template preparation: Prepare correction target templates, record the edge contours of the correction targets, and ensure that the fitting accuracy of each target template meets the standards.

[0097] During the template production process, pay attention to the fitting error and require that the deviation of the template center position is not greater than the specified threshold. After the template is generated, real-time contour matching is performed to ensure that the deviation of the hole position and the modified target template meets the requirements.

[0098] The template data is uploaded to the measurement system in the form of a JSON file so that it can be calculated by matching with the target template in subsequent measurements.

[0099] 5. Camera calibration and parameter calculation

[0100] When the measuring device is initialized, the camera is calibrated and the internal and external parameters of the camera are calculated to ensure that the multi-eye vision system can accurately reflect the relative position relationship between the cameras.

[0101] Detailed description:

[0102] Calibration plate photography: Use the coded dot calibration plate for camera calibration. Each camera needs to take at least 20 calibration plate images, and ensure that each camera group is shot by at least two cameras to ensure the accuracy of calibration.

[0103] Calculation of camera intrinsic and extrinsic parameters: Using the image of the calibration plate, first calculate the intrinsic parameters of each camera, then calculate the extrinsic parameters between cameras to ensure that the relative position relationship between cameras is accurately established.

[0104] Calibration result optimization: When calculating the external parameters, manually delete the calibration plate images that are not related to other cameras to avoid calibration result deviations due to association errors. Each camera should be associated with at least two other cameras to ensure the reliability of the calibration results.

[0105] Through the iterative optimization process, the error is reduced until the solution results of the intrinsic and extrinsic parameters meet the convergence conditions.

[0106] 6. Workpiece hole position measurement

[0107] During the first measurement, upload the calibration data and template data to measure the workpiece hole position.

[0108] Detailed description:

[0109] Before measuring the workpiece hole position, the calibrated camera internal and external parameters and hole template data are uploaded to the measurement system. Through template matching technology, the system will automatically locate the center position of the workpiece hole position.

[0110] Use a camera to capture an image of the workpiece and calculate the precise position of the workpiece hole in the camera image by matching it with the template data.

[0111] The system calculates the final position of the workpiece hole in the equipment coordinate system and the workpiece coordinate system based on the transformation relationship between the camera coordinate system and the workpiece coordinate system.

[0112] During the measurement process, the system will calibrate the camera position in real time based on the data of the correction target to ensure that each measurement is performed in an accurate coordinate system. The measurement results will be displayed through a visual interface, and the user can view the deviation between the final position of the hole and the nominal position.

[0113] Embodiment 8: This embodiment provides a computer storage medium, a display and control unit, which are used to display the task execution status, scheduling information and execution results, and allow the user to intervene and adjust the task scheduling process.

[0114] Embodiment 9: This embodiment provides a computer, including a processor and a storage medium. When the processor reads a computer program stored in the storage medium, the computer executes the method provided in embodiment 1.

[0115] Embodiment 10: This embodiment provides a computer program product, which is a computer program. When the computer program is executed, the method provided in embodiment 1 is implemented.

[0116] Implementation Method XI: Combination Figure 1-3 This embodiment is described in detail by using specific examples to further describe the technical solution provided above. Specifically:

[0117] Different from the traditional method, this embodiment proposes a multi-eye vision hole position measurement system and device for automobile workpieces under fixed structure, which innovatively adopts full coverage and multi-point measurement to achieve comprehensive detection of all workpieces and their various points. It solves the problem that random inspection cannot detect all workpieces and selective inspection cannot cover all points of workpieces. In addition, the measurement device adopts a completely fixed structure and does not require manual and mechanical arm collaboration, which solves the problems of device instability and slow detection speed during motion detection, and maintains the efficiency of the production line while ensuring that the accuracy meets the standards.

[0118] The workpiece hole position detection process described in this embodiment has the following technical solutions:

[0119] Step 1: Feature template matching to obtain the hole center position

[0120] Feature template matching locates the hole center by loading the feature template image of the workpiece hole in the preprocessing process, and uses the image correspondence operator to calculate the gradient field of the image. The feature template image is the product of the non-zero value of the workpiece hole mask and the gradient of the corresponding hole image in the x and y directions, that is, the mask image with gradient field information. The feature template image is mainly used to perform similarity measurement calculations in the image captured by the current camera. The similarity measurement is based on the correlation of the gradient field. The template pixels are transformed and aligned with the original image pixels using the interpolation method. The sum of the gradient point products of the corresponding positions of the current image and the feature template image is calculated by sliding the window to measure the similarity between the two images. As shown in Formula 1, S(i, j) and T(i, j) are the gradient values ​​of the feature template image at the corresponding (i, j) position and the current matching image.

[0121]

[0122] Combined with algorithms such as feature pyramid, the center position of the workpiece hole under the corresponding camera can be quickly located. The workpiece hole taken by the same camera under the same light source group is photographed repeatedly for multiple times, and the average of multiple calculations is used as the final center position result of the workpiece hole on the image taken by the camera under the light source group.

[0123] The correction target in the measuring device can be used to correct the camera position when it is offset. The offset of the camera will cause deviations in the observation of the workpiece hole position, destroying the positional relationship between cameras established through the camera calibration process. The center position of the correction target is obtained in the same way as the center position of the workpiece hole. The transformation parameters of the correction target are calculated from the calibrated target center position and the target center position after the camera is moved, that is, the position transformation parameters of the camera position from before to after the movement are obtained. The center position of the workpiece hole in the camera image after the movement is corrected by the transformation parameters and transformed back to the camera coordinate system formed by the original camera position relationship.

[0124] Step 2: Calculate the device coordinate system spatial position

[0125] Solve the position of the device coordinate system in space. For the center point and camera position results of a single workpiece hole under different light sources and camera combinations, calculate the 3D point position of the image center point in the camera coordinate system. Obtain the camera intrinsic parameters, rotation, translation and other external parameters through the codable dot calibration plate, map the randomly initialized 3D points from the device coordinate system to the camera coordinate system through rotation and translation operations, and use the corresponding camera intrinsic parameters to transform the 3D points in the camera coordinate system into 2D points in the image coordinate system corresponding to the camera with the serial number. By narrowing the gap between the initialized projected 2D point and the image center point obtained in step 1, use Newton's method for iterative optimization to finally obtain the 3D point position corresponding to the center of each workpiece hole in the device coordinate system. The calculation process is shown in Formula 2, where is the desired world coordinate, p c is the coordinate of the center point of the workpiece hole in the image. Through the iterative process, the sum of the distances between the image points after imaging and projection by each camera and the points obtained by matching the image features is minimized.

[0126]

[0127] Step 3: Solve the workpiece coordinate system mapping

[0128] The device coordinate system is mapped to the workpiece coordinate system. In order to determine the position of the center point of the workpiece hole on the workpiece in the workpiece coordinate system, an iterative optimization method is used to continuously narrow the gap between the 3D point position of the workpiece hole in the device coordinate system and the actual nominal coordinate position of the workpiece hole in the workpiece coordinate system, thereby obtaining the corresponding optimal rotation and translation parameters of the workpiece hole center point from the device coordinate system to the workpiece coordinate system. The 3D coordinates of the workpiece hole center point obtained in the device coordinate system are transformed to the workpiece coordinate system through the rotation and translation parameters to obtain the final workpiece hole coordinate value.

[0129] From the beginning of the measurement process when the camera captures the image to the end of the calculation and detection, compared with the average time of 30-60 minutes for the manual + combined method, the time consumption of this implementation can be well controlled within the range of 10 to 30 seconds. The detection accuracy is ±0.1mm, and the detection accuracy is ±0.02mm after 10 repeated measurements, which can quickly measure the hole position of automotive workpieces.

[0130] During the specific operation:

[0131] 1. Camera configuration and installation

[0132] Adjust the camera position according to the workpiece installation position and the workpiece hole position to be measured. Each camera can capture multiple hole features to ensure that each hole position is observed by at least two cameras. Start the camera, adjust the camera aperture and focal length, fix the aperture size of each camera, and observe the camera shooting dynamic picture at the same time to select the center hole position of the observation image, and rotate the focal length until the hole position is clear.

[0133] 2. Correction target production and installation

[0134] The camera correction target is made by combining square iron sheets and circular marking paper. The correction target is installed on the internal structural column of the measuring device. For each camera, at least two or more correction targets can be observed in the image it observes. The correction target should be installed to avoid being too close, which will lead to a large deviation in the calculation result of the correction rotation vector. At the same time, the selection of the correction target should avoid the problem of incomplete observation of the correction target caused by the movement of the measured workpiece and the offset of the camera.

[0135] 3. Allocation of light sources and camera groups

[0136] In response to the measurement requirements of different workpiece hole positions, it is necessary to allocate and process the cameras fixedly installed in Implementation Method 1, and set different light source modules for different hole positions of the workpiece. For example, the top-level light source of the measuring device is configured for the upper side hole of the workpiece. By setting whether the serial number camera under each light source group is started or not, the light source modules and the camera groups under the modules are finally configured for different hole positions of the workpiece. At the same time, each camera involved in the measurement can be reused under different light source modules.

[0137] 4. Correction of target and target hole template production

[0138] During the initialization configuration stage of the measuring device, it is necessary to manually draw the edge contours of the workpiece hole position and the corrected target under each camera observation, and generate the corresponding hole position and target template. During the template making process, it is necessary to observe the deviation of the center position of the fitted template. In the real-time contour matching, the position fluctuation is not greater than the specified threshold, which is an effective indicator of successful template making. At the same time, in the production of the workpiece hole template, it is also necessary to add the nominal coordinates corresponding to the hole position to establish the position of the hole position in the workpiece coordinate system. During the measurement process, the hole template and the corrected target template and other data are uploaded to the measurement system in the form of a json file, so that the corresponding calculations are performed in the subsequent measurement by matching the corresponding template with the hole position and target.

[0139] 5. Camera calibration and parameter calculation

[0140] The camera calibration process is also required in the initialization stage of the measuring device. The positional relationship between cameras needs to be obtained through camera calibration. When shooting the calibration plate, it should be ensured that at least two cameras between each group of shooting cameras can capture the calibration plate. At the same time, each camera needs to capture more than twenty calibration plate images. Since the codable dot calibration plate is used for camera calibration, the problem of being unable to solve due to incomplete shooting of the calibration plate can be avoided during the image shooting process.

[0141] In the process of camera parameter settlement, the intrinsic parameters are calculated first, and then the extrinsic parameter results are obtained. According to the extrinsic parameter results, the calibration plate images in the front row are manually deleted. During the deletion process, pay attention to the association between cameras. For example, for any camera, if there is no camera associated with it, the solution is doomed to fail and needs to be recalibrated. Repeat this process to reduce the error result to below the threshold of parameter solution convergence until the model fitting is completed and the internal and external parameters are successfully settled. During the measurement process, the internal and external parameters are uploaded to the measurement system for measurement use.

[0142] 6. Workpiece hole position measurement

[0143] In the first measurement, the calibration data and template data need to be uploaded to the measurement system, the workpiece holes to be measured are set to participate in the fitting of the coordinate system, and the center position coordinates of the workpiece holes are measured.

[0144] The technical solution provided by the present invention is further described in detail above through several specific implementation modes in order to highlight the advantages and benefits of the technical solution provided by the present invention. However, the several specific implementation modes described above are not intended to be used as limitations on the present invention. Any reasonable modification and improvement of the present invention, combination of implementation modes and equivalent substitution within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A multi-eye vision hole position measurement method for structural parts, characterized in that: include: The step of locating the center position of the hole from the workpiece image by loading a feature template image of the workpiece hole; Steps to calibrate the camera's position offset; The step of calculating the accurate coordinate position of the workpiece hole in three-dimensional space according to the camera position, light source conditions and image data of the hole; The steps to convert the 3D coordinate data in the device coordinate system to the workpiece coordinate system.

2. A multi-vision hole position measurement method for structural parts according to claim 1, characterized in that: The similarity is calculated using the image gradient field to locate the center of the hole in the workpiece image.

3. A multi-vision hole position measurement method for structural parts according to claim 1, characterized in that: The position offset of the camera is calibrated using a correction target.

4. The multi-vision hole position measurement method for structural parts according to claim 1, characterized in that: The accurate coordinate position of the workpiece hole in three-dimensional space is solved through the three-dimensional reconstruction method.

5. A multi-eye vision hole position measurement device for structural parts, characterized in that: include: A module for locating the center position of the hole from the workpiece image by loading the feature template image of the workpiece hole; A module for calibrating the position offset of the camera; A module that calculates the exact coordinate position of the workpiece hole in three-dimensional space based on the camera position, light source conditions, and image data of the hole; A module that converts the 3D coordinate data in the device coordinate system to the workpiece coordinate system.

6. A multi-eye vision hole position measurement device for structural parts, characterized in that: The method for implementing claim 1 comprises: The task model definition unit is used to create a logical relationship model of the task, which includes the input, output, dependency and execution conditions of the task; An automated scheduling unit, used to automatically generate a scheduling order for task execution according to the dependency relationship in the task model, and determine the priority of task execution according to the scheduling result; An execution unit, which is used to execute tasks in the order indicated by the scheduling unit and dynamically adjust the task execution strategy according to real-time execution data; The result feedback unit is used to feed back the task results to the system after the task is completed, for verification, summary and subsequent processing, and automatically generate a report based on the execution results; The adaptive optimization unit is connected to the execution unit and is used to adjust the task execution strategy according to the execution feedback data during the task execution process to optimize the execution efficiency of subsequent tasks.

7. A multi-eye vision hole position measurement system for structural parts, characterized in that: The device according to claim 6 further comprises: The display and control unit is used to display the task execution status, scheduling information and execution results, and allow the user to intervene and adjust the task scheduling process.

8. Computer storage medium, display and control unit, used to display task execution status, scheduling information and execution results, and allow users to intervene and adjust the task scheduling process.

9. A computer, comprising a processor and a storage medium, characterized in that: When the processor reads the computer program stored in the storage medium, the computer executes the method of claim 1 .

10. A computer program product, being a computer program, characterized in that When the computer program is executed, the method of claim 1 is implemented.

Images