Metal forging size detection method and system

Abstract

The invention relates to the field of machining and manufacturing, in particular to a method and system for detecting the size of a metal forge piece, and the method comprises the following steps: calibrating a three-dimensional modeling camera, and completing camera calibration, hand-eye calibration and turntable calibration in combination with an intelligent error feedback system and a multi-calibration-plate auxiliary system; standard part scanning and three-dimensional modeling, including auxiliary positioning of punctuation, scanning modeling, point cloud processing, point selection and feature extraction, and data storage in a database; scanning a test piece and performing three-dimensional modeling, wherein the process comprises three-dimensional scanning and data storage of the test piece in a normal-temperature or high-temperature environment, point cloud processing, multi-level point cloud registration, feature extraction and matching, distance calculation and judgment on whether the test piece is qualified or not; and outputting and storing the measurement data, and generating a test report to record the measurement data for inspection. According to the invention, high-precision size detection of the metal forge piece with a complex structure in a harsh environment is realized, the accuracy of a result is ensured, the batch detection efficiency and reliability are improved, and the method is suitable for industrial application scenes with high requirements.

Description

Technical Field

[0001] The invention relates to three-dimensional modeling, point cloud processing, feature extraction, registration algorithm and thermal expansion correction technology, is applied to the field of machining manufacturing, and is a metal forging size detection method and system. Background Art

[0002] The current widely used method for detecting the size of motor vehicle metal forgings in factories mainly relies on manual measurement and basic 3D scanning technology. In terms of manual measurement, factories usually use traditional measuring tools such as calipers and micrometers to manually measure the key dimensions of metal forgings and inspect product quality. Although this method is simple to operate, the measurement accuracy depends on the operator's experience and skills, and there are often large subjective errors. In addition, manual measurement can only cover a few key parts. It is difficult to fully detect metal forgings with complex shapes or multiple dimensions, and it is easy to miss or misjudge. In addition, manual measurement cannot directly detect high-temperature forgings.

[0003] In terms of 3D modeling, existing solutions usually use low-precision 3D scanning equipment to perform simple scans on the surface of metal forgings to generate basic point cloud data. After preliminary denoising, these point cloud data are usually compared with the standard model through a simple point-to-point comparison method. The lack of precision makes it difficult to ensure the accuracy and reliability of the test results; in addition, the data processing and analysis process is usually manual and relies on manual operation, which is inefficient and difficult to adapt to the needs of mass production.

[0004] These problems have resulted in obvious deficiencies in the accuracy, efficiency and data analysis capabilities of existing solutions. The repeatability and reliability of detection are poor, making it difficult to meet the high standards of precision testing required by modern manufacturing. Summary of the invention

[0005] The purpose of the present invention is to provide a metal forging dimension detection method and system to solve the problems raised in the above background technology.

[0006] In one aspect, the present invention provides a method for detecting the size of a metal forging, comprising the following steps:

[0007] S1, 3D modeling camera calibration;

[0008] S2. Scan and 3D model standard metal forging parts, including auxiliary positioning and marking, scanning and modeling, point cloud processing, point selection and feature extraction, and database storage data;

[0009] S3, scanning and 3D modeling of metal forging test pieces, including 3D scanning and data storage, point cloud processing, point cloud registration, feature extraction and matching, distance calculation and qualification judgment;

[0010] S4. Output the measured data and store it, and generate a test report to record the measured data for inspection.

[0011] Furthermore, the above scheme includes the following steps and contents:

[0012] S1 includes: the calibration equipment includes a binocular industrial camera, a robotic arm, a turntable and a black and white checkerboard calibration board. Specifically, the calibration steps are as follows:

[0013] S11. Equipment inspection and preparation: Install the binocular industrial camera firmly on the end tool of the robot arm, ensure that its field of view covers the area to be scanned, and use anti-entanglement flexible cables to avoid interfering with the movement of the robot arm; the turntable is placed horizontally and fixed on a stable table to ensure that it does not shake during rotation; the calibration plate is installed in a position relative to the turntable and the camera field of view to ensure that the camera is clearly visible when shooting at all angles; check and calibrate the position sensor on the robot arm, the angle encoder of the turntable, and the force / torque sensor to ensure that the sensor works accurately. All equipment is connected to the central control system to ensure stable transmission of power and signal lines, and reserve space for the full range of movement of the robot arm.

[0014] Software and intelligent system preparation: Start the calibration software, load the automated calibration process, ensure that the software has integrated the intelligent error feedback system and the multi-calibration board auxiliary system, set the error threshold in the calibration software interface, configure the error detection standard of the intelligent feedback system, ensure real-time monitoring and alarm, connect the control system of the camera and the robotic arm, and ensure that the automated control function is available.

[0015] S12, camera internal and external parameter calibration, including the following steps:

[0016] S121. Integration of multi-calibration plate auxiliary system: The multi-calibration plate auxiliary system is used to place multiple black and white grid calibration plates around the robot arm and turntable to ensure that the calibration plates cover different orientations and heights, thereby providing the camera with image data from multiple perspectives. The multi-calibration plate auxiliary system can obtain redundant calibration data from multiple perspectives, and cross-validate by combining the information of multiple calibration plates to eliminate the error accumulation caused by a single perspective, ensure the robustness of the calibration process, and improve the calibration accuracy.

[0017] Robotic arm motion planning: Program the robotic arm to move to different positions along a preset path and automatically capture multi-view images of the calibration plate.

[0018] S122, Image acquisition and intelligent error feedback: After the camera acquires multiple images of the calibration plate, the intelligent error feedback system checks the image clarity, distortion, and corner detection accuracy in real time. If it detects that the image is blurred or the corner extraction error is too large, the system will automatically adjust the camera's focus and exposure settings, or prompt the operator to reshoot.

[0019] Set a reprojection error threshold. If the reprojection error in the image exceeds the set range, the system will issue a warning through the user interface and provide adjustment suggestions.

[0020] S123, internal parameter calculation and optimization: Use Zhang's Method to process multiple collected calibration plate images and calculate the camera's internal parameters, including focal length, principal point position, distortion coefficient and other parameters.

[0021] Error optimization: Combined with the intelligent error feedback system, the Levenberg-Marquardt global optimization algorithm is applied to optimize the camera intrinsic parameters to further reduce the reprojection error.

[0022] S124. External parameter calculation and optimization: Use the calibration algorithm to extract external parameter data from multiple images of the calibration plate. The data contains the rotation matrix and displacement vector of the camera to represent the position and direction of the camera in the world coordinate system. With the help of the multi-calibration plate auxiliary system, redundant data is obtained from different positions and angles to reduce the deviation of external parameter calculation caused by shooting errors at a certain angle. The multi-calibration plate auxiliary system integrates the external parameter information of multiple calibration plates for cross-validation to improve the accuracy of external parameter calibration.

[0023] S13, Hand-eye calibration: Design the motion path of the robot arm so that it can shoot the calibration plate from multiple spatial positions and angles. The path planning is automatically completed by the central control system to ensure that the position information of each posture is complete and evenly covered.

[0024] Real-time error feedback and adjustment: The intelligent error feedback system monitors the position and motion trajectory of the end of the robot arm. When any path deviation or joint jitter is detected, the system automatically adjusts the motion path of the robot arm to ensure that the end tool remains in the accurate position.

[0025] S14, turntable calibration, including the following steps:

[0026] S141, automatic rotation and image acquisition: fix the calibration plate on the turntable, start the turntable automatic rotation, rotate at fixed angle increments, and capture an image of the calibration plate each time the camera rotates; the intelligent error feedback system monitors the rotation accuracy in real time through the turntable's angle encoder. If the rotation angle does not match the preset, the system will adjust the turntable speed or angle to ensure that the calibration plate is always in the best shooting position.

[0027] S142, accurate calculation of rotation axis and center, corner point detection and geometric derivation: The rotation axis and rotation center of the turntable are calculated using the geometric derivation method through the corner points of the calibration plate extracted from each image; combined with the detection results of the intelligent error feedback system, the system will automatically optimize the calculation of the rotation axis and center position to ensure accurate results.

[0028] S15, perform comprehensive calibration and error optimization, including the following:

[0029] Multi-module calibration data fusion: Integrate the results of camera intrinsic parameters, extrinsic parameters, hand-eye calibration and turntable calibration to generate a unified coordinate system and model for the entire system; the central control system applies a global optimization algorithm to optimize the relative position relationship between each physical module to further reduce system errors.

[0030] Intelligent error feedback and automatic adjustment: The intelligent error feedback system analyzes the error sources in each calibration data step in real time. If systematic errors are detected during the calibration process, the system will automatically provide adjustment suggestions or re-execute some calibration steps. Based on the error analysis results, the system can automatically adjust the motion trajectory of the robot arm, the rotation speed of the turntable or the parameters of the camera to improve the overall calibration accuracy.

[0031] S16. Summary and Report: The central control system automatically generates a report containing detailed information about the calibration process, including camera intrinsic parameters, extrinsic parameters, hand-eye transformation matrix, turntable rotation axis data, error analysis results, and optimization strategies. The report is stored in the cloud and can be used for reference and analysis by other devices. Through long-term error monitoring and feedback, the central control system can automatically remind you when to perform calibration again according to the situation, ensuring that the equipment is in the best working condition for a long time.

[0032] The S2 includes: after completing the camera calibration, starting to scan and three-dimensionally model the metal forging standard parts, this process includes auxiliary positioning and marking, scanning and modeling, point cloud processing, point selection and feature extraction, and database storage data.

[0033] Specifically, the auxiliary positioning mark method in the present invention adopts a color-coded patch method, and color-coded patches are affixed to the positions of the marking points on the surface of the metal forging standard parts, so that these points are easier to be recognized by the camera during scanning, and thus more prominent in the three-dimensional model. This method is simple and easy to implement, the markings are clearly visible, and will not affect the physical properties of the metal forging standard parts.

[0034] Specifically, the scanning and modeling include starting the calibrated camera, placing the metal forging standard parts on the shelf on the turntable, and performing multi-angle scanning on the metal forging standard parts after patching, so that the camera can obtain three-dimensional data of the standard parts from at least three different angles and in combination with multiple scans of the turntable to ensure that all patches are covered; based on the helix.toolki three-dimensional visualization framework, the data collected by the camera is processed into a three-dimensional point cloud and displayed, and the generated point cloud file is saved in the xyz file format, where each line represents the coordinates (x, y, z) of a three-dimensional point.

[0035] Furthermore, the generated point cloud file is processed, and the filter in the PCL (Point Cloud Library) point cloud library is used to reduce noise and filter the point cloud file. The denoised and filtered point cloud data is saved as a PCD file and an updated xyz file for subsequent analysis.

[0036] Furthermore, in point selection, 3D visualization software is used to select the patch position in the point cloud with the mouse, and the KD-Tree structure of PCL is used to search for the nearest point. For each selected point, the point with the shortest distance is searched within the preset radius, and its coordinates are recorded and saved as the matching point. The system integrates the SIFT (Scale-Invariant Feature Transform) algorithm to automatically extract key points without human intervention by analyzing the geometric surface features of standard metal forgings, such as corners, edges, protrusions, etc., and combines it with the auxiliary point selection method to perform intelligent and precise point selection.

[0037] Furthermore, every two points that need to measure the distance between points are grouped together and recorded as a point pair. The Euclidean distance between the two points is calculated and recorded as the calibration value. If points A and B are a group, the calculation formula is as follows:

[0038]

[0039] Set the allowable deviation range for each group of points, such as ±0.1 mm;

[0040] The corresponding relationship of each group of points, the standard distance value on the drawing, the actual calibration value and the set deviation range are stored in the database respectively.

[0041] Furthermore, in feature extraction, the FPFH (Fast Point Feature Histograms) algorithm in the PCL (Point Cloud Library) point cloud library is used. For each point, its neighborhood points within a certain radius are searched, the normal directions of these points are calculated, the normals of the neighborhood points are statistically analyzed, and the angle distribution between the normals and the center point normal is calculated to form a feature vector to extract the local geometric features of each selected point; the covariance matrix of the FPFH feature vector of each selected point is calculated to describe the correlation between features. The covariance matrix is ​​defined as:

[0042]

[0043] Among them, f i is the feature vector, is the mean of the eigenvector, n is the number of neighborhood points, and T is the transposed sign.

[0044] The eigenvectors and covariance matrices are stored in a database as the benchmark data for the selected points of the metal forging standard parts.

[0045] The S3 includes: scanning and three-dimensional modeling of the metal forging test piece, and the process includes three-dimensional scanning and data storage, point cloud processing, point cloud registration, feature extraction and matching, distance calculation and qualification judgment.

[0046] On the one hand, if the metal forging test piece is a room temperature workpiece, the processing flow is as follows:

[0047] Specifically, the three-dimensional scanning and data saving include starting a calibrated camera, placing a metal forging test piece on a shelf on a turntable, and performing multi-angle scanning on the metal forging test piece, requiring the camera to obtain three-dimensional data of the metal forging test piece from at least three different angles and in combination with multiple scans of the turntable to ensure that the required measurement range is covered; based on the helix.toolki three-dimensional visualization framework, the data collected by the camera is processed into a three-dimensional point cloud and displayed, and the generated point cloud file is saved in an xyz file format, where each row represents the coordinates (x, y, z) of a three-dimensional point.

[0048] Specifically, the point cloud processing uses filters in the PCL (Point Cloud Library) point cloud library to reduce noise and filter the point cloud file, and saves the denoised and filtered point cloud file as a PCD file and an updated xyz file for subsequent analysis.

[0049] Specifically, the point cloud registration uses PCA, GICP and ICP technology to perform multi-level registration to accurately align the point cloud of the metal forging test piece with the point cloud of the metal forging standard piece. The process includes the following steps:

[0050] S31. Perform preliminary alignment based on PCA (Principal Component Analysis): calculate the principal component axes of the metal forging test piece point cloud. The principal component axes are the directions of maximum variance of the point cloud data, corresponding to the X, Y, and Z axes. Through principal component analysis, the main directions of the point cloud are obtained and aligned with the principal axes of the metal forging standard part point cloud; calculate the projection of each point on the principal component axis to form a new coordinate system, so that the long axis of the point cloud is aligned with the long axis of the metal forging standard part point cloud, eliminating most of the initial posture differences.

[0051] S32. Refine the registration based on GICP (Generalized Iterative Closest Point): Use the GICP algorithm to approximate the point set near each point as a plane, establish a local plane model, and establish the matching of local geometric structures by calculating the normals and normal distances of these planes; reduce the overall registration error by calculating the local plane relationship between the point cloud of the metal forging test piece and the neighborhood points of the point cloud of the metal forging standard part; repeatedly iterate and optimize the registration results. In each iteration, the position and posture of the point cloud of the metal forging test piece are updated by minimizing the mean squared error (MSE) between the point clouds until the registration error converges.

[0052] S33. Precise alignment based on ICP (Iterative Closest Point): Use the KD-Tree structure of PCL to quickly find the point closest to the metal forging test piece point cloud and the metal forging standard part point cloud; calculate the error between each pair of nearest neighbor points and minimize these errors. After each iteration, update the position and rotation angle of the metal forging test piece point cloud to achieve the best alignment with the metal forging standard part point cloud. When the error change is less than the preset threshold or the maximum number of iterations is reached, terminate the iteration.

[0053] Furthermore, after point cloud registration is completed, feature extraction and matching are performed.

[0054] Specifically, the FPFH algorithm is used to extract the local geometric features of the metal forging test piece point cloud, generate the FPFH feature vector, generate the feature vector describing the point cloud features through neighborhood calculation and normal statistics, and calculate the covariance matrix of the metal forging test piece point cloud feature vector; read the FPFH feature vector and covariance matrix of the metal forging standard parts from the database, and use the cosine similarity measurement method to calculate the similarity between the metal forging test piece features and the metal forging standard parts features. The cosine similarity formula is as follows:

[0055]

[0056] Among them, A and B are FPFH feature vectors;

[0057] Within the defined range, the calibration point features are matched, the most appropriate point is found in the point cloud of the metal forging test piece, and stored in the database.

[0058] Use PCL's KD-Tree structure to perform nearest neighbor search. For a pair of points, such as point A and point B, select the 10 points with the smallest distance near point A and the 10 points with the most similar features to point A to form the point set ∝A; select the 10 points with the smallest distance near point B and the 10 points with the most similar features to point B to form the point set βB , each point set contains twenty selected points.

[0059] Furthermore, distance calculation and whether the workpiece is qualified are performed.

[0060] Specifically, calculate the point set ∝ A Any point and point set β B The Euclidean distance between any points in the forging is recorded and compared with the standard distance between the corresponding point pairs of the metal forging standard parts. If all the calculated distances are within the allowable range, it is judged as a qualified product, otherwise it is a failed product.

[0061] Preferably, the average value and standard deviation of all point-pair distances are calculated, and these statistical results are recorded, stored in a database, and a detailed detection report is generated.

[0062] On the other hand, if the metal forging test piece is a high-temperature piece, when the metal forging test piece is photographed under high temperature conditions for 3D modeling, the generated point cloud will deviate from its actual size due to the thermal expansion effect of the metal forging test piece. In order to correct this deviation, the point cloud needs to be scaled and adjusted according to the thermal expansion coefficient of the material, which includes the following calculation steps and processes:

[0063] The thermal expansion coefficient α of the metal forging test piece material is known (unit: °C -1 ), the temperature T of the test piece when it is photographed hot (Unit: °C), reference temperature T of the test piece ref (Unit: °C);

[0064] Use the camera to shoot the high temperature test piece and generate the initial point cloud P hot (x hot ,y hot , z hot ), perform noise reduction and filtering operations on the point cloud file and save the data;

[0065] Calculate the temperature difference:

[0066] △T=T hot -T ref ;

[0067] Calculate the expansion coefficient:

[0068] γ=1+α·△T;

[0069] Adjust the initial point cloud to convert the point cloud at high temperature into the point cloud at reference temperature:

[0070]

[0071] Point-by-point processing, that is, the following calculation is performed for each point:

[0072]

[0073] Furthermore, after completing the above operations, continue with the steps of point cloud registration, feature extraction and matching, distance calculation and qualification judgment.

[0074] The S4 includes: outputting and storing the measurement data, and generating a test report to record the measurement data for inspection.

[0075] Specifically, the distance between points of the metal forging test piece, the distance interval and the comparison results with the standard value are stored in a database, and the database includes the metal forging test piece ID, measurement time, measurement results and detailed data fields.

[0076] Furthermore, a test report is generated to record in detail the test process, raw data, calculation results, comparative analysis and final judgment results. The report format includes charts, data tables and text descriptions to ensure that the information is clear and easy to read.

[0077] In another aspect of the present invention, a metal forging size detection system is provided, the system comprising:

[0078] Camera calibration module, completes camera calibration, hand-eye calibration, and turntable calibration;

[0079] Metal forging standard parts processing system, which scans and 3D models metal forging standard parts. The process includes auxiliary positioning and marking, scanning and modeling, point cloud processing, point selection and feature extraction, and database storage data;

[0080] Metal forging test piece processing system, which scans and 3D models metal forging test pieces. The process includes 3D scanning and data storage, point cloud processing, point cloud registration, feature extraction and matching, distance calculation and qualification judgment;

[0081] The data output and storage module generates a test report to record the measurement data for inspection.

[0082] Furthermore, the camera calibration procedure in the system includes camera intrinsic and extrinsic parameter calibration and optimization calculation, wherein a multi-calibration plate auxiliary system is used to optimize the calibration accuracy.

[0083] Furthermore, the calibration procedure in the system also includes hand-eye calibration, turntable calibration, intelligent error feedback and automatic adjustment.

[0084] Furthermore, in the system, the data of standard metal forging parts collected by the camera are processed into three-dimensional point clouds and displayed based on the helix.toolki three-dimensional visualization framework; the point cloud files are denoised and filtered using filters in the PCL point cloud library; the local geometric features of each selected point are extracted using the FPFH algorithm and the covariance matrix of each selected point is calculated and stored in the database as benchmark data for standard metal forging parts.

[0085] Furthermore, in the system, the point cloud registration of the metal forging test piece uses PCA, GICP and ICP technology to perform multi-level registration to accurately align the point cloud of the metal forging test piece with the point cloud of the standard part.

[0086] Furthermore, in the system, after the point cloud registration is completed, the features of the calibration points are compared to find the most suitable point, and the 10 points with the smallest distance and the 10 points with the most similar features are selected around the point to form a point set α, and 20 points under the same rules around the points that have a corresponding relationship with the point are found to form a point set β, and the Euclidean distance between any two points in the two point sets is calculated and compared with the standard value. If it is within the allowed range, it is judged as a qualified product, otherwise it is an unqualified product.

[0087] Furthermore, in the system, if the metal forging test piece is a high-temperature piece, the thermal expansion effect causes the generated point cloud to deviate from its actual size, and the point cloud is scaled and adjusted according to the thermal expansion coefficient of the material to correct this deviation.

[0088] According to the above content, the present invention has the following advantages and beneficial effects:

[0089] 1. Integrate the intelligent error feedback system and the multi-calibration plate auxiliary system in the camera calibration process, thereby significantly improving the efficiency and accuracy of calibration. The intelligent error feedback system reduces human intervention and repeated operations by analyzing and correcting calibration errors in real time, ensuring calibration accuracy in complex operating environments; the multi-calibration plate auxiliary system can obtain redundant calibration data from multiple perspectives, and cross-validate by combining the information of multiple calibration plates, eliminating the error accumulation caused by a single perspective, ensuring the robustness of the calibration process system, and improving calibration accuracy.

[0090] 2. Combine the color-coded patch-assisted point selection method with the SIFT algorithm for automatic point selection based on geometric surface features to achieve flexible and accurate point selection under complex conditions. The color-coded patch method provides intuitive and high-contrast identification on uniform surfaces, which facilitates manual point selection and rapid processing; while the SIFT algorithm automatically extracts complex geometric surface features to achieve high-precision point selection of "edge points" and "corner points" without manual operation. The combination of the two not only improves the accuracy and efficiency of overall point selection, but also enhances the system's applicability to complex structural metal forgings.

[0091] 3. When aligning the point cloud, PCA technology is used to analyze the main direction of the point cloud to roughly locate the object and achieve rapid preliminary alignment; GICP technology is used to further optimize local matching based on preliminary alignment to deal with local errors on complex surfaces; ICP technology is used to fine-tune the distance between point clouds to ensure global and precise alignment of the two. This multi-level alignment process gives full play to the advantages of each algorithm, effectively improves the point cloud alignment accuracy and system robustness, and is suitable for high-precision point cloud alignment application scenarios with complex geometric structures of metal forgings.

[0092] 4. Find the calibration point A on the point cloud of the metal forging test piece, and find the 10 points closest to the calibration point A and the 10 points with the most similar eigenvalues ​​to the calibration point A, and form the point set ∝A together. Then find 20 points with the same regularity near another calibration point B that has a corresponding relationship with the calibration point A to form the point set β B , calculate the point set ∝ A Any point and point set β B The Euclidean distance between any points in the sample is calculated to compare the calculated results with the standard value interval to detect whether the size of the metal forging is qualified. This method reduces the impact of local anomalies or noise on the results, improves the robustness of the system, and makes the detection results more authoritative and credible; at the same time, through the comprehensive analysis of multiple groups of point pairs, the results are more comprehensive, ensuring that the size detection results of complex-shaped metal forgings are more reliable.

[0093] 5. By adopting temperature compensation model and non-contact scanning technology, the detection error problem caused by shape change of metal forgings due to high-temperature thermal expansion is effectively solved, ensuring that the 3D point cloud model generated in a high-temperature environment still has high precision, broadening the application scenarios of 3D scanning modeling and detection, and making the system suitable for various harsh industrial environments including hot processing measurement of motor vehicle metal forgings. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] Figure 1 The figure is a flow chart of the steps of the metal forging size detection method of the present invention.

[0095] Figure 2 It is the overall structural block diagram of the metal forging size detection system of the present invention. DETAILED DESCRIPTION

[0096] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0097] The present invention provides Figure 1A method for detecting the size of a metal forging is shown, comprising the following steps:

[0098] S1, 3D modeling camera calibration;

[0099] S2, metal forging standard parts processing, scanning and 3D modeling of standard parts, including auxiliary positioning and marking, scanning and modeling, point cloud processing, point selection and feature extraction, and database storage data;

[0100] S3, metal forging test piece processing, scanning and 3D modeling of the test piece, including 3D scanning and data storage, point cloud processing, point cloud registration, feature extraction and matching, distance calculation and qualification judgment;

[0101] S4, output the measurement data and store it, generate a test report to record the measurement data for inspection;

[0102] In this embodiment, in step S1, the hardware equipment used in the calibration scheme includes an industrial-grade binocular camera, a robotic arm, a turntable, and a black and white checkerboard calibration board. The calibration steps are as follows:

[0103] S11. Equipment inspection and preparation: Install the binocular industrial camera firmly on the end tool of the robot arm, ensure that its field of view covers the area to be scanned, and use anti-entanglement flexible cables to avoid interfering with the movement of the robot arm; the turntable is placed horizontally and fixed on a stable table to ensure that it does not shake during rotation; the calibration plate is installed in a position relative to the turntable and the camera field of view to ensure clear visibility when shooting at all angles; check and calibrate the position sensor on the robot arm, the angle encoder of the turntable, and the force / torque sensor to ensure that the sensor works accurately. All equipment is connected to the central control system to ensure stable transmission of power and signal lines, and reserve space for the full range of movement of the robot arm.

[0104] Software and intelligent system preparation: Start the calibration software, load the automated calibration process, ensure that the software has integrated the intelligent error feedback system and the multi-calibration board auxiliary system, set the error threshold in the calibration software interface, configure the error detection standard of the intelligent feedback system, ensure real-time monitoring and alarm, connect the control system of the camera and the robotic arm, and ensure that the automated control function is available.

[0105] S12, camera internal and external parameter calibration, including the following steps:

[0106] (1) Integration of multi-calibration plate auxiliary system: A multi-calibration plate auxiliary system is used to place multiple black and white grid calibration plates around the robot arm and turntable to ensure that the calibration plates cover different orientations and heights, thereby providing the camera with image data from multiple perspectives. The multi-calibration plate auxiliary system obtains redundant calibration data from multiple perspectives and cross-validates by combining the information from multiple calibration plates to eliminate the error accumulation caused by a single perspective, thereby ensuring the robustness of the calibration process system and improving the calibration accuracy.

[0107] Robotic arm motion planning: Program the robotic arm to move to different positions along a preset path and automatically capture multi-view images of the calibration plate.

[0108] (2) Image acquisition and intelligent error feedback: Intelligent image quality inspection. After the camera acquires multiple images of the calibration plate, the intelligent error feedback system checks the image clarity, distortion, and corner detection accuracy in real time. If blur or large corner extraction errors are detected, the system automatically adjusts the camera's focus and exposure settings, or prompts the operator to retake the image.

[0109] Set a reprojection error threshold. If the reprojection error in the image exceeds the set range, the system will issue a warning through the user interface and provide adjustment suggestions.

[0110] (3) Internal parameter calculation and optimization: Zhang's method is used to process multiple calibration plate images to calculate the camera's internal parameters, including focal length, principal point position, distortion coefficient and other parameters.

[0111] Error optimization: Combined with the error feedback system, the Levenberg-Marquardt global optimization algorithm is applied to optimize the camera intrinsic parameters to further reduce the reprojection error.

[0112] (4) External parameter calculation and optimization: Use the calibration algorithm to extract external parameter data from multiple images of the calibration plate. The data includes the rotation matrix and displacement vector of the camera to represent the position and direction of the camera in the world coordinate system. With the help of the multi-calibration plate auxiliary system, redundant data is obtained from different positions and angles to reduce the deviation of external parameter calculation caused by shooting errors at a certain angle. The system integrates the external parameter information of multiple calibration plates for cross-validation to improve the accuracy of external parameter calibration.

[0113] S13. Hand-eye calibration: Robotic arm motion path planning: Design the motion path of the robotic arm so that it can shoot the calibration plate from multiple spatial positions and angles. The path planning is automatically completed by the system to ensure that the position information of each posture is complete and evenly covered.

[0114] Real-time error feedback and adjustment: The intelligent error feedback system monitors the position and motion trajectory of the end of the robot arm. When any path deviation or joint jitter is detected, the system automatically adjusts the motion path of the robot arm to ensure that the end tool remains in the accurate position.

[0115] S14, turntable calibration, including the following steps:

[0116] (1) Automatic rotation and image acquisition: Fix the calibration plate on the turntable, start the turntable to rotate automatically, rotate at fixed angle increments, and capture an image of the calibration plate with each rotation. The intelligent error feedback system monitors the rotation accuracy in real time through the turntable's angle encoder. If the rotation angle does not match the preset, the system will adjust the turntable speed or angle to ensure that the calibration plate is always in the best shooting position.

[0117] (2) Accurate calculation of the rotation axis and center, corner point detection and geometric derivation: The rotation axis and center of the turntable are calculated using the geometric derivation method through the corner points of the calibration plate extracted from each image. Combined with the detection results of the intelligent feedback system, the system automatically optimizes the calculation of the rotation axis and center position to ensure accurate results.

[0118] S15, perform comprehensive calibration and error optimization, including the following:

[0119] Multi-module calibration data fusion: Integrate the results of camera intrinsic parameters, hand-eye calibration and turntable calibration to generate a unified coordinate system and model for the entire system; the system applies a global optimization algorithm to optimize the relative position relationship between modules to further reduce system errors.

[0120] Intelligent error feedback and automatic adjustment: The integrated error feedback system analyzes the error sources in each calibration data step in real time. If systematic errors are detected during the calibration process, the system will automatically provide adjustment suggestions or re-execute some calibration steps. Based on the error analysis results, the system can automatically adjust the motion trajectory of the robot arm, the rotation speed of the turntable or the parameters of the camera to improve the overall calibration accuracy.

[0121] S16. Summary and Report: The system automatically generates a report containing detailed information about the calibration process, including camera intrinsic parameters, hand-eye transformation matrix, turntable rotation axis data, error analysis results, and optimization strategies. The report is stored in the cloud and can be used for reference and analysis by other devices. Through long-term error monitoring and feedback, the system can automatically remind you when to perform calibration again according to the situation, ensuring that the equipment is in the best working condition for a long time.

[0122] In this embodiment, in step S2, after completing the camera calibration, scanning and three-dimensional modeling of the standard part are started.

[0123] It should be specifically explained in this embodiment that this process includes the steps of auxiliary positioning and marking, scanning and modeling, point cloud processing, point selection and feature extraction, and database storage data.

[0124] The auxiliary positioning marking adopts a color-coded patch method, and color-coded patches are pasted on the workpiece surface where the marking points need to be marked, so that these points are easier to be recognized by the camera during scanning, and thus more prominent in the three-dimensional model. This method is simple and easy to implement, the markings are clearly visible, and will not affect the physical properties of the workpiece.

[0125] The scanning and modeling process includes starting the calibrated camera system, placing the standard parts on the shelf on the turntable, and performing multi-angle scanning on the standard parts after the points are attached. The camera is required to obtain the three-dimensional data of the standard parts by scanning multiple times from at least three different angles and cooperating with the turntable to ensure that all the attachment points are covered; based on the helix.toolki three-dimensional visualization framework, the data collected by the camera is processed into a three-dimensional point cloud and displayed, and the generated point cloud is saved in an xyz file format, where each line represents the coordinates (x, y, z) of a three-dimensional point.

[0126] The point cloud processing uses filters in the PCL (Point Cloud Library) point cloud library to reduce noise and filter the point cloud file, and saves the denoised and filtered point cloud data as a PCD file and an updated xyz file for subsequent use.

[0127] The point selection uses three-dimensional visualization software, and the point position in the point cloud is selected by mouse click to ensure that the selected point is as consistent as possible with the actual point position. The nearest point search is performed using the KD-Tree structure of PCL. For each selected point, the point with the shortest distance is searched within a preset radius, and its coordinates are recorded and saved as a matching point. The system integrates the SIFT (Scale-Invariant Feature Transform) algorithm, and automatically extracts key points without manual intervention by analyzing the geometric surface features of the workpiece, such as corners, edges, protrusions, etc., and combines it with the point sticking to perform intelligent and precise point selection.

[0128] It should be specifically explained in this embodiment that after the point selection is completed, every two points that need to measure the distance are listed as a group, the Euclidean distance between the two points is calculated, and the distance is recorded as the calibration value. If points A and B are a group, the calculation formula is as follows:

[0129]

[0130] Set the allowable deviation range for each group of points, such as ±0.1 mm.

[0131] The corresponding relationship of each group of points, the standard distance value on the drawing, the actual calibration value and the set deviation range are stored in the database respectively.

[0132] The feature extraction uses the FPFH (Fast Point Feature Histograms) algorithm in the PCL (Point Cloud Library) point cloud library. For each selected point, it searches for its neighboring points within a certain radius, calculates the normal direction of these points, performs statistical analysis on the normals of the neighboring points, calculates the angle distribution between the normal of the center point, and forms a feature vector to extract the local geometric features of each selected point; calculates the covariance matrix of the FPFH feature vector of each selected point to describe the correlation between features. The covariance matrix is ​​defined as:

[0133]

[0134] Among them, f i is the feature vector, is the mean of the eigenvector, n is the number of neighborhood points, and T is the transposed sign.

[0135] The eigenvectors and covariance matrices are stored in the database as the benchmark data of the selected points of the standard parts.

[0136] In this embodiment, in step S3, the metal forging test piece is processed, and the test piece is scanned and three-dimensionally modeled. The process includes three-dimensional scanning and data storage, point cloud processing, point cloud alignment, feature extraction and matching, distance calculation and qualification judgment.

[0137] It should be specifically explained in this embodiment that, on the one hand, if the test piece is a normal temperature test piece, the processing flow is as follows:

[0138] The three-dimensional scanning and data preservation process includes starting the calibrated camera system, placing the test piece on a rack on a turntable, and scanning the test piece at multiple angles. The camera is required to obtain three-dimensional data of the test piece by scanning multiple times from at least three different angles and in conjunction with the rotation of the turntable to ensure that the required measurement range is covered; based on the helix.toolki three-dimensional visualization framework, the data collected by the camera is processed into a three-dimensional point cloud and displayed, and the generated point cloud is saved in an xyz file format, where each row represents the coordinates (x, y, z) of a three-dimensional point.

[0139] The point cloud processing uses filters in the PCL (Point Cloud Library) point cloud library to reduce noise and filter the point cloud file, and saves the denoised and filtered point cloud data as a PCD file and an updated xyz file for subsequent use.

[0140] The point cloud registration uses PCA, GICP and ICP technology to perform multi-level registration to accurately align the point cloud of the test piece with the point cloud of the standard piece. The process includes the following steps:

[0141] S31. Perform preliminary alignment based on PCA: Calculate the principal component axes of the test piece point cloud. The principal component axes are the directions of maximum variance of the point cloud data, corresponding to the X, Y, and Z axes. Obtain the main directions of the point cloud through principal component analysis and align them with the principal axes of the standard part. Calculate the projection of each point on the principal component axes to form a new coordinate system so that the long axis of the point cloud is aligned with the long axis of the standard part, eliminating most of the initial posture differences.

[0142] S32. GICP-based refined registration: Use the GICP algorithm to approximate the point set near each point as a plane, establish a local plane model, and establish the matching of local geometric structures by calculating the normals and normal distances of these planes; reduce the overall registration error by calculating the local plane relationship between the test piece point cloud and the neighborhood points of the standard part point cloud; repeatedly iterate and optimize the registration results. During each iteration, the position and posture of the test piece point cloud are updated by minimizing the mean square error function between the point clouds until the registration error converges.

[0143] S33, ICP-based precise registration: Use PCL's KD-Tree structure to quickly find the point in the test piece point cloud that is closest to the standard part point cloud; calculate the error between each pair of nearest neighbor points and minimize these errors. After each iteration, update the position and rotation angle of the test piece point cloud to achieve optimal alignment with the standard part point cloud; terminate the iteration when the error change is less than the preset threshold or the maximum number of iterations is reached.

[0144] After point cloud registration is completed, feature extraction and matching are performed.

[0145] The feature extraction and comparison uses the FPFH algorithm to extract the local geometric features of the test piece point cloud, generates the FPFH feature vector, generates the feature vector describing the point cloud features through neighborhood calculation and normal statistics; calculates the covariance matrix of the test piece point cloud feature vector for subsequent feature comparison; reads the FPFH feature vector and covariance matrix of the standard part from the database, and uses the cosine similarity measurement method to calculate the similarity between the test piece feature and the standard part feature. The cosine similarity formula is as follows:

[0146]

[0147] Among them, A and B are FPFH feature vectors;

[0148] Within the designated range, compare the calibration point features, find the most suitable point in the test piece point cloud, and store it in the database.

[0149] Use PCL's KD-Tree structure to perform nearest neighbor search. For a pair of points that need to measure distance, such as point A and point B, select the 10 points with the smallest distance near point A and the 10 points with the most similar features to point A to form the point set ∝A; select the 10 points with the smallest distance near point B and the 10 points with the most similar features to point B to form the point set β B , each point set contains twenty selected points.

[0150] After feature extraction and comparison are completed, distance calculation and qualification judgment are performed.

[0151] Calculate the point set ∝ A Any point and point set β B The Euclidean distance between any points in the forging is recorded and compared with the standard distance between the corresponding point pairs of the metal forging standard parts. If all the calculated distances are within the allowable range, it is judged as a qualified product, otherwise it is a failed product.

[0152] Calculate the mean and standard deviation of the point-pair distances, record these statistical results, store the results and whether they are qualified in the database, and generate a detailed test report.

[0153] It should be further explained in this embodiment that, on the other hand, if the metal forging test piece is a high-temperature piece, when the test piece is photographed under high temperature conditions for 3D modeling, the generated point cloud will deviate from its actual size due to the thermal expansion effect of the test piece. In order to correct this deviation, the point cloud needs to be scaled and adjusted according to the thermal expansion coefficient of the material, which includes the following calculation steps and processes:

[0154] The thermal expansion coefficient α of the metal forging test piece material is known (unit: °C -1 ), the temperature T of the test piece when it is photographed hot (Unit: °C), reference temperature T of the test piece ref (Unit: °C);

[0155] Use a camera to shoot the high-temperature metal forging test piece to generate the initial point cloud P hot (x hot ,y hot , z hot ), perform noise reduction and filtering operations on the point cloud file and save the data;

[0156] Calculate the temperature difference:

[0157] △T=T hot -T ref ;

[0158] Calculate the expansion coefficient:

[0159] γ=1+α·△T;

[0160] Adjust the initial point cloud to convert the point cloud at high temperature into the point cloud at reference temperature:

[0161]

[0162] Point-by-point processing, that is, the following calculation is performed for each point:

[0163]

[0164] After completing the above operations, continue with the point cloud registration, feature extraction and matching, distance calculation and qualification judgment steps.

[0165] In this embodiment, in step S4, the measurement data is output and stored, and a test report is generated to record the measurement data for inspection.

[0166] The point distances and distance intervals of the metal forging test piece and the comparison results with the standard values ​​are stored in a database, and the database includes the metal forging test piece ID, measurement time, measurement results, and detailed data fields.

[0167] Generate a test report that records the test process, raw data, calculation results, comparison analysis and final judgment results in detail. The report format includes charts, data tables and text descriptions to ensure that the information is clear and easy to read.

[0168] In this embodiment, Figure 2 A metal forging size detection system, the system comprising:

[0169] Camera calibration module, completes camera calibration, hand-eye calibration, and turntable calibration;

[0170] Metal forging standard parts processing system, which scans and 3D models standard parts. The process includes auxiliary positioning and marking, scanning and modeling, point cloud processing, point selection and feature extraction, and database storage data.

[0171] Metal forging test piece processing system, which scans and 3D models the test pieces. The process includes 3D scanning and data storage, point cloud processing, point cloud registration, feature extraction and matching, distance calculation and qualification judgment;

[0172] The data output and storage module generates a test report to record the measurement data for inspection.

[0173] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting the size of a metal forging, characterized in that: The method comprises the following steps: S1, 3D modeling camera calibration; S2. Scan and 3D model standard metal forging parts, including auxiliary positioning and marking, scanning and modeling, point cloud processing, point selection and feature extraction, and database storage data; S3, scanning and 3D modeling of metal forging test pieces, including 3D scanning and data storage, point cloud processing, point cloud registration, feature extraction and matching, distance calculation and qualification judgment; S4. Output the measured data and store it, and generate a test report to record the measured data for inspection.

2. The method according to claim 1, characterized in that In step S1, a multi-calibration plate auxiliary system is integrated during the calibration of the camera's intrinsic and extrinsic parameters: by placing multiple black and white grid calibration plates around the robotic arm and turntable, the camera is provided with image data from multiple perspectives. The system processes redundant calibration data for cross-validation, eliminates error accumulation caused by a single perspective, and ensures the robustness and accuracy of the calibration process.

3. The method according to claim 1, characterized in that In step S1, the intelligent error feedback system is integrated and used in the camera internal and external parameter calibration, hand-eye calibration, and turntable calibration processes: in the camera internal and external parameter calibration, the intelligent error feedback system checks the image quality in real time. If there is a large difference in quality, the system automatically adjusts the camera parameters and applies the global optimization algorithm to optimize the camera internal and external parameters; in the hand-eye calibration, the intelligent error feedback system monitors the posture and motion trajectory of the end of the robot arm. When path deviation or joint jitter is detected, the system automatically adjusts the motion path of the robot arm to ensure that the end tool remains in the accurate position; in the turntable calibration, the intelligent error feedback system monitors the rotation accuracy in real time through the turntable's angle encoder, automatically optimizes the calculation of the rotation axis and center position, and ensures that the calculation results are accurate.

4. The method according to claim 1, characterized in that In step S2, when selecting points for standard parts, the color-coded patch-assisted point selection method is combined with the SIFT algorithm-based automatic point selection method based on geometric surface features: the color-coded patch method is used to assist in manual point selection on a uniform surface, and the SIFT algorithm is used to automatically extract complex geometric surface features to select edge points and corner points with high precision without manual operation.

5. The method according to claim 1, characterized in that In step S2, after completing the standard part point selection, the FPFH algorithm in the PCL point cloud library is used to calculate the feature vector of the point, extract the local geometric features of the point, and calculate the covariance matrix of the FPFH feature vector of the point to describe the correlation between the features. The covariance matrix is ​​defined as: Among them, f i is the feature vector, is the mean of the eigenvector, n is the number of neighborhood points, and T is the transposition sign; The eigenvectors and covariance matrices are stored in the database as the benchmark data of the selected points of the standard parts.

6. The method according to claim 1, characterized in that In step S3, after the test piece is scanned to generate a test piece point cloud, PCA, GICP and ICP techniques are used to perform multi-level registration to accurately align the point cloud of the test piece with the point cloud of the standard part.

7. The method according to claim 1, characterized in that In step S3, in feature extraction and matching point selection, the FPFH algorithm is used to extract the local geometric features of the test piece point cloud, generate the FPFH feature vector, and calculate the covariance matrix of the test piece point cloud feature vector; the FPFH feature vector and covariance matrix of the standard part are read from the database, and the cosine similarity measurement method is used to calculate the similarity between the test piece feature and the standard part feature. The cosine similarity formula is as follows: Among them, A and B are FPFH feature vectors; Within the demarcated range, the calibration point features are matched, the most suitable point is found in the test piece point cloud, and it is stored in the database as a confirmed point.

8. The method according to claim 1, characterized in that In step S3, when checking whether the Euclidean distance between the selected point pairs meets the standard to determine whether the product is qualified, if the pair of points is point A and point B, the 10 points with the smallest distance near point A and the 10 points with the most similar features to point A are selected to form a point set ∝ A ; Select the 10 points with the smallest distance near point B and the 10 points with the most similar features to point B to form the point set β B , each point set contains twenty selected points; calculate the point set ∝ A Any point in the set β B The Euclidean distance between any points in the sample is recorded and compared with the standard distance between the corresponding point pairs of the standard parts. If all the calculated distances are within the allowable error range, it is judged as a qualified product, otherwise it is a failed product. At the same time, the average value and standard deviation of the distances of all point pairs are calculated, and these statistical results are recorded. The results are stored in the database and a detailed test report is generated for subsequent inspection.

9. The method according to claim 1, characterized in that In step S3, if the test piece is a high-temperature piece, when the test piece is photographed under high temperature conditions for 3D modeling, the generated point cloud will deviate from its actual size due to the thermal expansion effect of the test piece. In order to correct this deviation, the point cloud needs to be scaled according to the thermal expansion coefficient of the material, which includes the following calculation steps and processes: Know the thermal expansion coefficient α of the test piece material and the temperature T of the test piece when it is photographed hot , reference temperature T of the test piece ref ; Use the camera to shoot the high temperature test piece and generate the initial point cloud P hot (x hot ,y hot , z hot ), perform noise reduction and filtering operations on the point cloud file and save the data; Calculate the temperature difference: △T=T hot -T ref ; Calculate the expansion coefficient: γ=1+α·△T; Adjust the initial point cloud to convert the point cloud at high temperature into the point cloud at reference temperature: Point-by-point processing, that is, the following calculation is performed for each point: After completing the above operations, continue with the point cloud registration, feature extraction and matching, distance calculation and qualification judgment steps.

10. A metal forging size detection system, the system is used to implement the method described in any one of claims 1 to 9, characterized in that: The system comprises the following parts: Camera calibration module, completes camera calibration, hand-eye calibration, and turntable calibration; Metal forging standard parts processing system, which scans and 3D models standard parts. The process includes auxiliary positioning and marking, scanning and modeling, point cloud processing, point selection and feature extraction, and database storage data; Metal forging test piece processing system, which scans and 3D models the test pieces. The process includes 3D scanning and data storage, point cloud processing, point cloud registration, feature extraction and matching, distance calculation and qualification judgment; Data output and storage module generates reports to record measurement data for inspection.

Images