Robot gear tray loading method based on 3D vision

Abstract

The invention relates to a robot gear disc loading method based on 3D vision, and belongs to the technical field of industrial robot automation. The method comprises the steps that connection between a 3D camera and a robot is established, hand-eye calibration is completed, a template library of a gear type circular workpiece and a template library of a material disc type non-circular workpiece are established respectively, template matching pose output is carried out, and the robot gear disc loading precision is improved; outputting a high-precision pose matrix of the workpiece; and the grabbing points are optimally sorted according to the running direction of the robot, the placing points of the gears are sorted according to the preset sequence of the teaching points of the robot, and regular visual correction is executed to maintain the long-term precision of the system. The problems of low efficiency, error proneness, poor flexibility and the like of traditional manual tray loading are solved, high precision, high rhythm and full automation of multi-model gear tray loading operation are achieved, and the intelligent level and stability of a production line are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of automation technology, and more specifically to a method for mounting robot gears based on 3D vision. Background Technology

[0002] Currently, both domestically and internationally, the heat treatment process for small and medium-sized gears (reduction gearboxes) relies on manual loading and unloading of trays. This traditional manual operation method has the following drawbacks: First, production efficiency is low and labor costs are high. The heat treatment production line involves a wide variety of gear models, and operators need to perform a lot of identification, marking, and pre-assembly work before loading the gears onto trays. This cumbersome operation results in a slow overall processing speed, which cannot meet the needs of modern, efficient production.

[0003] Secondly, it has poor reliability and is prone to errors. Relying entirely on human visual identification and manual placement, it is extremely easy to make operational errors such as misplacing or misplacing items under fatigue or high-intensity working pressure, resulting in mixed workpieces. This not only directly affects the quality of heat treatment but may also cause batch product scrapping, leading to economic losses.

[0004] Furthermore, there is a lack of adaptability and flexibility. Faced with the trend of multi-variety, small-batch production, the manual mode struggles to quickly and accurately adapt to the matching requirements of different gear models and corresponding material trays. Each product change requires relearning and adaptation, severely restricting the flexibility of the production line and preventing rapid production changeovers.

[0005] Although industrial robots and vision technology have been applied in many fields, directly introducing conventional automation solutions in the specific scenario of gear heat treatment and pallet loading still faces challenges. For example, traditional robot loading and unloading usually rely on precision tooling fixtures and fixed programming paths, which cannot intelligently handle gears and pallets with various irregular shapes, nor can they compensate for workpiece placement deviations and long-term accuracy drift of the system.

[0006] In summary, there is an urgent need for a 3D vision-based robotic gear loading method that can efficiently, accurately, and reliably complete automated gear loading for various models, and maintain stable operating rhythm and accuracy over a long period of time. Summary of the Invention

[0007] The purpose of this invention is to provide a 3D vision-based robotic gear loading method that can efficiently, accurately, and reliably complete automated gear loading for various types of gears, and maintain a stable operating rhythm and accuracy over a long period of time.

[0008] The above objective is achieved through the following technical solution: a robot gear assembly method based on 3D vision, comprising the following steps: S1, establish the connection between the 3D vision camera and the robot; S2, Calibration of the 3D Vision Camera; S3, Workpiece Template Establishment: Collect standard images of non-circular workpieces such as trays and circular workpieces such as gears, preprocess the 3D point cloud data of the collected standard images, extract key features, and construct tray template libraries and gear template libraries respectively. S4, Template Matching Pose Output: During the production process, on-site scene images are acquired, and instances of non-circular workpieces such as material trays and circular workpieces such as gears are segmented from the scene point cloud. After the instances are initially aligned with the templates in the template library, registration is performed. The matching error is iteratively optimized to a preset threshold, and the pose matrix of the workpiece is output and stored. S5, Pose adjustment and sorting of gripping points: Extract the coordinates of gripping points of multiple gears, and sort the coordinate values ​​of gripping points in ascending or descending order according to the robot's preset running direction to determine the robot's gripping order. S6, Sorting of placement points: Sort the placement points of the gears according to the preset order of the robot teaching points, and record the storage information after the gears are placed in place; S7, regular visual calibration to maintain traying accuracy.

[0009] This invention replaces traditional manual labor with an automated system, avoiding problems such as manual marking, pre-assembly, and error-prone processes. Furthermore, through an optimized grasping point sorting algorithm, it plans the robot's most efficient movement path, significantly shortening the cycle time and ensuring a stable production pace far exceeding manual speed. A periodic visual correction mechanism automatically detects and compensates for accuracy drift caused by mechanical wear, thermal deformation, and other factors during production breaks. This ensures the system maintains its initial high accuracy even under long-term, continuous operation, achieving "preventive maintenance." This reduces batch quality incidents and equipment downtime caused by accuracy inaccuracies, resulting in low maintenance costs and no disruption to production time.

[0010] A further technical solution is that step S3 specifically includes the following steps: S31, Collect standard images of non-circular workpieces such as trays and circular workpieces such as gears; S32 processes the 3D point cloud data of the workpiece standard image according to the vision engineering process. Through data preprocessing, the point cloud quality is improved. At the same time, noise is eliminated and key features are extracted to ensure the accuracy of template matching. S33, extract gear or tray features, establish a workpiece template library, edit the templates, and define relevant variables including boundaries, matching thresholds, center points, and gripping points.

[0011] Thus, through a complete set of data processing, feature extraction, and model building processes, highly standardized, reusable, and accurate "visual knowledge" is provided for the entire automated tray loading system. This is the core technical guarantee that enables the invention to have outstanding advantages such as high precision, high flexibility, high efficiency, and strong robustness.

[0012] A further technical solution is that, in step S33, the key feature extraction process for establishing the gear-type circular workpiece template library includes: Principal component analysis is performed on the neighborhood of each point in the gear point cloud to obtain the direction corresponding to the minimum eigenvalue as the normal vector of the point, so as to reflect the orientation of the gear surface; By calculating the Gaussian curvature or average curvature of neighboring points, the characteristic regions of the gear, including the tooth tip and tooth root, can be identified. The least squares method was used to fit a circle to the point cloud of the gear end face, and the error function was established and the center coordinates and radius of the circle were optimized. Calculate the point-pair feature histogram to describe the geometric properties of the gear, including tooth profile angle and tooth pitch, and build a gear template library based on the above features.

[0013] This approach achieves a complete and robust description of complex gear geometry: a hybrid feature model combining macroscopic geometry (center, radius), mesoscopic regions (tooth tip, tooth root), and microscopic features (tooth profile PFH) allows the template to comprehensively represent a gear. This significantly enhances the template's discriminative power and robustness. Even with partial occlusion or slight adhesion to the background and other gears, the system can accurately identify and match based on rich feature information. Precise normal vector and curvature information provide high-quality initial corresponding point pairs for the subsequent ICP fine registration algorithm, guiding the registration process to quickly converge to the correct position. The least-squares fitted circle provides a stable and accurate grasping reference center, overcoming the center point drift problem caused by occlusion or defects of individual teeth, ensuring the long-term repeatability accuracy of the robot's grasping pose. It also enhances the ability to distinguish subtle differences in gear models; different gear models may have subtle differences in tooth profile and pitch, which cannot be described by a simple center and radius. Microscopic features can keenly capture these differences, enabling the template library to effectively distinguish gear models with similar appearances, fundamentally preventing the risk of picking the wrong model on a mixed production line, and achieving true intelligence.

[0014] A further technical solution is that, in step S33, the key feature extraction process for establishing the template library of non-circular workpieces of the material tray type includes: Based on the consistency between the normal vector and curvature of the point cloud, the point cloud of the material tray is divided into different regions including the bottom surface and side walls; Extract the point cloud of the outer contour of the tray and generate a closed polygon; Fit a rectangle to the point cloud of the outer contour of the material tray, and calculate the length, width and diagonal length of the rectangle; Extract the global aspect ratio features of the material trays and construct a material tray template library based on the region division results and geometric parameters.

[0015] In this way, stable identification and precise positioning of irregular workpieces are achieved. By clearly distinguishing the bottom surface and sidewalls through region segmentation, the system can accurately calculate the true posture of the workpiece tray in three-dimensional space (e.g., whether it is horizontal). Combining the length, width, and angle parameters obtained from fitting a rectangle to the outer contour, a precise pose matrix containing position and orientation can be output. This ensures that the robot can accurately place the gear directly above the designated slot on the workpiece tray, avoiding placement collisions or jamming caused by inaccurate workpiece tray pose estimation. When the production line needs to switch workpiece trays, the system can automatically call the correct workpiece tray template by quickly matching these features and pair it with the corresponding gear template, greatly enhancing the flexibility of the production line and effectively preventing mismatched gear and workpiece tray loading errors. The stable features extracted by this method provide a reliable environmental model for the robot to plan collision-free, high-precision placement paths, which is a technical prerequisite for achieving precise docking of gears and workpiece tray slots.

[0016] A further technical solution is that, in step S32, the step of preprocessing the 3D point cloud data of the image includes: Denoising and filtering: Based on the local density of the point cloud, isolated points that deviate from the point cloud mean by more than a preset threshold are removed, and outliers caused by sensor noise or reflection are eliminated. While smoothing the point cloud data, the edge features of the workpiece are preserved, and key structures, including gear teeth and the boundary line between the side wall and bottom of the tray, are avoided from being overly blurred. Downsampling: Divide the 3D space into cubic voxels of a preset size, and replace the original points with the centroids of all points within each voxel, reducing the amount of point cloud data while maintaining the overall shape of the workpiece; Invalid point filtering: Delete invalid points with non-numerical coordinates to avoid missing data caused by reflection or occlusion interfering with subsequent feature extraction.

[0017] Thus, through preprocessing, the system effectively overcomes the problems of noise, glare, and data loss commonly faced by 3D vision sensors in real industrial environments. This enables the method of the present invention to not only work under ideal laboratory conditions but also adapt to the complex lighting and workpiece surface conditions in real workshop environments, exhibiting strong environmental adaptability and operational stability, and reducing the risk of production line interruptions due to visual recognition failures.

[0018] A further technical solution is that, in step S4, the template and the instance are initially aligned based on geometric features, and the ICP algorithm is used for registration. The specific process is as follows: after the template point cloud and the workpiece instance point cloud are initially aligned, the distance error between corresponding points between the two point clouds is iteratively calculated, and the pose of the instance point cloud is adjusted until the error is less than a preset threshold to complete the fine registration. The pose optimization adopts the least squares method, and the pose parameters of the workpiece instance are adjusted by minimizing the error function.

[0019] Through iterative optimization, the ICP algorithm can effectively compensate for deviations caused by workpiece manufacturing tolerances, placement tilt, and residual errors from coarse matching. The final output pose matrix has extremely high accuracy, providing the robot with absolute position and orientation information sufficient for precise grasping and reliable placement. It transforms the results of previous steps (preprocessing, template establishment, and coarse matching) into precise spatial commands that can be executed by the robot, which is the decisive link in achieving the goals of high precision, high stability, and fully automated operation in the entire technical solution.

[0020] A further technical solution is that, in step S5, the sorting key value of each grasping point is calculated based on the robot's running direction vector, and the key values ​​are arranged in ascending or descending order.

[0021] A further technical solution is that step S5 specifically includes the following steps: Extract the coordinates (x) of the gripping points of multiple gears. i , y i , z i ), where i is the gear number, i≥1; Define the robot's running direction vector D=(d x , d y , d z ), where d x d y d z d represents the motion direction coefficients along the X, Y, and Z axes of the coordinate system, respectively. x dy , d z All ∈ {-1, 0, 1}; If the robot's direction of movement is X, then use formula k i =d x ×x i Calculate the sorting key value; if the robot's running direction is the Y direction, then use formula k. i =d y ×y i Calculate the sort key value; Sort by key value k i Arrange the gripping points in ascending or descending order to determine the robot's gripping sequence.

[0022] The sorting logic of this invention is decoupled from specific gear models and tray layouts, reducing the computational load on the robot control system. More importantly, it greatly reduces mechanical vibration and impact caused by sudden stops, violent acceleration and deceleration, or unusual paths, improving the stability of equipment operation. By sorting spatially discrete gripping points according to the robot's macroscopic movement direction, it completely avoids inefficient movement between points. The robot can complete gripping sequentially along a smooth and continuous direction, with a near-shortest movement path and significantly reduced idle travel time, thereby directly and effectively shortening the cycle time of a single operation and significantly improving overall production efficiency.

[0023] A further technical solution is that step S7 specifically includes the following steps: During the initial calibration, the camera acquires image data of the three spheres on the visual calibration device, and establishes a reference coordinate system CORRbase (x, y, z) based on the coordinates of the three spheres; According to the preset cycle, the camera once again acquires image data of the three spheres and calculates the deviation between the current coordinates of the three spheres and the reference coordinate system CORRbase (x, y, z). If the deviation exceeds the preset accuracy threshold, an alarm message will be issued, prompting the operator to manually compensate the deviation data to the robot tool coordinate system to complete the visual correction.

[0024] This invention's periodic calibration mechanism addresses the accuracy degradation that occurs during long-term operation of industrial automation systems, shifting accuracy management from "post-failure repair" to "prevention before deviations." Through periodic inspections, minute system deviations can be detected and corrected promptly before perceptible degradation in palletizing accuracy occurs. By quantifying and visualizing the accuracy status, managers can clearly understand the system's health, enabling predictive maintenance. This reduces the risk of unexpected downtime due to sudden loss of accuracy, enhancing the reliability of production planning and the predictability of the entire intelligent manufacturing system.

[0025] A further technical solution is that step S2 includes the following specific steps: S21, fix the camera to the end of the robot, fix the calibration plate in the working environment, so that the calibration plate is below the 3D vision camera and the distance between the two is a preset distance, and adjust the 3D vision camera to a pose parallel to the calibration plate. S22, Adjust the 2D parameters of the 3D vision camera until the calibration board image is neither overexposed nor underexposed. Adjust the 3D parameters of the 3D vision camera and set the surface smoothing and noise removal mode of the point cloud post-processing to Normal to reduce the fluctuation range of the point cloud and ensure that the round point cloud on the calibration board is full. S23, start the 3D vision camera automatic calibration interface service and connect it to the robot. The teaching robot moves along the preset path, and the 3D vision camera collects the calibration data of each path point. Based on the collected data, solve the transformation matrix of the 3D vision camera relative to the robot end effector and the transformation matrix of the robot end effector relative to the base coordinate system. Combine them to obtain the transformation matrix from the base coordinate system to the 3D vision camera coordinate system, and complete the hand-eye calibration.

[0026] This invention provides a 3D vision-based robotic gear assembly method. By constructing a complete vision-robot collaborative control system, it achieves full automation and intelligence in the gear heat treatment assembly process. Compared with existing technologies, this invention has the following specific technical advantages: (1) The method provided by this invention is applicable to automatic tray loading in gear heat treatment processes, meets the matching requirements of various gears and customized trays, has a wide range of applicable gear sizes, good adaptability, and high stability. (2) It successfully replaced the traditional inefficient and error-prone manual operation. By establishing an expandable gear and tray template library, it achieved rapid identification and accurate matching of multiple models and specifications of workpieces, which greatly improved the flexibility of the production line. (3) The system innovatively introduces a periodic visual correction mechanism based on the reference coordinate system, which can detect and compensate for the system accuracy drift during production breaks, thereby achieving accuracy maintenance and stability under long-term operation with extremely low maintenance costs. Moreover, the correction is carried out during the production preparation stage without occupying the production cycle.

[0027] This invention not only solves the inherent problems of low efficiency and high error rate in manual pallet loading, but also forms a complete solution with high precision, high cycle time, high flexibility and long-term stability. It provides a reliable technical path for automated loading and unloading of gear heat treatment and other similar irregular parts, and has broad industrial application prospects. Attached Figure Description

[0028] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0029] Figure 1 This is a schematic flowchart of a robot gear mounting method based on 3D vision according to one embodiment of the present invention. Detailed Implementation

[0030] The present invention will now be described in detail with reference to the accompanying drawings. This description is merely illustrative and explanatory, and should not be construed as limiting the scope of protection of the present invention. Furthermore, those skilled in the art can combine the features in the embodiments described herein and in different embodiments accordingly based on the description in this document.

[0031] The embodiments of the present invention are as follows, with reference to Figure 1 A method for mounting robot gears on a disc based on 3D vision includes the following steps: S1, establish the connection between the 3D vision camera and the robot; S2, Calibration of the 3D Vision Camera; S3, Workpiece Template Establishment: Collect standard images of non-circular workpieces such as trays and circular workpieces such as gears, preprocess the 3D point cloud data of the collected standard images, extract key features, and construct tray template libraries and gear template libraries respectively. S4, Template Matching Pose Output: During the production process, on-site scene images are acquired, and instances of non-circular workpieces such as material trays and circular workpieces such as gears are segmented from the scene point cloud. After the instances are initially aligned with the templates in the template library, registration is performed. The matching error is iteratively optimized to a preset threshold, and the pose matrix of the workpiece is output and stored. S5, Pose adjustment and sorting of gripping points: Extract the coordinates of gripping points of multiple gears, and sort the coordinate values ​​of gripping points in ascending or descending order according to the robot's preset running direction to determine the robot's gripping order. S6, Sorting of placement points: Sort the placement points of the gears according to the preset order of the robot teaching points, and record the storage information after the gears are placed in place; S7, regular visual calibration to maintain traying accuracy.

[0032] In practical applications, the hardware and software versions of the robot and camera are checked, a network connection is established, robot communication is configured, and standard interface communication is tested. The 3D vision-based robot gear loading method provided by this invention achieves full automation and intelligence of the gear loading process by constructing an integrated vision-robot control system. By establishing an expandable gear and tray template library, this method can quickly adapt to gears of different models and sizes, as well as non-standard trays, without the need for complex mechanical adjustments or program rewriting. This effectively solves the problem of rapid production changeover in multi-variety, small-batch heat treatment production, greatly improving the flexibility of the production line.

[0033] By utilizing depth information from 3D vision and combining it with a template matching strategy of "coarse matching + fine registration," the robot can accurately identify the pose of the workpiece in three-dimensional space, effectively compensating for placement errors and manufacturing tolerances. This makes the robot's grasping and placement actions accurate and reliable, significantly reducing the risks of tray misalignment and detachment, and ensuring the stability of the heat treatment process and the consistency of product quality.

[0034] By replacing traditional manual operations with automated systems, not only are problems such as manual marking, pre-assembly, and error-prone processes avoided, but also the most efficient movement path of the robot is planned through an optimized grasping point sorting algorithm, which greatly shortens the cycle time of a single operation and makes the production pace stable and far exceeds the speed of manual operation.

[0035] The periodic visual calibration mechanism can automatically detect and compensate for accuracy drift caused by factors such as mechanical wear and thermal deformation during production breaks. This ensures that the system maintains its initial high accuracy even under long-term, continuous operation, achieving "preventive maintenance," reducing batch quality accidents and equipment downtime caused by accuracy inaccuracies, with low maintenance costs and no disruption to production cycle time.

[0036] Based on the above embodiments, in another embodiment of the present invention, step S3 specifically includes the following steps: S31, Collect standard images of non-circular workpieces such as trays and circular workpieces such as gears; S32 processes the 3D point cloud data of the workpiece standard image according to the vision engineering process. Through data preprocessing, the point cloud quality is improved. At the same time, noise is eliminated and key features are extracted to ensure the accuracy of template matching. S33, extract gear or tray features, establish a workpiece template library, edit the templates, and define relevant variables including boundaries, matching thresholds, center points, and gripping points.

[0037] In practical applications, under controlled and ideal lighting conditions, empty trays (non-circular workpieces) and single gears (circular workpieces) are scanned from multiple angles to obtain high-resolution 3D point cloud data as "standard images." This step ensures the accuracy and consistency of the template source data. A series of preprocessing steps are performed on the acquired raw point clouds: data preprocessing includes: eliminating sensor noise and reflective anomalies through denoising and filtering algorithms (such as statistical outlier removal); significantly reducing the data volume while preserving the workpiece geometry through downsampling (such as voxelized mesh filtering), thus improving subsequent processing speed; and filtering out invalid data caused by the loss of depth information. Key feature extraction: Based on the high-quality point cloud, the essential geometric features of the workpiece are calculated and extracted. For gears, feature regions such as tooth tips and roots are identified through normal vector analysis and curvature calculation, and their center and radius are fitted using the least squares method. For trays, regions are segmented based on normal vector consistency, and their outer contour rectangles are fitted to obtain parameters such as dimensions and angles. The key features extracted above, preprocessing parameters, and relevant variables defined for matching and robot operation (such as workpiece boundaries, successful matching thresholds, gripping reference center points, precise gripping point coordinates and attitudes) are integrated together to form templates that can be directly called by the system, and then classified and stored in the gear template library and the tray template library.

[0038] Thus, through a complete set of data processing, feature extraction, and model building processes, highly standardized, reusable, and accurate "visual knowledge" is provided for the entire automated tray loading system. This is the core technical guarantee that enables the invention to have outstanding advantages such as high precision, high flexibility, high efficiency, and strong robustness.

[0039] Based on the above embodiments, in another embodiment of the present invention, the key feature extraction process in step S33 of the gear-type circular workpiece template library establishment process includes: Principal component analysis is performed on the neighborhood of each point in the gear point cloud to obtain the direction corresponding to the minimum eigenvalue as the normal vector of the point, so as to reflect the orientation of the gear surface; By calculating the Gaussian curvature or average curvature of neighboring points, the characteristic regions of the gear, including the tooth tip and tooth root, can be identified. The least squares method was used to fit a circle to the point cloud of the gear end face, and the error function was established and the center coordinates and radius of the circle were optimized. Calculate the point-pair feature histogram to describe the geometric properties of the gear, including tooth profile angle and tooth pitch, and build a gear template library based on the above features.

[0040] In application, gear feature extraction is a refined process that progresses from the surface to the core, from the macroscopic to the microscopic, aiming to comprehensively describe its complex geometric shape, including: Surface orientation analysis (normal vector calculation): For each point in the gear point cloud, analyze the distribution of points in its surrounding small neighborhood. Calculate the principal orientation of this neighborhood point cloud using principal component analysis. The eigenvector corresponding to the smallest eigenvalue is the normal vector direction of that point.

[0041] Feature region identification (curvature calculation): Based on the normal vector, the Gaussian curvature or average curvature of the neighborhood of each point is further calculated. Significant changes in curvature mark the boundaries of geometric features. Accordingly, the gear point cloud can be clearly segmented into different feature regions: the tooth tip region (with obvious curvature features), the tooth root region (with equally obvious curvature features but different from the tooth tip), and the relatively flat end face region.

[0042] Macroscopic geometric positioning (circle fitting): Primarily based on the identified, relatively flat point cloud of the gear end face, a circle is fitted using the least squares method. By establishing and optimizing the error function, the two-dimensional coordinates (X, Y) of the circle center and the radius that best match these end face points are iteratively calculated. This locates the core position and approximate size of the gear in the plane.

[0043] Microscopic geometric description (point-pair feature histogram): To address tooth profile variations and improve discriminative power, a point-pair feature histogram of the gear is calculated. This feature generates a high-dimensional descriptor by statistically analyzing the relative distances, normal vector angles, and other relationships between a large number of point pairs in the point cloud. This descriptor can quantitatively capture fine geometric attributes of the gear tooth profile, such as angles and pitch, forming a unique "geometric fingerprint."

[0044] This approach achieves a complete and robust description of complex gear geometry: a hybrid feature model combining macroscopic geometry (center, radius), mesoscopic regions (tooth tip, tooth root), and microscopic features (tooth profile PFH) allows the template to comprehensively represent a gear. This significantly enhances the template's discriminative power and robustness. Even with partial occlusion or slight adhesion to the background and other gears, the system can accurately identify and match based on rich feature information. Precise normal vector and curvature information provide high-quality initial corresponding point pairs for the subsequent ICP fine registration algorithm, guiding the registration process to quickly converge to the correct position. The least-squares fitted circle provides a stable and accurate grasping reference center, overcoming the center point drift problem caused by occlusion or defects of individual teeth, ensuring the long-term repeatability accuracy of the robot's grasping pose. It also enhances the ability to distinguish subtle differences in gear models; different gear models may have subtle differences in tooth profile and pitch, which cannot be described by a simple center and radius. Microscopic features can keenly capture these differences, enabling the template library to effectively distinguish gear models with similar appearances, fundamentally preventing the risk of picking the wrong model on a mixed production line, and achieving true intelligence.

[0045] Based on the above embodiments, in another embodiment of the present invention, the key feature extraction process of the process of establishing the template library for non-circular workpieces of the material tray type in step S33 includes: Based on the consistency between the normal vector and curvature of the point cloud, the point cloud of the material tray is divided into different regions including the bottom surface and side walls; Extract the point cloud of the outer contour of the tray and generate a closed polygon; Fit a rectangle to the point cloud of the outer contour of the material tray, and calculate the length, width and diagonal length of the rectangle; Extract the global aspect ratio features of the material trays and construct a material tray template library based on the region division results and geometric parameters.

[0046] Feature extraction of the feed tray aims to transform its irregular physical structure into a series of computable, matchable geometric parameters, including: Region semantic segmentation: First, the normal vector (surface orientation) and curvature (surface bending degree) of each point in the material tray point cloud are analyzed. Based on the consistency of these geometric attributes, the point cloud of the material tray is automatically segmented into different functional regions, mainly the large-area bottom surface region with the normal vector pointing vertically upward, and the side wall region perpendicular to the bottom surface, giving the system the ability to understand the structure of the material tray.

[0047] Outer contour extraction and representation: Based on the segmentation results, the outermost edge point set of the material tray is extracted, and a closed polygon is generated to describe its boundary shape in the top view.

[0048] Macroscopic geometric parameterization: To achieve fast and stable matching, the outer contour polygon obtained in the previous step is approximated by a rectangle that best fits its shape. By fitting the rectangle, key dimensions such as the length, width, and diagonal length of the tray can be accurately calculated, and irregular shapes can be converted into precise values.

[0049] Global Feature Construction: Further calculation of global features such as the aspect ratio of the tray. This plays a crucial role in distinguishing trays of different specifications but similar shapes (such as rectangular trays of different sizes).

[0050] In this way, stable identification and precise positioning of irregular workpieces are achieved. By clearly distinguishing the bottom surface and sidewalls through region segmentation, the system can accurately calculate the true posture of the workpiece tray in three-dimensional space (e.g., whether it is horizontal). Combining the length, width, and angle parameters obtained from fitting a rectangle to the outer contour, a precise pose matrix containing position and orientation can be output. This ensures that the robot can accurately place the gear directly above the designated slot on the workpiece tray, avoiding placement collisions or jamming caused by inaccurate workpiece tray pose estimation. When the production line needs to switch workpiece trays, the system can automatically call the correct workpiece tray template by quickly matching these features and pair it with the corresponding gear template, greatly enhancing the flexibility of the production line and effectively preventing mismatched gear and workpiece tray loading errors. The stable features extracted by this method provide a reliable environmental model for the robot to plan collision-free, high-precision placement paths, which is a technical prerequisite for achieving precise docking of gears and workpiece tray slots.

[0051] Based on the above embodiments, in another embodiment of the present invention, step S32, the step of preprocessing the 3D point cloud data of the image, includes: Denoising and filtering: Based on the local density of the point cloud, isolated points that deviate from the point cloud mean by more than a preset threshold are removed, and outliers caused by sensor noise or reflection are eliminated. While smoothing the point cloud data, the edge features of the workpiece are preserved, and key structures, including gear teeth and the boundary line between the side wall and bottom of the tray, are avoided from being overly blurred. Downsampling: Divide the 3D space into cubic voxels of a preset size, and replace the original points with the centroids of all points within each voxel, reducing the amount of point cloud data while maintaining the overall shape of the workpiece; Invalid point filtering: Delete invalid points with non-numerical coordinates to avoid missing data caused by reflection or occlusion interfering with subsequent feature extraction.

[0052] Point cloud preprocessing aims to "purify" and "refine" raw, coarse 3D data into high-quality data suitable for high-precision feature extraction. It consists of three core steps: Noise Reduction and Filtering: A statistical filtering algorithm is employed to analyze the distance distribution between each point in the point cloud and its neighbors. Points whose distance deviates from the local point cloud average by more than a preset threshold are identified as isolated noise points (usually caused by random sensor errors or reflections from the workpiece surface) and removed. The key to this process lies in the precise setting of the threshold, which requires a balance between smoothing noise and preserving key edge features. This ensures that the core geometric structures that determine the workpiece pose, such as each tooth of the gear and the boundary line between the sidewall and bottom of the tray, are completely preserved without being excessively blurred.

[0053] Downsampling: Point clouds are simplified using a voxelization mesh method. The 3D space is divided into countless tiny, uniformly sized cubic voxels. The centroid (average position) of all 3D points falling within each voxel is calculated, and this single centroid represents all the original points within that voxel. This operation dramatically reduces the point cloud data size from millions to hundreds of thousands, while preserving the overall shape and macroscopic contour of the workpiece to the greatest extent possible because centroids are used.

[0054] Invalid point filtering: In 3D vision, laser absorption or specular reflection can prevent sensors from acquiring valid depth information, resulting in invalid points with coordinates (NaN, NaN, NaN). These points contain no spatial information but severely interfere with subsequent calculations. This step identifies and removes these invalid points by traversing all points, ensuring that the point cloud data used in the calculations consists of real and valid spatial coordinates.

[0055] Thus, through preprocessing, the system effectively overcomes the problems of noise, glare, and data loss commonly faced by 3D vision sensors in real industrial environments. This enables the method of the present invention to not only work under ideal laboratory conditions but also adapt to the complex lighting and workpiece surface conditions in real workshop environments, exhibiting strong environmental adaptability and operational stability, and reducing the risk of production line interruptions due to visual recognition failures.

[0056] Based on the above embodiments, in another embodiment of the present invention, in step S4, the template and the instance are initially aligned based on geometric features, and the ICP algorithm is used for registration. The specific process is as follows: after the template point cloud and the workpiece instance point cloud are initially aligned, the distance error between corresponding points between the two point clouds is iteratively calculated, and the pose of the instance point cloud is adjusted until the error is less than a preset threshold to complete the fine registration. The pose optimization adopts the least squares method, and the pose parameters of the workpiece instance are adjusted by minimizing the error function.

[0057] This process is initiated after preliminary alignment (coarse matching) based on geometric features, and aims to accurately align the template point cloud with the point cloud of the workpiece instance in the scene. Its core is the combination of iterative nearest point algorithm and least squares optimization.

[0058] Initial alignment: Starting from the approximate pose of the workpiece obtained in the previous coarse matching step, the pre-established high-quality template point cloud and the workpiece instance point cloud segmented from the scene are placed in three-dimensional space.

[0059] Iterative Fine-Fixing Loop: The ICP algorithm enters an automatically iterative optimization loop. For each point in the template point cloud, it finds its nearest neighbor in the instance point cloud, forming an initial pair of corresponding points. Based on the currently found pairs, the algorithm calculates an optimal rigid body transformation (i.e., a rotation matrix and a translation vector) to improve the overall matching accuracy. This step uses the least squares method as the optimization strategy, precisely solving for the parameters of this transformation by minimizing the average sum of squared distances between all pairs of corresponding points (i.e., the error function). Application of the transformation: The calculated rotation and translation transformations are applied to the entire instance point cloud, causing it to spatially converge towards the template point cloud.

[0060] Convergence criterion: The above iterative process is executed repeatedly. After each iteration, the algorithm re-evaluates the overall matching error between the two point clouds. When this error decreases below a preset threshold or reaches the maximum number of iterations, the iteration loop terminates. At this point, the pose of the instance point cloud has been adjusted to the optimal state, which is the final high-precision pose matrix output.

[0061] Through iterative optimization, the ICP algorithm can effectively compensate for deviations caused by workpiece manufacturing tolerances, placement tilt, and residual errors from coarse matching. The final output pose matrix has extremely high accuracy, providing the robot with absolute position and orientation information sufficient for precise grasping and reliable placement. It transforms the results of previous steps (preprocessing, template establishment, and coarse matching) into precise spatial commands that can be executed by the robot, which is the decisive link in achieving the goals of high precision, high stability, and fully automated operation in the entire technical solution.

[0062] Based on the above embodiments, in another embodiment of the present invention, the sorting key value of each grasping point is calculated according to the robot's running direction vector, and the key values ​​are arranged in ascending or descending order.

[0063] Based on the above embodiments, in another embodiment of the present invention, step S5 specifically includes the following steps: Extract the coordinates (x) of the gripping points of multiple gears. i , y i , z i ), where i is the gear number, i≥1; Define the robot's running direction vector D=(d x , d y , d z ), where d x d y d z d represents the motion direction coefficients along the X, Y, and Z axes of the coordinate system, respectively. x dy , d z All ∈ {-1, 0, 1}; If the robot's direction of movement is X, then use formula k i =d x ×x i Calculate the sorting key value; if the robot's running direction is the Y direction, then use formula k. i =d y ×y i Calculate the sort key value; Sort by key value k i Arrange the gripping points in ascending or descending order to determine the robot's gripping sequence.

[0064] In practical applications, the system obtains the precise gripping point coordinates (x, y) of all gears to be gripped from step S4. i , y i ,z i This forms a set of target points to be processed. Based on the actual layout of the robot workstation and the optimal motion path, a robot running direction vector D=(d x , d y , d z This vector, in coefficient form (-1, 0, 1), explicitly specifies the robot's primary direction of movement in the base coordinate system. Based on the primary direction of movement, a scalar sorting key value k is calculated for each gripping point. iThis invention provides a method for sorting according to the robot's running direction, either X or Y. The above formula cleverly unifies the sorting logic for different movement directions (forward and reverse) by using the positive or negative (±1) value of the coefficients. Finally, all gripping points are sorted according to their calculated key value k. i Arrange the data in ascending or descending order. This sequence represents the final order in which the robot's end effector proceeds to grasp the objects.

[0065] The sorting logic of this invention is decoupled from specific gear models and tray layouts, reducing the computational load on the robot control system. More importantly, it greatly reduces mechanical vibration and impact caused by movement stops, abrupt accelerations and decelerations, or unusual paths, improving the stability of equipment operation. By sorting spatially discrete gripping points according to the robot's macroscopic movement direction, it completely avoids inefficient movement between points. The robot can complete gripping sequentially along a smooth and continuous direction, with a near-shortest movement path and significantly reduced idle travel time, thereby directly and effectively shortening the cycle time of a single operation and significantly improving overall production efficiency.

[0066] Based on the above embodiments, in another embodiment of the present invention, step S7 specifically includes the following steps: During the initial calibration, the camera acquires image data of the three spheres on the visual calibration device, and establishes a reference coordinate system CORRbase (x, y, z) based on the coordinates of the three spheres; According to the preset cycle, the camera once again acquires image data of the three spheres and calculates the deviation between the current coordinates of the three spheres and the reference coordinate system CORRbase (x, y, z). ; If the deviation exceeds the preset accuracy threshold, an alarm message will be issued, prompting the operator to manually compensate the deviation data to the robot tool coordinate system to complete the visual correction.

[0067] In practical applications, after the system's initial installation or major overhaul, during a non-production period, a 3D camera is used to photograph the vision correction device (which has three non-coordinated reference spheres) fixed in the robot's workspace. A high-precision algorithm calculates the center coordinates of the three spheres, establishing a high-precision, stable reference coordinate system, CORR_base. This coordinate system serves as the "origin" or "scale" for all subsequent accuracy measurements. The system automatically executes the correction procedure according to a preset cycle (e.g., once per shift or once daily). The camera then photographs the same correction device again and calculates the current real-time coordinate system CORR_current defined by the three spheres. The system compares CORR_current with CORR_base, calculating the pose deviation (typically including translational and rotational deviations in three directions). This deviation is compared to a preset accuracy threshold set by the system. If the deviation is within tolerance, the system silently passes; if it exceeds the threshold, an alarm is immediately triggered. The alarm message clearly indicates to the operator that there is an accuracy error. The operator manually inputs the deviation data provided by the system into the robot's tool coordinate system parameters. This operation is equivalent to fine-tuning the robot's "ruler," thereby systematically compensating for the accuracy drift that has occurred and restoring the system to its factory precision.

[0068] This invention's periodic calibration mechanism addresses the accuracy degradation that occurs during long-term operation of industrial automation systems, shifting accuracy management from "post-failure repair" to "prevention before deviations." Through periodic inspections, minute system deviations can be detected and corrected promptly before perceptible degradation in palletizing accuracy occurs. By quantifying and visualizing the accuracy status, managers can clearly understand the system's health, enabling predictive maintenance. This reduces the risk of unexpected downtime due to sudden loss of accuracy, enhancing the reliability of production planning and the predictability of the entire intelligent manufacturing system.

[0069] Based on the above embodiments, in another embodiment of the present invention, step S2 specifically includes the following steps: S21, fix the camera to the end of the robot, fix the calibration plate in the working environment, so that the calibration plate is below the 3D vision camera and the distance between the two is a preset distance (about 1300mm), and adjust the 3D vision camera to a pose parallel to the calibration plate. S22, Adjust the 2D parameters of the 3D vision camera until the calibration board image is neither overexposed nor underexposed. Adjust the 3D parameters of the 3D vision camera and set the surface smoothing and noise removal mode of the point cloud post-processing to Normal to reduce the fluctuation range of the point cloud and ensure that the round point cloud on the calibration board is full. S23, start the 3D vision camera automatic calibration interface service and connect it to the robot. The teaching robot moves along the preset path, and the 3D vision camera collects the calibration data of each path point. Based on the collected data, solve the transformation matrix of the 3D vision camera relative to the robot end effector and the transformation matrix of the robot end effector relative to the base coordinate system. Combine them to obtain the transformation matrix from the base coordinate system to the 3D vision camera coordinate system, and complete the hand-eye calibration.

[0070] The transformation relationship between the base coordinate system (B) and the camera coordinate system (C) is established through hand-eye calibration. The robot is taught a path, and the robot path is aligned with the camera coordinate system. The robot moves according to the set path, and the camera completes image acquisition at each path point to obtain calibration data, thus completing the calibration. ; in The transformation matrix from the base coordinate system to the camera coordinate system is obtained through hand-eye calibration. (Camera relative end-effector transformation) and the robot provided It is composed of (the transformation of the end relative to the base coordinate system).

[0071] This invention not only solves the inherent problems of low efficiency and high error rate in manual pallet loading, but also forms a complete solution with high precision, high cycle time, high flexibility and long-term stability. It provides a reliable technical path for automated loading and unloading of gear heat treatment and other similar irregular parts, and has broad industrial application prospects.

[0072] For those skilled in the art, various improvements and modifications can be made without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention.

Claims

1. A 3D vision based robotic gear wheel wheeling method, characterized in that, The method comprises the following steps: S1, establishing the connection between the 3D vision camera and the robot; S2, calibrating the 3D vision camera; S3, establishing the workpiece template: collecting the standard images of the non-circular workpieces of the tray type and the circular workpieces of the gear type, pre-processing the 3D point cloud data of the collected standard images, extracting the key features, and respectively constructing the tray template library and the gear template library; S4, template matching pose output: collecting the scene images in the production process, segmenting the instances of the non-circular workpieces of the tray type and the circular workpieces of the gear type from the scene point cloud, aligning the instances with the templates in the template library, and then registering, iteratively optimizing the matching error to a preset threshold, and outputting and storing the pose matrix of the workpiece; S5, pose adjustment and sorting of the grabbing points: extracting the coordinates of the grabbing points of multiple gears, arranging the coordinate values of the grabbing points in ascending or descending order according to the preset running direction of the robot, and determining the grabbing sequence of the robot; S6, sorting of the placing points: sorting the placing points of the gears according to the preset sequence of the robot teaching points, and recording the storage information after the gears are placed in place; S7, regular visual correction, maintaining the tray loading accuracy.

2. The 3D vision based robotic gear wheel palletizing method of claim 1, wherein, The step S3 specifically comprises the following steps: S31, collecting the standard images of the non-circular workpieces of the tray type and the circular workpieces of the gear type; S32, processing the 3D point cloud data of the workpiece standard images according to the process of visual engineering, improving the point cloud quality through data preprocessing, and calculating and eliminating noise and extracting key features to ensure the accuracy of template matching; S33, extracting the features of the gears or trays, establishing the workpiece template library, and editing the templates to define related variables including the boundary, matching threshold, center point and grabbing point.

3. The 3D vision based robotic gear wheel palletizing method of claim 2, wherein, In the step S33, the key feature extraction process of the gear type circular workpiece template library establishment process comprises: Performing principal component analysis on the neighborhood of each point in the gear point cloud, obtaining the direction corresponding to the minimum eigenvalue as the normal vector of the point to reflect the surface orientation of the gear; Identifying the feature regions of the gear including the tooth tip and the tooth root by calculating the Gaussian curvature or the average curvature of the neighborhood points; Fitting a circle to the gear end face point cloud using the least squares method, establishing an error function and optimizing the center coordinates and the radius of the circle; Calculating the point pair feature histogram to describe the geometric properties of the gear including the tooth angle and the pitch, and constructing the gear template library based on the above features.

4. The 3D vision based robotic gear wheel palletizing method of claim 3, wherein, In the step S33, the key feature extraction process of the tray type non-circular workpiece template library establishment process comprises: According to the consistency of the normal vector and the curvature of the point cloud, the tray point cloud is divided into different regions including the bottom surface and the side wall; Extracting the outer contour point cloud of the tray and generating a closed polygon; Fitting a rectangle to the tray outer contour point cloud, and calculating the length, width and diagonal length of the rectangle; Extracting the global feature of the length-width ratio of the tray, and constructing the tray template library based on the region division result and the geometric parameters.

5. The robot gear wheel wheeling method based on 3D vision according to any one of claims 2-4, characterized in that, In the step S32, the pre-processing step of the 3D point cloud data of the image comprises: De-noising and filtering: based on the local density of point cloud, remove isolated points deviating from the mean value of point cloud by more than a preset threshold, eliminate abnormal values caused by sensor noise or reflection, smooth the point cloud data while preserving the edge features of the workpiece, avoid excessive blur of key structures including gear tooth profile, intersection line between disc side wall and bottom surface; Down-sampling: divide the 3D space into cubic voxels of a preset size, replace the original points with the centroid of all points in each voxel, reduce the amount of point cloud data while maintaining the overall shape of the workpiece; Invalid point filtering: delete invalid points with non-numeric coordinates, avoid missing data caused by reflection or obstruction to interfere with subsequent feature extraction.

6. The 3D vision based robotic gear wheel palletizing method of claim 5, wherein, In step S4, the template and the instance are preliminarily aligned based on geometric features, and the ICP algorithm is used for registration. The specific process is as follows: after the initial alignment of the template point cloud and the workpiece instance point cloud, the distance error between the corresponding points of the two point clouds is iteratively calculated, the pose of the instance point cloud is adjusted until the error is less than a preset threshold, and the fine registration is completed. The pose optimization uses the least squares method to adjust the pose parameters of the workpiece instance by minimizing the error function.

7. The 3D vision based robotic gear wheel palletizing method of claim 5, wherein, In step S5, the sorting key value of each grabbing point is calculated according to the running direction vector of the robot, and the key values are arranged in ascending or descending order.

8. The 3D vision based robotic gear wheel palletizing method of claim 7, wherein, The specific steps of step S5 include: extracting a plurality of gear grabbing point coordinates (x i , y i , z i ), wherein i is a gear serial number, i≥1; A robot running direction vector D = (d x , d y , d z ) is defined, wherein d x , d y , d z are movement direction coefficients along the X, Y, Z axes of the coordinate system, and d x , dy , d z are all ∈{-1, 0, 1}; If the robot running direction is the X direction, the sorting key value is calculated by the formula k i = d x x x i If the robot running direction is the Y direction, the sorting key value is calculated by the formula k i = d y x y i ​ Sort by key value k i Arrange the gripping points in ascending or descending order to determine the robot's gripping sequence.

9. The 3D vision based robotic gear wheel palletizing method of claim 1, wherein, The specific steps of step S7 include: When first correcting, the camera collects image data of the three balls on the visual correction device, and a reference coordinate system CORRbase (x, y, z) is established based on the coordinates of the three balls; At a preset period, the camera collects image data of the three balls again, and calculates the deviation value of the current coordinates of the three balls from the reference coordinate system CORRbase (x, y, z); If the deviation value exceeds a preset accuracy threshold, an alarm information is sent to prompt an operator to manually compensate the deviation data to the robot tool coordinate system to complete the visual correction.

10. The 3D vision based robotic gear wheel palletizing method of claim 1, wherein, The specific steps of step S2 include: S21, fix the camera at the end of the robot, fix the calibration board in the working environment, so that the calibration board is located below the 3D vision camera and the distance between them is a preset distance, and adjust the 3D vision camera to a pose parallel to the calibration board; S22, adjust the 2D parameters of the 3D vision camera to the calibration board image without overexposure or overdarkness phenomenon, and adjust the 3D parameters of the 3D vision camera to set the surface smoothing and noise removal mode of point cloud post-processing to Normal, so as to reduce the fluctuation range of point cloud and ensure that the point cloud of the round dots on the calibration board is full; S23, start the 3D vision camera automatic calibration interface service and connect with the robot, teach the robot to move along a preset path, the 3D vision camera collects calibration data of each path point, solves the transformation matrix of the 3D vision camera relative to the end of the robot and the transformation matrix of the end of the robot relative to the base coordinate system based on the collected data, and combines to obtain the transformation matrix from the base coordinate system to the 3D vision camera coordinate system, and completes the hand-eye calibration.

Images