Vision analysis based industrial part alignment method, system, and storage medium

Abstract

The application relates to the technical field of image processing, and discloses an industrial part alignment method and system based on visual analysis and a storage medium. The method comprises the following steps: an industrial part image is collected by a three-view camera, and an enhanced image is obtained through gradient amplitude local contrast enhancement preprocessing. Edge contours and key control points are identified through hierarchical feature extraction, and a multi-dimensional feature set is formed. A dynamic reference coordinate system is established based on the feature set, and a part space attitude matrix is obtained. Attitude deviation is compensated through Z-axis offset and rotation coupling error analysis. The attitude adjustment is decomposed into multiple sub-stages, a variable speed planning strategy is adopted to optimize the alignment trajectory, and accurate part alignment is realized. The application solves the problems of insufficient multi-view visual information fusion, lack of special recognition algorithm for industrial part geometric features, and inaccurate Z-axis offset and rotation coupling error compensation in the attitude adjustment process, and improves the accuracy and stability of industrial part alignment.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, system and storage medium for aligning industrial parts based on visual analysis. Background Technology

[0002] Existing industrial part alignment methods primarily employ image processing techniques based on single-view or fixed binocular vision, extracting features and identifying positions by acquiring two-dimensional image information of the parts. Traditional methods typically use standard edge detection algorithms, such as the Canny operator, to extract the part contour, then use template matching or geometric fitting to calculate the deviation between the part and the target position, and finally achieve part alignment through simple position compensation. These methods can achieve a certain level of alignment accuracy when processing parts with regular shapes and good surface quality.

[0003] However, single-view or fixed binocular configurations cannot acquire complete spatial information of parts, resulting in limited ability to recognize complex 3D poses. Secondly, traditional image preprocessing methods are sensitive to changes in illumination and surface reflections, which can easily lead to feature extraction errors in industrial environments. Thirdly, existing feature extraction methods lack specific optimization for the regular geometric features of industrial parts, resulting in insufficient accuracy in recognizing key features such as corners and line intersections. Finally, traditional methods ignore the multi-dimensional coupling errors generated during end effector pose adjustment, especially the complex coupling relationship between Z-axis offset and rotational motion.

[0004] Based on the above analysis, existing technologies face progressive technical challenges in aligning precision industrial parts: how to obtain more complete spatial information of the parts through multi-view visual fusion, how to design dedicated feature extraction algorithms for the geometric features of industrial parts, how to establish a dynamic coordinate system adapted to the geometric characteristics of the parts, and how to effectively compensate for multi-dimensional coupling errors during the posture adjustment process. These problems are interconnected and progressively deepen; simply solving one problem cannot meet the overall requirements of high-precision alignment. Systematic technological innovation is needed to achieve an organic combination of multi-view information fusion, dedicated feature extraction, dynamic coordinate establishment, and coupling error compensation. Summary of the Invention

[0005] This application provides a visual analysis-based industrial parts alignment method, system, and storage medium to address the problems of insufficient fusion of multi-view visual information, lack of dedicated recognition algorithms for the geometric features of industrial parts, and inaccurate compensation for Z-axis offset and rotational coupling errors during posture adjustment in the prior art, thereby improving the accuracy and stability of industrial parts alignment.

[0006] In a first aspect, this application provides a visual analysis-based method for aligning industrial parts, the visual analysis-based method for aligning industrial parts comprising:

[0007] The images of industrial parts are acquired and processed by a three-view camera, and the acquired images are preprocessed by gradient amplitude local contrast enhancement to obtain enhanced images of the parts.

[0008] Based on the enhanced image of the part, the edge contour and key control points of the part are identified and processed by hierarchical feature extraction to obtain a multi-dimensional feature set of the part;

[0009] Based on the multi-dimensional feature set of the part, the geometric center of the part is set as the origin of the coordinate system and a dynamic reference coordinate system is established to obtain the spatial posture matrix of the part.

[0010] Based on the difference between the spatial attitude matrix of the part and the target attitude, the attitude deviation is compensated by Z-axis offset and rotation coupling error analysis to obtain the compensated attitude parameters.

[0011] Based on the compensated attitude parameters, the attitude adjustment process is decomposed into multiple sub-stages and the alignment trajectory is optimized using a variable speed planning strategy to obtain the part alignment state.

[0012] Secondly, this application provides a vision analysis-based industrial parts alignment system, the vision analysis-based industrial parts alignment system comprising:

[0013] The acquisition module is used to acquire and process images of industrial parts through a three-view camera, and to preprocess the acquired images by gradient amplitude local contrast enhancement to obtain enhanced images of the parts.

[0014] The recognition module is used to identify the edge contour and key control points of the part based on the enhanced image of the part through hierarchical feature extraction, so as to obtain a multi-dimensional feature set of the part.

[0015] The setting module is used to set the geometric center of the part as the origin of the coordinate system and establish a dynamic reference coordinate system based on the multi-dimensional feature set of the part, so as to obtain the spatial posture matrix of the part.

[0016] The compensation module is used to compensate for the attitude deviation based on the difference between the part's spatial attitude matrix and the target attitude, through Z-axis offset and rotational coupling error analysis, to obtain the compensated attitude parameters.

[0017] The optimization module is used to decompose the attitude adjustment process into multiple sub-stages based on the compensated attitude parameters and optimize the alignment trajectory through a variable speed planning strategy to obtain the part alignment state.

[0018] Thirdly, a vision analysis-based industrial parts alignment device is provided, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor invokes the instructions in the memory to cause the vision analysis-based industrial parts alignment device to execute the aforementioned vision analysis-based industrial parts alignment method.

[0019] Fourthly, a computer-readable storage medium is provided, wherein instructions are stored therein, which, when executed on a computer, cause the computer to perform the aforementioned visual analysis-based industrial parts alignment method.

[0020] The technical solution provided in this application effectively solves the problem that traditional single-view or fixed binocular configurations cannot obtain complete spatial information by acquiring and processing images of industrial parts through a three-view camera and preprocessing the acquired images by gradient magnitude local contrast enhancement. The three-view camera configuration can simultaneously acquire front, side and top view images of the parts, providing richer geometric constraints for subsequent three-dimensional pose calculation. The gradient magnitude local contrast enhancement technology is specifically optimized for the metallic reflective properties of the surface of industrial parts. It automatically identifies edge regions by calculating the gradient direction consistency index of the 8-neighborhood of a pixel and adopts differentiated contrast enhancement strategies for different regions, which significantly improves the problem of unstable feature extraction under complex lighting conditions in traditional methods. By extracting features in a hierarchical manner to identify and process the edge contours and key control points of parts, a multi-dimensional feature set of technical features is obtained. A three-level feature system is established, consisting of pixel-level edges, feature-level key points, and geometric-level topology. The improved SURF algorithm is designed with dedicated descriptors for corner points, line intersections, and curvature change points of industrial parts. The feature consistency verification algorithm effectively eliminates false features by checking the correlation constraints between the three levels of features. This multi-level feature extraction and verification mechanism has higher reliability and accuracy than traditional single feature extraction methods. Based on the multi-dimensional feature set of parts, the geometric center of the parts is set as the origin of the coordinate system, and a dynamic reference coordinate system is established to obtain the technical features of the spatial attitude matrix of the parts. The geometric center of the parts is determined by least squares fitting, and the principal component analysis method is used to calculate the principal axis direction of the parts. This overcomes the limitation of traditional fixed coordinate systems that cannot adapt to different geometric characteristics of parts. The dynamic coordinate system can adaptively adjust according to the actual geometric characteristics of the parts, making the attitude representation more accurate and stable.

[0021] The technical characteristics of the compensated attitude parameters are obtained by analyzing the Z-axis offset and rotational coupling error to compensate for the attitude deviation. A detailed analysis was conducted specifically on the complex error coupling relationships generated during the end effector attitude adjustment process. A mathematical model between attitude parameter changes and alignment errors was established using Jacobi matrix calculation. The inverse kinematics algorithm can predict the position offset during attitude adjustment and generate corresponding compensation trajectories. This feedforward compensation mechanism effectively solves the problem of decreased alignment accuracy caused by neglecting multi-dimensional coupling errors in traditional methods, showing significant advantages, especially in handling the coupling relationship between Z-axis offset and rotational motion. The attitude adjustment process is decomposed into multiple sub-stages, and the alignment trajectory is optimized using a variable speed planning strategy to obtain the technical characteristics of the part alignment state. Piecewise linear interpolation ensures the smoothness and continuity of the motion trajectory, while the variable speed planning strategy achieves the best balance between efficiency and accuracy by automatically reducing the motion speed when approaching the target position. The closed-loop feedback control mechanism can monitor the part's position and attitude changes in real time and adjust the trajectory accordingly. This multi-level trajectory optimization and control strategy has higher robustness and adaptability than traditional open-loop control methods, enabling stable high-precision part alignment in complex industrial environments. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of one embodiment of the visual analysis-based industrial parts alignment method in this application.

[0024] Figure 2 This is a schematic diagram of one embodiment of the visual analysis-based industrial parts alignment system in this application.

[0025] Figure 3 This is a schematic block diagram of the structure of an industrial parts alignment device based on visual analysis in an embodiment of the present invention. Detailed Implementation

[0026] This application provides a method, system, and storage medium for aligning industrial parts based on visual analysis. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] For ease of understanding, the specific process of the embodiments of this application is described below. Please refer to [link / reference]. Figure 1 One embodiment of the visual analysis-based industrial parts alignment method in this application includes:

[0028] Step S101: The industrial parts are image acquired and processed by a three-view camera, and the acquired images are preprocessed by gradient amplitude local contrast enhancement to obtain enhanced images of the parts.

[0029] Step S102: Based on the enhanced image of the part, the edge contour and key control points of the part are identified and processed through hierarchical feature extraction to obtain a multi-dimensional feature set of the part.

[0030] Step S103: Based on the multi-dimensional feature set of the part, set the geometric center of the part as the origin of the coordinate system and establish a dynamic reference coordinate system to obtain the spatial attitude matrix of the part.

[0031] Step S104: Based on the difference between the part's spatial attitude matrix and the target attitude, the attitude deviation is compensated by Z-axis offset and rotation coupling error analysis to obtain the compensated attitude parameters.

[0032] Step S105: Based on the compensated attitude parameters, the attitude adjustment process is decomposed into multiple sub-stages and the alignment trajectory is optimized using a variable speed planning strategy to obtain the part alignment state.

[0033] It is understood that the executing entity of this application can be a vision analysis-based industrial parts alignment system, or it can be a terminal or a server; the specific implementation is not limited here. This application's embodiments use a server as an example for illustration.

[0034] Specifically, image acquisition is achieved using a three-view camera configuration. Two cameras are mounted at a 45-degree angle to form a binocular stereo vision setup, while a third camera is mounted vertically downwards for top-down monitoring. This asymmetrical three-view configuration can simultaneously acquire front, side, and top-down image information of the part, providing more comprehensive spatial information acquisition capabilities compared to traditional fixed binocular configurations. After image acquisition, the brightness distribution of the LED ring light source is adjusted using a regional light intensity control algorithm. This algorithm automatically adjusts the light intensity of different areas based on the surface reflection characteristics of the part, eliminating interference from metal surface reflections. Subsequently, local contrast enhancement processing based on gradient amplitude is performed. This process first calculates the gradient direction consistency index of the 8-neighborhood of a pixel, identifying the edge regions of the part by comparing the gradient direction differences between adjacent pixels. The gradient direction consistency index calculation involves similarity analysis of the gradient vectors of the eight neighboring pixels around each pixel. When the gradient direction difference between adjacent pixels is less than a set threshold, they are considered to belong to the same region; when the difference is large, they are identified as edge regions. Based on edge region identification data, gradient magnitude calculation is used to perform differentiated contrast enhancement on different regions. Stronger enhancement parameters are applied to edge regions to highlight contour features, while weaker enhancement parameters are applied to smooth regions to maintain image naturalness. Finally, multi-scale denoising processing is performed using a wavelet transform denoising algorithm. Wavelet transform can decompose image signals at different frequency scales, selectively removing high-frequency noise while preserving low-frequency geometric features of parts.

[0035] Hierarchical feature extraction is used to identify the edge contours and key control points of parts. First, an enhanced image is input to improve the Canny edge detection algorithm. This algorithm employs a dual-threshold adaptive adjustment mechanism, dynamically setting high and low threshold parameters based on the part's material and surface roughness characteristics. The dual-threshold mechanism sets strong and weak edge thresholds; strong edge pixels are directly marked as edge points, while weak edge pixels must be connected to strong edge pixels to be confirmed as edge points. This mechanism effectively reduces noise-induced false positives while maintaining edge continuity. Based on the edge contour data, an improved SURF algorithm is used for key point detection. This algorithm first calculates the Hessian matrix response value of the edge pixels. The Hessian matrix contains the second-order partial derivative information of the image at that point, enabling the detection of local extrema. A non-maximum suppression algorithm filters the response values, comparing the response values ​​of each pixel with its neighboring pixels, retaining only the points with local maxima as candidate key points. For the regular geometric features of industrial parts, dedicated descriptors for straight line segments and arc segments are designed. These descriptors determine whether a point belongs to a straight line or a curve by analyzing the gradient distribution pattern around the key point. The curvature change detection algorithm identifies locations of curvature abrupt changes by calculating the curvature value of each point on the part's contour. Curvature calculation involves locally fitting the contour line and calculating its second derivative. The Hough transform performs geometric detection on the part's straight and circular features, establishing the part's geometric topology, which describes the spatial relationships between the part's various geometric elements. The feature consistency verification algorithm performs correlation constraint verification on three levels of features, eliminating false features by checking the consistency between pixel-level edges, feature-level keypoints, and geometric topology. A weighted voting mechanism assigns weight coefficients based on the stability and repeatability of feature points from different perspectives, fusing them to generate a multi-dimensional feature set for the part that includes position, orientation, and scale information.

[0036] A dynamic reference coordinate system is established based on a multi-dimensional feature set of the part. Least squares fitting determines the main geometric contour of the part by minimizing the sum of squared distances from the part's contour points to the fitted curve, thereby calculating the geometric center coordinates of the part. Principal component analysis (PCA) calculates the principal axis directions of the feature set. This method determines the main change directions of the part by constructing the covariance matrix of the feature points and performing eigenvalue decomposition. The covariance matrix reflects the distribution characteristics of feature points in different directions; the eigenvector corresponding to the largest eigenvalue represents the direction of the greatest data change, i.e., the principal axis direction of the part. The eigenvector corresponding to the largest eigenvalue is set as the X-axis direction, the eigenvector corresponding to the second largest eigenvalue is set as the Y-axis direction, and the Z-axis direction is determined by the right-hand rule, forming a three-dimensional coordinate system. A multi-view geometric constraint attitude calculation algorithm constructs an overdetermined system of equations using the feature correspondences of the three views. The rotation matrix and translation vector of the part are solved using singular value decomposition, yielding a spatial attitude matrix of the part containing complete spatial attitude information.

[0037] Attitude deviations are compensated through Z-axis offset and rotation coupling error analysis. The attitude deviation matrix is ​​obtained by calculating the difference between the current part spatial attitude matrix and the target attitude matrix; this matrix contains both rotational and displacement error information. The Jacobian matrix calculates the partial derivatives of the part attitude parameters with respect to alignment errors, establishing a mathematical model of error propagation. This matrix describes the degree of influence of small changes in attitude parameters on alignment accuracy. Z-axis offset and rotation coupling analysis specifically addresses the complex error coupling relationships generated during end effector attitude adjustment. When the actuator performs rotational adjustment, the offset between the rotation center and the part's center of gravity leads to additional positional offsets. The inverse kinematics algorithm calculates the required joint angle changes based on the target and current attitudes, while simultaneously predicting the positional offsets of each intermediate state during the adjustment process. The feedforward compensation mechanism pre-calculates the compensation trajectory before executing the attitude adjustment command, dynamically adjusts control parameters to offset the expected error effects, and generates compensated attitude parameters containing both static and dynamic compensation parameters.

[0038] Trajectory optimization is performed based on the compensated attitude parameters. The axis-angle representation converts the rotation matrix into a combination of rotation axes and rotation angles, avoiding the singularity problem inherent in Euler angle representation. Three-dimensional coordinate decomposition breaks down translational deviations into independent displacements in the X, Y, and Z directions, facilitating subsequent trajectory planning. Piecewise linear interpolation decomposes the entire alignment process into multiple continuous motion sub-stages, maintaining linear changes in motion parameters within each sub-stage to ensure smooth trajectory. A variable-speed planning strategy dynamically adjusts the motion speed based on the distance between the part's current position and the target position; higher speeds are used to improve efficiency when the distance is greater, while automatically reducing speed to improve accuracy when approaching the target position. A closed-loop feedback control mechanism continuously monitors the real-time position and attitude changes of the part, automatically triggering a trajectory replanning program when a deviation from the expected trajectory is detected, achieving high-precision part alignment.

[0039] In one specific embodiment, the process of performing step S101 may specifically include the following steps:

[0040] Two cameras are installed at a 45-degree angle for binocular stereo vision acquisition, and a third camera is used to capture top-down images vertically downwards, resulting in original images of the parts from multiple perspectives.

[0041] Based on the original multi-view images of the parts, the brightness distribution of the LED ring light source is adjusted by a regional light intensity control algorithm to obtain a part image that eliminates reflective interference.

[0042] Based on the gradient direction consistency index calculation of the 8-neighborhood of pixels in the part image, the edge region of the part is automatically identified to obtain edge region identification data.

[0043] Based on the edge region identification data, the contrast of different regions is differentially enhanced by calculating the gradient magnitude, resulting in a locally contrast-enhanced image.

[0044] The locally contrast-enhanced image is input into a wavelet transform denoising algorithm for multi-scale denoising processing to obtain the enhanced image of the part.

[0045] Specifically, the binocular stereo vision acquisition utilizes two cameras mounted at a 45-degree angle to acquire 3D spatial information. Unlike traditional horizontally symmetrical binocular systems, this 45-degree angle allows for the acquisition of the side profile information of the part. Simultaneously, a third camera mounted vertically downwards acquires a top-down view, forming a complete perspective covering the front, sides, and top of the part. The three cameras are simultaneously triggered to acquire multiple original images of the part from different perspectives, each with the same resolution. Each image contains information about the part's edges, textures, and geometric features from that perspective. Camera calibration parameters ensure accurate geometric correspondence and spatial coordinate transformation between the three perspectives.

[0046] The regional light intensity control algorithm adjusts the brightness distribution of the LED ring light source based on the surface material characteristics of the part. This algorithm first analyzes the brightness distribution histograms of different regions in the original multi-view images of the part, identifying the locations of excessively bright and dark areas. The LED ring light source consists of multiple independently controlled light-emitting units, each corresponding to a specific area of ​​the part's surface. The algorithm calculates the difference between the average brightness value and the target brightness value for each region, and adjusts the current intensity of the corresponding LED unit based on the magnitude of the difference. Overly bright areas have reduced LED brightness output, while overly dark areas have increased LED brightness output. Light intensity adjustment employs closed-loop feedback control, monitoring the adjusted image brightness distribution in real time until the brightness of each region is uniformly distributed within a preset range, resulting in a part image free from reflective interference.

[0047] Gradient direction consistency index calculation is based on edge region identification using an 8-neighborhood of each pixel. The algorithm calculates the gradient direction difference between each pixel in the part image and its eight neighboring pixels. The gradient direction is calculated using the Sobel operator, containing horizontal and vertical gradient components. The gradient direction angle is obtained by calculating the ratio of the two components using the arctangent function. The consistency index calculation involves comparing the gradient direction angle difference between the center pixel and its eight neighboring pixels. Pixels in edge regions exhibit significant gradient direction differences from their neighbors due to drastic grayscale changes, resulting in a lower consistency index. Conversely, pixels in smooth regions have relatively consistent gradient directions, leading to a higher consistency index. By setting a consistency threshold, the image is segmented into edge regions and smooth regions, forming edge region identification data.

[0048] Gradient magnitude calculation achieves differentiated contrast enhancement by analyzing the intensity of grayscale changes in pixels. This process first performs Sobel gradient calculation on the part image to obtain the horizontal and vertical gradient components for each pixel. The gradient magnitude is obtained by taking the square root of the sum of the squares of the two components, reflecting the severity of the grayscale change at that pixel. Based on edge region identification data, different contrast enhancement parameters are applied to edge regions and smooth regions. Edge regions, due to their larger gradient magnitudes, use stronger enhancement coefficients to amplify grayscale contrast, highlighting the part's outline and details. Smooth regions, with their smaller gradient magnitudes, use weaker enhancement coefficients to avoid noise amplification. Contrast enhancement is achieved through linear transformation, multiplying the original pixel value by the corresponding enhancement coefficient and adding an offset to ensure the enhanced pixel value remains within the effective range, resulting in a locally contrast-enhanced image.

[0049] The wavelet transform denoising algorithm removes noise while preserving the geometric features of the parts through multi-scale decomposition. This algorithm decomposes the locally contrast-enhanced image into sub-band signals at different frequency scales. The wavelet decomposition process uses discrete wavelet transform, performing convolution operations on the image through low-pass and high-pass filters. The low-pass filter output contains the low-frequency information of the image, i.e., the main geometric features, while the high-pass filter output contains the high-frequency information of the image, i.e., edge details and noise. Multi-scale decomposition is achieved by recursively decomposing the low-frequency sub-bands, forming a multi-level frequency representation. Denoising processing is performed on the high-frequency sub-bands; by setting a threshold, high-frequency coefficients with amplitudes less than the threshold are set to zero, removing noise components while preserving useful edge information. The wavelet reconstruction process synthesizes the processed sub-band signals at each scale into a complete image through inverse wavelet transform, resulting in a part-enhanced image that has both removed noise and preserved the geometric features of the parts.

[0050] In one specific embodiment, the process of performing step S102 may specifically include the following steps:

[0051] The enhanced image of the part is input into the improved Canny edge detection algorithm and subjected to dual threshold adaptive adjustment to obtain the edge contour data of the part.

[0052] Based on the edge contour data of the part, the improved SURF algorithm is used to perform key point detection processing on the corner points, straight line intersections and curvature change points of the part to obtain the feature control points of the part.

[0053] Based on the control points of the part features, the Hough transform is used to perform geometric detection processing on the straight and circular features of the part to obtain the geometric topology of the part.

[0054] Based on the geometric topology of the part, the correlation constraint relationship between the three levels of features is verified by the feature consistency verification algorithm to obtain a reliable feature point set;

[0055] The reliable feature point set is input into a weighted voting mechanism for feature fusion processing to obtain a multi-dimensional feature set of the part that includes position, orientation and scale information.

[0056] Specifically, the improved Canny edge detection algorithm processes the enhanced part image through a dual-threshold adaptive adjustment mechanism. This algorithm first performs Gaussian filtering on the image to remove noise, then calculates the gradient magnitude and direction for each pixel. The dual-threshold adaptive adjustment dynamically sets high and low threshold parameters based on the part's material and surface roughness characteristics. The high threshold is used to identify strong edge pixels, and the low threshold is used to identify weak edge pixels. The algorithm determines the threshold range by analyzing the histogram distribution of the image's gradient magnitude. Pixels with gradient magnitudes higher than the high threshold are directly marked as edge points. Pixels with gradient magnitudes between the high and low thresholds need to have their neighborhood connectivity checked; only weak edge pixels connected to strong edge pixels are retained. The edge connection process is implemented through an 8-connected domain search, expanding from strong edge pixels to adjacent weak edge pixels to form a continuous edge chain, obtaining the part's edge contour data. This adaptive mechanism solves the problem of instability in traditional fixed-threshold methods on parts with different materials.

[0057] The improved SURF algorithm performs keypoint detection based on part edge contour data, and is specifically optimized for the regular geometric features of industrial parts. The algorithm first calculates the Hessian matrix response value of edge pixels. The Hessian matrix contains the second-order partial derivative information of the image at that point, and the probability of the point being a keypoint is evaluated by calculating the determinant value of the matrix. A non-maximum suppression algorithm is performed in three-dimensional space, comparing not only the response values ​​of a pixel with its spatial neighborhood but also its response values ​​at different scales to ensure that the detected keypoints are scale-invariant. For corners, line intersections, and curvature change points, the algorithm designs dedicated descriptor calculation methods. Corner detection analyzes the gradient distribution pattern around the keypoint; when the gradient direction changes drastically around the point, it is identified as a corner. Line intersection detection calculates the number and direction of line segments in the neighborhood of the keypoint; when two or more line segments with different directions intersect, it is identified as an intersection. Curvature change point detection calculates the local curvature of the edge contour. When the curvature value exceeds a set threshold, it is marked as a curvature change point, forming part feature control points containing different types of geometric features.

[0058] The Hough transform performs geometric detection of straight and circular features based on the control points of the part's features. This transform converts points in image space to a parameter space for feature detection. Straight line detection uses the standard Hough transform, mapping each edge point in the image to a sine curve in the parameter space. The intersection of multiple sine curves corresponds to a straight line feature in the image. The parameter space uses polar coordinates, including distance and angle parameters. The parameters of the straight line are determined by finding local maxima in the parameter space. Circular detection uses the circular Hough transform, mapping each edge point in the image to a three-dimensional parameter space, including the center coordinates and radius parameters. The algorithm uses a cumulative voting mechanism to find peak values ​​in the parameter space; the peak position corresponds to the detected circular feature parameters. A minimum voting threshold is set during the detection process to filter noise and false features; only parameter combinations with more than the threshold are considered valid geometric features. By analyzing the spatial relationships between the detected straight and circular features, a topological graph describing the part's geometry is established. This graph contains the parameter information and interconnections of each geometric element, forming the part's geometric topology.

[0059] The feature consistency verification algorithm performs correlation constraint verification on three levels of features: pixel-level edges, feature-level keypoints, and geometric-level topology. This algorithm identifies and eliminates false features by examining the spatial correspondence between features at different levels. Pixel-level and feature-level consistency verification checks whether each feature control point is located on its corresponding edge contour. By calculating the distance from the feature point to the nearest edge pixel, feature points with a distance exceeding a threshold are marked as inconsistent. Feature-level and geometric-level consistency verification checks whether the feature control points conform to the detected geometric features; corner points should be located at the intersection of straight lines, and curvature change points should be located at specific positions on arcs. Geometric-level internal consistency verification checks the logical relationships between different geometric features; intersecting straight lines should have corresponding corner features at their intersection points, and tangent straight lines and circles should have corresponding feature points at their tangency points. The verification process employs a voting mechanism, where each feature point is evaluated based on its consistency score across the three levels. Only feature points with scores exceeding a set threshold are retained in the reliable feature point set. This multi-level verification mechanism effectively solves the problems of inaccurate feature extraction and false feature interference in traditional methods.

[0060] A weighted voting mechanism performs feature fusion processing on a reliable feature point set. This mechanism assigns weight coefficients based on the stability and repeatability of feature points across different viewpoints. Stability is assessed by calculating the positional deviation of feature points in multiple detections; feature points with smaller positional deviations receive higher stability weights. Repeatability is assessed by examining the correspondence of feature points in images from different viewpoints; feature points that can be consistently detected across multiple viewpoints receive higher repeatability weights. Weight calculation comprehensively considers stability and repeatability weights, obtaining a comprehensive weight coefficient for each feature point through weighted averaging. The feature fusion process performs a weighted average of corresponding feature points from three viewpoints. Position information is calculated to obtain 3D coordinates through weighted averaging, orientation information is calculated to obtain the principal direction angle through weighted averaging, and scale information is calculated to obtain the feature scale through weighted averaging. The fused feature description vector contains the 3D position coordinates, principal direction angle, and feature scale of each feature point, forming a multi-dimensional feature set for the part.

[0061] In one specific embodiment, the process of performing key point detection processing on part corners, line intersections, and curvature change points using an improved SURF algorithm can specifically include the following steps:

[0062] The Hessian matrix is ​​input into the part edge contour data to calculate the response value and obtain the Hessian response value of the pixel point on the part edge.

[0063] Based on the Hessian response value, the local extrema of the response value are filtered out using a non-maximum suppression algorithm to obtain the coordinates of candidate key points.

[0064] Based on the position coordinates of candidate key points, the regular geometric features of the part are described by calculating special descriptors for line segments and arc segments to obtain key point feature descriptors;

[0065] Based on the key point feature descriptor, the curvature change detection algorithm is used to identify the locations of curvature abrupt changes in the part contour, thereby obtaining curvature change feature points;

[0066] The coordinates of curvature change feature points and candidate key points are fused and matched to obtain part feature control points that include corner points, straight line intersections and curvature change points.

[0067] Specifically, Hessian matrix calculation evaluates the local geometric characteristics of each edge pixel by performing second-order partial derivative operations on the part edge contour data. This matrix contains the second-order partial derivative information of the image at that point in the horizontal, vertical, and cross directions. The Hessian matrix calculation process first applies Gaussian filtering to the edge contour data, then calculates the second-order partial derivatives of the image in the horizontal, vertical, and cross directions. The second-order partial derivatives are obtained by differentiating the first-order gradient information again, reflecting the acceleration characteristics of image grayscale changes. The determinant value of the Hessian matrix is ​​used as the response value to evaluate the candidate strength of the pixel as a keypoint; a larger determinant value indicates more significant local geometric features at that point. The response value is calculated using the matrix determinant formula: the product of the horizontal and vertical second-order partial derivatives minus the square of the cross-second-order partial derivative. This calculation result reflects the curvature characteristics of the grayscale distribution around the pixel, yielding the Hessian response value corresponding to each edge pixel.

[0068] Non-maximum suppression (NMS) algorithms use Hessian response values ​​to filter local extrema. This algorithm identifies local maxima by comparing the response values ​​of each pixel with those of its neighbors. The algorithm operates in three-dimensional space, encompassing both the two-dimensional and scale-space coordinates of the image, ensuring that detected keypoints possess dual spatial and scale stability. The suppression process first sets a neighborhood window size, typically a 3x3 or 5x5 square window, and then compares the response values ​​of the center pixel with those of all its neighbors within the window. Only pixels with response values ​​greater than all neighboring pixels and exceeding a set threshold are retained as candidate keypoints; the response values ​​of other pixels are suppressed to zero. Scale-space NMS also compares the response values ​​of the pixel at adjacent scale levels to ensure that the point is also a local maximum in the scale dimension. After three-dimensional NMS, the remaining pixels form a set of candidate keypoints. Each candidate keypoint contains its two-dimensional coordinates in the image and its corresponding scale information, forming candidate keypoint position coordinate data.

[0069] The dedicated descriptor calculation for straight line segments and circular arc segments is designed for the regular geometric features of industrial parts. This descriptor determines the geometric feature type of a point by analyzing the gradient distribution pattern around the candidate keypoint. The calculation process for straight line segments first establishes a circular neighborhood around the keypoint, dividing the neighborhood into multiple fan-shaped regions, and then statistically analyzes the consistency of gradient directions within each fan-shaped region. When the gradient in a certain direction is dominant and its distribution is relatively concentrated, the keypoint is determined to be located on a straight line segment. The calculation for circular arc segments analyzes the continuous change characteristics of gradient directions; the gradient directions on a circular arc segment should exhibit a continuously gradually changing distribution pattern. The descriptor calculation also includes determining the principal direction. By statistically analyzing the histogram distribution of gradient directions within the keypoint's neighborhood, the dominant peak value of the gradient direction is found as the principal direction of the keypoint. Descriptor vector construction involves dividing the neighborhood into multiple sub-regions and calculating the gradient magnitude and direction statistics relative to the principal direction within each sub-region, forming a rotationally invariant feature descriptor. This descriptor can distinguish different geometric features on a part, such as straight edges, circular arc edges, and corner points, obtaining the keypoint feature descriptor corresponding to each candidate keypoint.

[0070] The curvature change detection algorithm identifies abrupt curvature changes on a part's contour based on key point feature descriptors. This algorithm detects points of abrupt geometric changes by calculating the local curvature of the contour line. The curvature calculation process first parametrically represents the part's edge contour, assigning each point on the contour an arc length parameter, and then calculates the first and second derivatives of the contour line at that point. The curvature value is obtained by the cross product of the first and second derivatives, reflecting the degree of curvature at that point. The algorithm calculates the curvature value point-by-point along the contour line, forming a curvature function curve, and then identifies curvature abrupt changes by analyzing the rate of change of the curvature function. Curvature abrupt change detection is performed by setting a curvature change threshold; when the curvature difference between adjacent points exceeds the threshold, it is marked as a curvature abrupt change point. This is particularly effective for corner points and arc connection points of the part, as these locations typically exhibit significant curvature abrupt changes. The algorithm also considers the directionality of curvature changes: positive curvature abrupt changes correspond to convex corner points, and negative curvature abrupt changes correspond to concave corner points, identifying all geometrically significant curvature change feature points.

[0071] The fusion matching process spatially maps and fuses the coordinates of curvature change feature points with those of candidate key points. This process establishes the correspondence by calculating the spatial distance and feature similarity between two sets of feature points. Spatial matching is performed by calculating the Euclidean distance between the curvature change feature points and the candidate key points; point pairs with a distance less than a set threshold are considered spatially corresponding. Feature matching compares the geometric attributes of the curvature change feature points with the feature descriptors of the candidate key points, including consistency of principal direction and matching of geometric feature types. Corner point matching requires that the curvature change feature points have significant curvature abrupt changes and that the feature descriptors of the candidate key points show straight line intersection features. Straight line intersection matching requires that two or more straight line segments intersect at that location and have corresponding corner point features. Curvature change point matching requires that the contour curvature changes continuously at that location and that the candidate key point is located on an arc or curve segment. Successfully matched feature point pairs undergo information fusion, integrating curvature information and key point descriptor information into a unified feature representation, resulting in a set of part feature control points containing three types: corner points, straight line intersections, and curvature change points.

[0072] In one specific embodiment, the process of executing step S103 may specifically include the following steps:

[0073] The multi-dimensional feature set of the part is input into the least squares method to fit the main geometric contour of the part, and the coordinates of the geometric center of the part are obtained.

[0074] Based on the geometric center coordinates of the part, the principal axis direction of the multi-dimensional feature set of the part is calculated by principal component analysis to obtain the principal axis feature vector of the part.

[0075] Based on the main axis feature vector of the part, the feature vector corresponding to the largest feature value is set as the X-axis direction and the feature vector corresponding to the second largest feature value is set as the Y-axis direction to obtain the coordinate axis direction vector of the part;

[0076] Based on the coordinate axis direction vector of the part, the Z-axis direction is determined by the right-hand rule, and a dynamic reference coordinate system is obtained with the geometric center coordinate of the part as the origin.

[0077] The dynamic reference coordinate system is input into the multi-view geometric constraint attitude solving algorithm to calculate the current attitude of the part, and the spatial attitude matrix of the part containing the rotation matrix and translation vector is obtained.

[0078] Specifically, the least squares fitting method determines the main geometric contour of a part by minimizing the sum of squared distances from feature points in the multi-dimensional feature set to the fitted curve. This method first selects a suitable fitting model based on the part's geometric characteristics. A circular fitting model is used for circular parts, a polygonal fitting model for rectangular parts, and a spline curve fitting model for complex-shaped parts. The fitting process establishes an objective function, using the sum of squared perpendicular distances from all feature points to the fitted contour as the optimization objective. The optimal fitting parameters are determined by minimizing the objective function. For circular fitting, the center coordinates and radius parameters are calculated; for rectangular fitting, the equation parameters of each edge are calculated; and for complex contour fitting, the coordinates of control points and curve parameters are calculated. The fitting algorithm employs an iterative optimization method, gradually adjusting the fitting parameters from the initial parameters until convergence to the optimal solution. The fitting quality is evaluated by calculating the fitting error and correlation coefficient to ensure the reliability of the fitting results. Based on the fitted geometric contour, the centroid position of the contour is calculated as the geometric center coordinates of the part. These coordinates are obtained by weighted averaging of the coordinates of all points on the fitted contour, with the weights determined according to the importance of each point on the contour.

[0079] Principal component analysis (PCA) calculates the principal axis directions of a part's multi-dimensional feature set based on the coordinates of its geometric center. This method determines the main directions of change of the part by analyzing the spatial distribution of feature points. The analysis process first translates the coordinates of all feature points to a coordinate system with the geometric center as the origin, eliminating the influence of positional offset. Then, a covariance matrix of the feature point coordinates is constructed, reflecting the distribution characteristics and correlations of feature points along different coordinate axes. The calculation of the covariance matrix involves statistical analysis of the translated feature point coordinates, calculating the covariance values ​​between different coordinate axes. Next, eigenvalue decomposition is performed on the covariance matrix to solve for the eigenvalues ​​and corresponding eigenvectors. Eigenvalue decomposition is achieved through numerical methods, including the Jacobi iteration method or QR decomposition. Eigenvalues ​​represent the distribution variance of feature points along the corresponding eigenvector direction; larger eigenvalues ​​indicate more significant changes in that direction. Eigenvectors represent the main directions of change of the covariance matrix, i.e., the principal axis directions of the part's feature point distribution. The eigenvectors obtained through eigenvalue decomposition constitute the set of principal axis eigenvectors of the part, describing the main geometric directions of the part in three-dimensional space.

[0080] The establishment of part coordinate axis direction vectors is based on the sorting and direction setting of the part's principal axis eigenvectors. This process first sorts the eigenvalues ​​from largest to smallest to determine the priority of the main changing directions. The eigenvector corresponding to the largest eigenvalue represents the direction of the greatest change in the distribution of part feature points, and is set as the X-axis direction. This direction typically corresponds to the part's major axis or main geometric direction. The eigenvector corresponding to the second largest eigenvalue represents the direction of the second greatest change in the distribution of part feature points, and is set as the Y-axis direction. This direction typically corresponds to the part's minor axis or secondary geometric direction. The selection of the X and Y axes ensures that the coordinate system best describes the geometric characteristics of the part, making the representation of the part in the new coordinate system the most concise. The directions of the eigenvectors need to be standardized to ensure that the vector length is unit length, facilitating subsequent coordinate transformation calculations. Simultaneously, the orthogonality of the X and Y axes needs to be checked. Although the eigenvectors obtained from principal component analysis are theoretically orthogonal, numerical calculations require verification and correction of potential errors. The correction process is achieved through the Schmitt orthogonalization method, ensuring that the X and Y axes are strictly orthogonal, ultimately yielding standardized part coordinate axis direction vectors.

[0081] The dynamic reference coordinate system is established by determining the Z-axis direction using the right-hand rule and setting the origin at the geometric center of the part. This coordinate system dynamically adjusts according to the actual geometric characteristics of the part, unlike the traditional fixed coordinate system. The process of determining the Z-axis direction using the right-hand rule is achieved by calculating the cross product of the X-axis vector and the Y-axis vector; the direction of the cross product is the positive direction of the Z-axis. The cross product calculation involves three components of the X-axis vector and the Y-axis vector, and the three components of the Z-axis vector are obtained through the cross product formula. Determining the Z-axis direction ensures that the coordinate system satisfies the conventions of a right-hand coordinate system, i.e., the X-axis, Y-axis, and Z-axis form a right-hand screw relationship. The origin is set to the previously calculated geometric center coordinates of the part, and this origin position best represents the spatial position of the part. The establishment process of the dynamic reference coordinate system also includes a consistency check of the coordinate system orientation to ensure that the orientation of the coordinate system conforms to the geometric characteristics of the part. Especially for parts with obvious orientation, it is necessary to ensure that the X-axis points to the primary direction of the part and the Y-axis points to the secondary direction. Once the coordinate system is established, all feature point coordinates are transformed to the new dynamic reference coordinate system, forming a standardized representation based on the geometric characteristics of the part.

[0082] The multi-view geometric constraint attitude calculation algorithm calculates the current spatial attitude of a part based on a dynamic reference coordinate system. This algorithm establishes a set of geometric constraint equations using the feature correspondences of three viewpoints. First, the algorithm establishes the correspondences of feature points between the three viewpoint images, determining the projection position of the same spatial point in different viewpoint images through feature matching. The correspondence establishment process combines feature descriptor matching and geometric consistency verification to ensure the accuracy and reliability of the matching. Based on the feature point correspondences, a multi-view geometric constraint equation set is established, describing the geometric relationships between spatial points, camera parameters, and image projections. The constraint equations use a perspective projection model to project three-dimensional spatial points onto a two-dimensional image plane. Due to the presence of observation data from three viewpoints, an overdetermined equation set is formed, with the number of equations exceeding the number of unknown parameters. Attitude calculation obtains the rotation matrix and translation vector of the part by solving the overdetermined equation set. The rotation matrix describes the orientation change of the part relative to the reference coordinate system, and the translation vector describes the position change of the part relative to the reference coordinate system. The solution process employs singular value decomposition, which can handle the overdetermined equation set and obtain the optimal solution in the least squares sense. The solution results contain six degrees of freedom attitude information, forming a part space attitude matrix that includes rotation matrix and translation vector.

[0083] In one specific embodiment, the process of executing step S104 may specifically include the following steps:

[0084] The difference between the part's spatial attitude matrix and the target attitude matrix is ​​calculated to obtain an attitude deviation matrix that includes rotation and displacement errors.

[0085] Based on the attitude deviation matrix, the partial derivatives of the part attitude parameters are calculated using the Jacobian matrix to obtain error sensitivity analysis data.

[0086] Based on error sensitivity analysis data, the position offset caused by the attitude adjustment of the end effector is calculated and processed through Z-axis offset and rotation coupling analysis to obtain the coupling error influence coefficient;

[0087] Based on the coupling error influence coefficient, the position offset during the attitude adjustment process is predicted by the inverse kinematics algorithm to obtain the predicted compensation trajectory.

[0088] The predicted compensation trajectory is input into the feedforward compensation mechanism for dynamic error compensation processing, resulting in compensated attitude parameters that include both static and dynamic compensation parameters.

[0089] Specifically, the attitude deviation matrix is ​​calculated by subtracting the part's spatial attitude matrix from the target attitude matrix. This calculation process handles the deviations of the rotation and translation parts separately. Rotation error calculation employs relative rotation matrix operations, performing a relative rotation calculation between the current rotation matrix and the target rotation matrix. The relative rotation matrix is ​​obtained through matrix multiplication, describing the rotational transformation required to reach the target attitude. The formula for the relative rotation matrix is ​​the target rotation matrix multiplied by the transpose of the current rotation matrix. The off-diagonal elements of the resulting matrix reflect the magnitude and direction of the rotation error. Displacement error calculation is obtained by directly subtracting the current position vector from the target position vector, yielding the displacement deviations in the X, Y, and Z directions in three-dimensional space. Each component of the displacement error vector represents the distance the part needs to adjust along the corresponding coordinate axis. The attitude deviation matrix integrates the rotation and displacement errors into a unified 4x4 homogeneous transformation matrix. The top-left 3x3 submatrix is ​​the rotation error matrix, and the top-right 3x1 subvector is the displacement error vector. This matrix fully describes the six degrees of freedom deviation between the part's current attitude and the target attitude.

[0090] The Jacobian matrix calculation is based on the partial derivative operation of the part's attitude parameters using the attitude deviation matrix. This matrix describes the influence of small changes in attitude parameters on the final alignment error. The construction process of the Jacobian matrix first converts the attitude deviation matrix into a six-dimensional vector representation, containing three rotation angle parameters and three displacement parameters. The rotation angle parameters are obtained by converting the rotation matrix to axis-angle representation, including rotation angles around the X, Y, and Z axes. The partial derivative calculation uses a numerical differentiation method, applying a small perturbation to each attitude parameter and calculating the change in alignment error before and after the perturbation. The partial derivative value is obtained by the ratio of the change to the perturbation. Each row of the Jacobian matrix corresponds to an error component, and each column corresponds to an attitude parameter. Matrix elements represent the sensitivity of the corresponding attitude parameter to the corresponding error component. The matrix calculation also considers the coupling relationships between parameters, especially the mutual influence between rotation and displacement parameters. Sensitivity analysis evaluates the system's stability by calculating the condition number and singular values ​​of the Jacobian matrix. Directions with larger condition numbers indicate higher error sensitivity, requiring more precise control. Finally, complete error sensitivity analysis data is obtained.

[0091] Z-axis offset and rotation coupling analysis specifically addresses the complex error coupling relationships generated during end-effector attitude adjustment. This analysis identifies key coupling error sources based on error sensitivity analysis data. The coupling analysis process first establishes a kinematic model of the end-effector, describing the mathematical relationship between the joint angles and the end-effector's position and attitude. Z-axis offset analysis focuses on the impact of vertical position changes on rotational adjustment. When the actuator rotates, the offset between the rotation center and the part's center of gravity causes unexpected positional changes in the Z-axis direction. Coupling error calculation analyzes the distance vector between the rotation center and the part's center of gravity, as well as the geometric relationship between the rotation angle and Z-axis displacement. The calculation process employs rigid body kinematics principles, decomposing the rotational motion into rotational components around different axes. The contribution of each rotational component to the Z-axis displacement is calculated through geometric projection. Influence coefficient calculation establishes a linear relationship model between the rotation angle and Z-axis displacement; the coefficient magnitude reflects the degree of influence of rotational adjustment on the Z-axis displacement. The coupling analysis also considers the case of simultaneous multi-axis rotation, calculating the total coupling error influence through the superposition principle, ultimately obtaining a coupling error influence coefficient matrix describing various coupling relationships.

[0092] The inverse kinematics algorithm predicts position offsets during attitude adjustment based on the coupling error influence coefficient. This algorithm determines the required joint motion trajectory by solving the inverse kinematics equations of the actuator. The inverse algorithm first calculates the required attitude adjustment, including rotational and displacement adjustments, based on the target and current attitudes. Then, it converts these attitude adjustments into the target position and orientation of the actuator end effector, and determines the target angles of each joint through inverse kinematics calculations. The inverse calculation process employs a numerical iterative method, gradually adjusting from the current joint angle until the end effector attitude requirements are met. In each iteration of the inverse calculation, the prediction process uses the coupling error influence coefficient to predict the possible position offset. The position offset prediction is obtained by multiplying the joint angle change by the influence coefficient matrix; the resulting vector describes the expected displacement deviation in each direction. The prediction algorithm also considers the influence of actuator dynamics, including the effects of inertia, friction, and flexibility on position accuracy. Trajectory optimization reduces the predicted position offset by adjusting the time allocation and velocity distribution of joint motions, forming a predicted compensation trajectory that satisfies the attitude adjustment requirements while minimizing position offset.

[0093] The feedforward compensation mechanism performs dynamic error compensation based on a predicted compensation trajectory. This mechanism pre-calculates and applies compensation amounts to offset anticipated error effects before executing attitude adjustment commands. Static compensation parameters are obtained through offline calibration, including inherent actuator error parameters such as joint clearance, transmission error, and systematic errors like structural deformation. A static compensation lookup table is established by conducting repeatable tests under different attitudes, statistically analyzing error distribution patterns under various conditions, and forming a compensation parameter table based on attitude and load. Dynamic compensation parameters are adjusted according to real-time motion state and environmental conditions, including velocity-related dynamic errors, inertial errors caused by acceleration, and thermal deformation errors caused by temperature changes. The feedforward controller design employs a predictive control algorithm, calculating compensation commands based on the predicted trajectory before the actuator begins movement. The calculation of the compensation command comprehensively considers the contributions of static and dynamic compensation, obtaining the total compensation amount through weighted superposition. During compensation execution, the compensation command is superimposed on the original control command to form a corrected actuator control command. The compensation effect is evaluated by real-time monitoring of the deviation between the actual trajectory and the expected trajectory; when the deviation exceeds the allowable range, the compensation parameters are automatically adjusted. The final output of the compensated attitude parameters includes rotation and displacement parameters that have been corrected by static and dynamic compensation, ensuring that the actuator can accurately achieve the expected attitude adjustment.

[0094] In one specific embodiment, the process of executing step S105 may specifically include the following steps:

[0095] The compensated attitude parameters are converted to the axis angle representation and then subjected to rotational deviation decomposition to obtain rotational deviation data containing the rotation axis and rotation angle.

[0096] Based on the rotational deviation data, the translational deviation is decomposed into displacements in the X, Y, and Z directions through three-dimensional coordinate decomposition to obtain three-dimensional displacement deviation data.

[0097] Based on three-dimensional displacement deviation data, the part alignment process is decomposed into multiple motion sub-stages using a piecewise linear interpolation method to obtain the staged motion parameters.

[0098] Based on the phased motion parameters, the motion speed is automatically reduced when approaching the target position through a variable velocity planning strategy to obtain an optimized alignment trajectory.

[0099] The optimized alignment trajectory is input into a closed-loop feedback control mechanism for real-time trajectory adjustment, resulting in a part alignment state that includes both positional and angular accuracy.

[0100] Specifically, the axis-angle representation conversion transforms the rotation matrix in the compensated attitude parameters into a combination of rotation axes and rotation angles, avoiding the singularity problem of Euler angle representation. The conversion process first extracts the rotation angle from the rotation matrix, determining its magnitude by calculating the trace of the rotation matrix. The rotation angle is equal to the result of applying an inverse cosine function to the trace of the matrix minus one and then dividing by two. The rotation axis is calculated by analyzing the antisymmetric part of the rotation matrix; the three components of the rotation axis vector are each equal to the difference between specific elements in the rotation matrix divided by the sine of the rotation angle. When the rotation angle is close to zero, a special handling method is used, employing Taylor expansion to avoid singularity problems in numerical calculations. The rotation axis vector needs to be normalized to ensure its length is unit length; normalization is achieved by multiplying the inverse of the vector length by the original vector. The advantage of axis-angle representation lies in its clear geometric meaning: the rotation axis indicates the direction of rotation, and the rotation angle indicates the magnitude of rotation, facilitating subsequent trajectory planning and control. The decomposition process also includes determining the direction of rotation. By using the right-hand rule, the positive direction of the rotation axis is associated with the sign of the rotation angle, ultimately yielding rotation deviation data containing the three-dimensional rotation axis vector and the scalar rotation angle.

[0101] Three-dimensional coordinate decomposition analyzes the directionality of translational deviations based on rotational deviation data. This decomposition process breaks down the composite spatial displacement into independent displacement components along the X, Y, and Z coordinate axes. The decomposition calculation first establishes a displacement analysis model related to rotational adjustment, considering the coupled effect of rotational motion on translational position. When a part rotates around a specific axis, the position of each point on the part changes accordingly; the amount of change is related to the distance from that point to the rotation axis and the rotation angle. Displacement decomposition is achieved through vector projection, projecting the total displacement vector onto the X, Y, and Z coordinate axes respectively to obtain the displacement components in each axis direction. The projection calculation uses dot product operations; the dot product of the displacement vector and the magnitude of the displacement along a unit coordinate axis is the displacement component in that direction. The decomposition process also considers the choice of coordinate system, using a coordinate system that matches the geometric characteristics of the part to ensure the physical meaning of the decomposition results. X-direction displacement typically corresponds to length adjustment, Y-direction displacement to width adjustment, and Z-direction displacement to height adjustment. The decomposition results form three-dimensional displacement deviation data, which clearly describes the amount of displacement adjustment required for the part in each direction, providing detailed motion requirement information for subsequent trajectory planning.

[0102] The piecewise linear interpolation method decomposes the part alignment process into multiple continuous motion sub-stages based on three-dimensional displacement deviation data. This method achieves a smooth motion trajectory by inserting several intermediate positions between the starting and target positions. The interpolation process first determines the number and position of segments. The number of segments is determined based on the total adjustment distance and accuracy requirements; more segments are used when the adjustment distance is large to ensure smooth motion. The calculation of intermediate positions employs either an equal division strategy or a variable step size strategy. The equal division strategy distributes the total adjustment evenly across each sub-stage, while the variable step size strategy dynamically adjusts the length of each segment according to the complexity of the motion. Linear interpolation calculation is achieved by establishing a linear relationship between adjacent position points, with the position within each sub-stage changing according to a linear function. Interpolation parameters include continuity constraints on position, velocity, and acceleration to ensure a smooth transition of motion parameters between sub-stages. The piecewise processing also considers the coordination of rotational and translational motions, ensuring temporal consistency between rotational and displacement adjustments through synchronous planning. Each motion sub-stage contains the starting position, ending position, motion time, and velocity parameters for that stage, forming a complete set of staged motion parameters. The division of motion sub-stages also takes into account the dynamic constraints of the actuator, including physical constraints such as maximum speed, maximum acceleration, and joint angle limits.

[0103] The variable speed planning strategy adaptively adjusts the motion speed of the part as it approaches the target position based on staged motion parameters. This strategy balances alignment efficiency and accuracy by dynamically adjusting the motion speed at each stage. Speed ​​planning employs either a trapezoidal or S-shaped speed curve. The trapezoidal curve includes acceleration, constant speed, and deceleration stages, while the S-shaped curve adds acceleration variations during the acceleration and deceleration stages for smoother motion. The planning process dynamically determines the motion speed based on the distance between the current and target positions; a higher speed is used for greater distances to improve efficiency, and a lower speed for closer distances to improve accuracy. The speed adjustment function uses exponential decay or polynomial functions, with function parameters adjusted according to the part's weight, inertia, and accuracy requirements. Automatic speed reduction is achieved by setting speed and distance thresholds. When the distance between the part and the target position is less than the set threshold, the speed is automatically reduced to a precision adjustment speed. Speed ​​planning also considers the coordination of motion in different directions, ensuring that the motion speeds in the X, Y, and Z directions are matched to avoid unnecessary coupling errors. The optimization process uses iterative calculations to find the optimal combination of speed parameters to minimize the total alignment time while meeting accuracy requirements. The final optimized alignment trajectory contains position, velocity, and acceleration information at each moment, forming complete motion planning data.

[0104] The closed-loop feedback control mechanism performs real-time trajectory adjustment based on the optimized aligned trajectory. This mechanism detects and corrects trajectory deviations by continuously monitoring the actual position and attitude of the parts. The feedback control system comprises three main components: a position sensor, an attitude sensor, and a controller. The position sensor provides real-time three-dimensional coordinate information of the parts, and the attitude sensor provides real-time rotation angle information. The controller employs a PID control algorithm or a more advanced adaptive control algorithm, calculating the correction control quantity by comparing the deviation between the actual trajectory and the expected trajectory. The deviation calculation includes two parts: position deviation and attitude deviation. The position deviation is obtained by the vector difference between the actual position and the expected position, and the attitude deviation is obtained by the relative transformation between the actual attitude matrix and the expected attitude matrix. The control algorithm calculates the corresponding correction quantity based on the magnitude and trend of the deviation, and the correction quantity includes the contributions of proportional, integral, and derivative terms. Real-time adjustment processing superimposes the correction quantity onto the original control command to form a real-time updated actuator control command. The trajectory adjustment also includes an anomaly detection function, automatically triggering an emergency stop or reprogramming when the deviation exceeds the safe range. Accuracy assessment is achieved by statistically analyzing the deviation between the final alignment result and the target state. Positional accuracy is obtained by calculating the root mean square value of the positional deviation, and angular accuracy is obtained by calculating the root mean square value of the angular deviation. The final assessment of the part alignment status includes determining whether the alignment was successful, evaluating the accuracy level, and statistically analyzing repeatability indicators, resulting in a complete alignment quality report.

[0105] The above describes the vision-based industrial parts alignment method in the embodiments of this application. The following describes the vision-based industrial parts alignment system in the embodiments of this application. Please refer to [link to relevant documentation]. Figure 2 One embodiment of the visual analysis-based industrial parts alignment system in this application includes:

[0106] The acquisition module is used to acquire and process images of industrial parts through a three-view camera, and to preprocess the acquired images by gradient amplitude local contrast enhancement to obtain enhanced images of the parts.

[0107] The recognition module is used to identify the edge contours and key control points of the part based on the enhanced image of the part through hierarchical feature extraction, so as to obtain a multi-dimensional feature set of the part.

[0108] The setting module is used to set the geometric center of the part as the origin of the coordinate system and establish a dynamic reference coordinate system based on the multi-dimensional feature set of the part, so as to obtain the spatial posture matrix of the part.

[0109] The compensation module is used to compensate for the attitude deviation based on the difference between the part's spatial attitude matrix and the target attitude, through Z-axis offset and rotational coupling error analysis, to obtain the compensated attitude parameters.

[0110] The optimization module is used to decompose the attitude adjustment process into multiple sub-stages based on the compensated attitude parameters and optimize the alignment trajectory through a variable speed planning strategy to obtain the part alignment state.

[0111] above Figure 2 The vision analysis-based industrial parts alignment system in this embodiment of the invention will be described in detail from the perspective of modular functional entities. The vision analysis-based industrial parts alignment device in this embodiment of the invention will be described in detail from the perspective of hardware processing.

[0112] Reference Figure 3 This invention also provides a vision analysis-based industrial parts alignment device, which can be a server, and its internal structure can be as follows: Figure 3 As shown, the vision-based industrial parts alignment device includes a processor, memory, display screen, input device, network interface, and database connected via a system bus. The processor, designed as a computer, provides computing and control capabilities. The memory of the vision-based industrial parts alignment device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the vision-based industrial parts alignment device stores the data corresponding to this embodiment. The network interface of the vision-based industrial parts alignment device is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the above-described method.

[0113] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the vision analysis-based industrial parts alignment device to which the present invention is applied.

[0114] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of the vision analysis-based industrial part alignment method.

[0115] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0116] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a vision-based industrial parts alignment device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0117] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for aligning industrial parts based on visual analysis, characterized in that, The method includes: The images of industrial parts are acquired and processed by a three-view camera, and the acquired images are preprocessed by gradient amplitude local contrast enhancement to obtain enhanced images of the parts. Based on the enhanced image of the part, the edge contour and key control points of the part are identified and processed by hierarchical feature extraction to obtain a multi-dimensional feature set of the part; Based on the multi-dimensional feature set of the part, the geometric center of the part is set as the origin of the coordinate system and a dynamic reference coordinate system is established to obtain the spatial posture matrix of the part. Based on the difference between the spatial attitude matrix of the part and the target attitude, the attitude deviation is compensated by Z-axis offset and rotation coupling error analysis to obtain the compensated attitude parameters. Based on the compensated attitude parameters, the attitude adjustment process is decomposed into multiple sub-stages, and the alignment trajectory is optimized using a variable velocity planning strategy to obtain the part alignment state. This includes: converting the compensated attitude parameters into an axis-angle representation for rotational deviation decomposition to obtain rotational deviation data containing the rotation axis and rotation angle; decomposing the translational deviation into displacements in the X, Y, and Z directions using three-dimensional coordinate decomposition to obtain three-dimensional displacement deviation data; decomposing the part alignment process into multiple motion sub-stages using piecewise linear interpolation to obtain staged motion parameters; automatically reducing the motion speed when approaching the target position using a variable velocity planning strategy to obtain an optimized alignment trajectory; and inputting the optimized alignment trajectory into a closed-loop feedback control mechanism for real-time trajectory adjustment to obtain the part alignment state including positional and angular accuracy.

2. The industrial part alignment method based on visual analysis according to claim 1, characterized in that, The process involves acquiring and processing images of industrial parts using a three-view camera, and preprocessing the acquired images using gradient magnitude local contrast enhancement to obtain enhanced images of the parts, including: Two cameras are installed at a 45-degree angle for binocular stereo vision acquisition, and a third camera is used to capture top-down images vertically downwards, resulting in original images of the parts from multiple perspectives. Based on the original multi-view images of the part, the brightness distribution of the LED ring light source is adjusted by a regional light intensity control algorithm to obtain a part image that eliminates reflective interference. Based on the gradient direction consistency index of the 8-neighborhood of the pixel in the part image, the edge region of the part is automatically identified to obtain edge region identification data. Based on the edge region identification data, the contrast of different regions is differentially enhanced by calculating the gradient magnitude to obtain a locally contrast-enhanced image; The local contrast-enhanced image is input into a wavelet transform denoising algorithm for multi-scale denoising processing to obtain the enhanced image of the part.

3. The industrial parts alignment method based on visual analysis according to claim 1, characterized in that, The process involves identifying and processing the edge contours and key control points of the part based on the enhanced image of the part through hierarchical feature extraction, resulting in a multi-dimensional feature set of the part, including: The enhanced image of the part is input into the improved Canny edge detection algorithm for dual threshold adaptive adjustment processing to obtain the edge contour data of the part. Based on the edge contour data of the part, the improved SURF algorithm is used to perform key point detection processing on the corner points, straight line intersections and curvature change points of the part to obtain the feature control points of the part. Based on the control points of the part features, the straight and circular features of the part are geometrically detected by Hough transform to obtain the geometric topology of the part. Based on the geometric topology of the part, the correlation constraint relationship between the three levels of features is verified by the feature consistency verification algorithm to obtain a reliable feature point set; The reliable feature point set is input into a weighted voting mechanism for feature fusion processing to obtain a multi-dimensional feature set of the part that includes position, orientation, and scale information.

4. The industrial parts alignment method based on visual analysis according to claim 3, characterized in that, The step involves using the improved SURF algorithm to perform key point detection processing on the corner points, line intersections, and curvature change points of the part based on the part edge contour data, to obtain the part feature control points, including: The Hessian matrix is ​​input into the edge contour data of the part for response value calculation, and the Hessian response value of the edge pixel of the part is obtained. Based on the Hessian response value, the local extrema of the response value are filtered using a non-maximum suppression algorithm to obtain the coordinates of candidate key points. Based on the position coordinates of the candidate key points, the regular geometric features of the part are described by calculating the special descriptors for line segments and arc segments to obtain the key point feature descriptors; Based on the key point feature descriptor, the curvature change detection algorithm is used to identify the curvature change abrupt location of the part contour to obtain curvature change feature points; The curvature change feature points are fused and matched with the candidate key point position coordinates to obtain the part feature control points, which include corner points, straight line intersections, and curvature change points.

5. The industrial parts alignment method based on visual analysis according to claim 1, characterized in that, The step of setting the geometric center of the part as the origin and establishing a dynamic reference coordinate system based on the multi-dimensional feature set of the part to obtain the spatial attitude matrix of the part includes: The multi-dimensional feature set of the part is input into the least squares method for fitting the main geometric contour of the part to obtain the geometric center coordinates of the part. Based on the geometric center coordinates of the part, the principal axis direction of the multi-dimensional feature set of the part is calculated by principal component analysis to obtain the principal axis feature vector of the part. Based on the main axis feature vector of the part, the feature vector corresponding to the largest feature value is set as the X-axis direction and the feature vector corresponding to the second largest feature value is set as the Y-axis direction to obtain the coordinate axis direction vector of the part; Based on the coordinate axis direction vector of the part, the Z-axis direction is determined by the right-hand rule, and a dynamic reference coordinate system is obtained with the geometric center coordinate of the part as the origin. The dynamic reference coordinate system is input into the multi-view geometric constraint attitude calculation algorithm to calculate the current attitude of the part, and the spatial attitude matrix of the part containing the rotation matrix and translation vector is obtained.

6. The industrial parts alignment method based on visual analysis according to claim 1, characterized in that, The process involves compensating for the attitude deviation based on the difference between the part's spatial attitude matrix and the target attitude through Z-axis offset and rotational coupling error analysis, resulting in compensated attitude parameters, including: The difference between the spatial attitude matrix of the part and the target attitude matrix is ​​calculated to obtain an attitude deviation matrix that includes rotation error and displacement error; Based on the attitude deviation matrix, the partial derivatives of the part attitude parameters are calculated using the Jacobian matrix to obtain error sensitivity analysis data. Based on the error sensitivity analysis data, the position offset caused by the attitude adjustment of the end effector is calculated and processed through Z-axis offset and rotation coupling analysis to obtain the coupling error influence coefficient; Based on the coupling error influence coefficient, the position offset during the attitude adjustment process is predicted using the inverse kinematics algorithm to obtain the predicted compensation trajectory. The predicted compensation trajectory is input into the feedforward compensation mechanism for dynamic error compensation processing to obtain the compensated attitude parameters, which include static compensation parameters and dynamic compensation parameters.

7. An industrial parts alignment system based on visual analysis, characterized in that, For implementing the vision analysis-based industrial part alignment method as described in any one of claims 1-6, the vision analysis-based industrial part alignment system comprises: The acquisition module is used to acquire and process images of industrial parts through a three-view camera, and to preprocess the acquired images by gradient amplitude local contrast enhancement to obtain enhanced images of the parts. The recognition module is used to identify the edge contour and key control points of the part based on the enhanced image of the part through hierarchical feature extraction, so as to obtain a multi-dimensional feature set of the part. The setting module is used to set the geometric center of the part as the origin of the coordinate system and establish a dynamic reference coordinate system based on the multi-dimensional feature set of the part, so as to obtain the spatial posture matrix of the part. The compensation module is used to compensate for the attitude deviation based on the difference between the part's spatial attitude matrix and the target attitude, through Z-axis offset and rotational coupling error analysis, to obtain the compensated attitude parameters. The optimization module is used to decompose the attitude adjustment process into multiple sub-stages based on the compensated attitude parameters and optimize the alignment trajectory using a variable speed planning strategy to obtain the part alignment state. This includes: converting the compensated attitude parameters into an axis-angle representation and performing rotational deviation decomposition to obtain rotational deviation data containing the rotation axis and rotation angle; decomposing the translational deviation into displacements in the X, Y, and Z directions using three-dimensional coordinate decomposition to obtain three-dimensional displacement deviation data; decomposing the part alignment process into multiple motion sub-stages using a piecewise linear interpolation method based on the three-dimensional displacement deviation data to obtain staged motion parameters; automatically reducing the motion speed when approaching the target position using a variable speed planning strategy based on the staged motion parameters to obtain an optimized alignment trajectory; and inputting the optimized alignment trajectory into a closed-loop feedback control mechanism for real-time trajectory adjustment to obtain the part alignment state including positional and angular accuracy.

8. An industrial parts alignment device based on visual analysis, characterized in that, It includes a memory and a processor, the memory storing a computer program that can run on the processor, the processor executing the computer program to implement the visual analysis-based industrial part alignment method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is run by the processor, it causes the processor to execute the visual analysis-based industrial part alignment method as described in any one of claims 1 to 6.

Images