Irregular workpiece machining and positioning method and system combined with machine vision

Abstract

The invention relates to an irregular workpiece processing and positioning method and system combined with machine vision, and the method comprises the steps: firstly obtaining a multi-view image data set which completely covers different areas of the surface of a to-be-processed workpiece, and providing rich and complete original materials for subsequent processing; three-dimensional reconstruction processing is carried out on the multi-view image data set, and generated three-dimensional model data can visually and accurately present the three-dimensional form of the workpiece to be machined; then a spatial feature set is extracted from three-dimensional model data, the spatial features of the workpiece can be deeply described from multiple dimensions, and a comprehensive and meticulous basis is provided for accurate positioning; and finally, the spatial feature set and a preset workpiece machining reference model are subjected to feature matching processing, generated positioning information can effectively drive machining equipment to execute workpiece machining path adjustment operation, the actual form of the workpiece can be dynamically adapted, the machining path is optimized, the machining precision and efficiency of the machining equipment are improved, and the machining efficiency is improved. And high-quality completion of workpiece machining is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision positioning, and more particularly to a method and system for machining and positioning irregular workpieces in combination with machine vision. Background Art

[0002] In the field of mechanical processing, the positioning of irregular workpieces is crucial. With the development of the manufacturing industry, workpiece shapes are becoming increasingly complex and diverse, and traditional positioning methods often fail to meet the demands of high-precision machining. Due to the unique shape of irregular workpieces, accurately determining their position within the machining coordinate system during machining has become a key issue in improving machining quality and efficiency.

[0003] When faced with irregular workpieces, existing machining positioning technologies mostly rely on simple two-dimensional measurements or limited feature extraction, failing to fully and accurately capture the workpiece's spatial characteristics, resulting in insufficient positioning accuracy. Furthermore, these methods struggle to adapt to complex and changing workpiece shapes, lacking in-depth exploration and comprehensive analysis of the workpiece's spatial characteristics, and are unable to effectively utilize the workpiece's diverse characteristics for precise positioning.

[0004] Therefore, when processing irregular workpieces, existing processing positioning technology cannot fully obtain multi-perspective image information of the workpiece, it is difficult to generate accurate three-dimensional model data, and it cannot deeply extract effective spatial features to achieve precise positioning and processing path adjustment. A new technical solution is urgently needed to improve the processing quality and efficiency of irregular workpieces. Summary of the Invention

[0005] In view of this, the present invention provides a method and system for machining and positioning irregular workpieces combined with machine vision.

[0006] An embodiment of the present invention provides an irregular workpiece processing and positioning method combined with machine vision, which is applied to an irregular workpiece processing and positioning system, the method comprising: acquiring a multi-perspective image data set of a workpiece to be processed, the multi-perspective image data set comprising image acquisition data covering different surface areas of the workpiece to be processed; performing three-dimensional reconstruction processing on the multi-perspective image data set to generate three-dimensional model data of the workpiece to be processed; extracting a spatial feature set of the three-dimensional model data, the spatial feature set comprising surface curvature distribution features, contour topological structure features and key geometric constraint features of the workpiece to be processed; performing feature matching processing on the spatial feature set and a preset workpiece processing reference model to generate positioning information of the workpiece to be processed in a processing coordinate system, the positioning information being used to drive the processing equipment to perform a workpiece processing path adjustment operation.

[0007] The present invention also provides an irregular workpiece processing and positioning system, comprising: a memory for storing program instructions and data; and a processor for coupling with the memory and executing instructions in the memory to implement the above method.

[0008] The present invention also provides a computer storage medium comprising instructions, which implement the above method when executed on a processor.

[0009] The embodiment of the present invention first obtains a multi-view image data set that comprehensively covers different surface areas of the workpiece to be processed, providing rich and complete original materials for subsequent processing; then the multi-view image data set is subjected to 3D reconstruction processing to generate 3D model data that can intuitively and accurately present the three-dimensional shape of the workpiece to be processed; then a spatial feature set is extracted from the 3D model data, which can deeply characterize the spatial characteristics of the workpiece from multiple dimensions, providing a comprehensive and detailed basis for precise positioning; finally, the spatial feature set is feature-matched with a preset workpiece processing reference model, and the generated positioning information can effectively drive the processing equipment to perform workpiece processing path adjustment operations, can dynamically adapt to the actual shape of the workpiece, optimize the processing path, improve the processing accuracy and efficiency of the processing equipment, and ensure high-quality completion of workpiece processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0011] Figure 1 A schematic flow chart of the steps of a method for machining and positioning an irregular workpiece combined with machine vision provided by an embodiment of the present invention.

[0012] Figure 2 This is a structural block diagram of an irregular workpiece processing and positioning system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0013] The technical solutions of the present invention will be described below in conjunction with the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation methods described in the following exemplary embodiments do not represent all implementation methods consistent with the present invention. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present invention. It should be noted that the terms "first", "second", etc. in the specification of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0014] See also Figure 1 , Figure 1This is a flow chart of an irregular workpiece processing and positioning method combined with machine vision provided by an embodiment of the present invention. The method is applied to an irregular workpiece processing and positioning system and may further include steps 110 to 140.

[0015] Step 110: Acquire a multi-view image data set of the workpiece to be processed, wherein the multi-view image data set includes image acquisition data covering different regions of the surface of the workpiece to be processed.

[0016] In an embodiment of the present invention, the workpiece to be processed has an irregular shape and a complex surface structure. In order to obtain a multi-view image data set, a plurality of industrial cameras are placed around the part. These industrial cameras are precisely arranged to ensure that they can cover different areas of the part surface. For example, camera A is set directly above the part, camera B is located at a 45-degree angle to the left of the part, and camera C is at a 60-degree angle to the right of the part, etc. Each camera is configured with set shooting parameters, such as the shooting resolution of camera A is set to 2048×1536 pixels and the frame rate is 10 frames per second; the resolution of camera B is 2592×1944 pixels and the frame rate is 8 frames per second, etc.

[0017] During the capture process, the part is placed on a stable workbench and kept stationary, allowing the camera to clearly capture its surface details. Each camera is activated simultaneously, capturing the part according to its own parameters. This ultimately yields a multi-view image dataset containing images of the part taken from various angles and positions. These images comprehensively cover different areas of the part's surface, providing a rich data foundation for subsequent 3D reconstruction.

[0018] Step 120: Perform three-dimensional reconstruction processing on the multi-view image data set to generate three-dimensional model data of the workpiece to be processed.

[0019] Next, the acquired multi-view image data set undergoes 3D reconstruction, generating 3D model data that accurately reflects the true shape and size of the workpiece being machined from these 2D image data. In this embodiment of the present invention, the 3D reconstruction process integrates and processes image information from each viewpoint to construct the 3D spatial structure of the part. This process involves multiple steps, including denoising, feature point extraction, 3D coordinate mapping, and surface fitting.

[0020] In an optional embodiment, performing three-dimensional reconstruction processing on the multi-view image data set to generate three-dimensional model data of the workpiece to be processed includes: Step 121: performing noise suppression processing on each image acquisition data in the multi-view image data set to obtain a denoised image data set.

[0021] In embodiments of the present invention, industrial cameras may be affected by various interference factors during the capture process, resulting in noise in the image. For example, electromagnetic interference in the environment and electronic noise from the camera itself can degrade image quality, affecting subsequent processing and analysis. Therefore, it is necessary to perform noise suppression on each image captured in the multi-view image dataset. Taking the image captured by camera A as an example, this image may contain Gaussian noise, which manifests as random grayscale variations within the image. A Gaussian filter algorithm is used to denoise the image. This algorithm smoothes the image and suppresses noise by taking a weighted average of each pixel in the image and its neighboring pixels. Specifically, for each pixel in the image, a weighted average is calculated based on the position and weight of its surrounding pixels, and this weighted average is used as the grayscale value of the pixel after denoising. After performing a similar Gaussian filter on the images captured by all cameras, a denoised image dataset is obtained. These images have significantly reduced noise and improved clarity, providing a more accurate data foundation for subsequent feature point extraction.

[0022] Step 122: extracting a feature point set of each image acquisition data in the denoised image data set, wherein the feature point set includes edge intersection points, curvature mutation points and texture key points.

[0023] It can be understood that the denoised image data set provides a good basis for feature point extraction. For the image of the workpiece to be processed, the extraction of feature points can help determine the key positions and structural information of the part surface. For example, in the image taken by camera B, the edge intersection points must be extracted first. The edge intersection points are the points where the part outline and the internal structure lines intersect. Next, the curvature mutation points must be identified. Curvature mutation points usually appear where the shape of the part surface changes drastically, such as the transition from a plane to a curved surface or the connection between curved surfaces of different curvatures. For texture key points, in this part image, the texture may come from the processing texture of the part surface or the texture characteristics of the material itself. Texture key points are the connection points between texture repeating units, which can reflect the distribution and direction of the texture. By extracting these different types of feature points, the characteristics of the part image can be fully described, providing key information for subsequent three-dimensional reconstruction.

[0024] In a preferred embodiment, extracting a feature point set of each image acquisition data in the denoised image data set includes: Step 1221: Calling an edge detection algorithm to perform gradient calculation on the denoised image data set to determine pixel areas in the image data whose pixel gradient amplitudes exceed a preset gradient threshold.

[0025] Taking an image captured by camera C as an example, the Canny edge detection algorithm is used to calculate the gradient. The Canny edge detection algorithm first calculates the gradient magnitude and direction for each pixel in the image. For each pixel in the image, the gradient magnitude and direction are obtained by calculating the rate of change of the grayscale in the horizontal and vertical directions. For example, for pixel P, the horizontal grayscale difference Gx and the vertical grayscale difference Gy are calculated. The gradient magnitude and direction are then calculated. The gradient magnitude is equal to the square root of the sum of the squares of the horizontal grayscale difference and the squares of the vertical grayscale difference, and the gradient direction is equal to the inverse tangent of the ratio of the vertical grayscale difference to the horizontal grayscale difference. A gradient threshold can be preset based on actual conditions, such as 50. The calculated gradient magnitude for each pixel is compared with the threshold to identify pixel regions where the gradient magnitude exceeds the threshold. These pixel regions typically correspond to the edges of objects in the image, as edges have more pronounced grayscale changes and larger gradient magnitudes.

[0026] Step 1222: Detect intersections between edge line segments within the pixel area, and add the intersections as edge intersections to the feature point set.

[0027] After determining the pixel area where the pixel gradient amplitude exceeds the preset gradient threshold, the area is further processed. Within this area, a method based on the Hough transform is used to detect the intersections between edge segments. The Hough transform can convert the straight line in the image space into the parameter space for representation, and determine the parameters of the straight line by finding the peak in the parameter space, thereby finding the straight line in the image. For the detected straight lines, the intersection points between them are calculated. For example, the coordinates of the intersection points are calculated by combining the equations of the straight lines (the equation of straight line 1 is y=k1x+b1, and the equation of straight line 2 is y=k2x+b2. By solving the system of equations composed of these two equations, the coordinates of the intersection points are obtained). These intersection points are added to the feature point set as edge intersection points. These edge intersection points can accurately identify the intersection position of different edge segments in the part image, which plays an important role in determining the contour and structure of the part.

[0028] Step 1223: Perform curvature analysis on the denoised image data set, identify a target area whose curvature change amplitude exceeds a preset curvature change threshold, and extract the center point of the target area as a curvature mutation point to be added to the feature point set.

[0029] For the denoised image data set, an algorithm based on curvature calculation is used to perform curvature analysis on the image. Taking the image taken by camera A as an example, for each pixel in the image, the curvature of the point is estimated by calculating the grayscale change of the surrounding pixels. Specifically, the curvature information is obtained by calculating the second-order derivative around the pixel (by performing a second-order difference calculation on the grayscale value around the pixel, the second-order derivative is obtained, and then the curvature is calculated). A curvature change threshold is preset, for example, set to 0.1. When the curvature change amplitude in one area exceeds the threshold, the area is identified as the target area. Then, the center point of the target area is extracted as the curvature mutation point and added to the feature point set. Curvature mutation points usually appear where the surface shape of the part changes sharply, such as the transition from a plane to a curved surface or the connection between surfaces of different curvatures. These points are of great significance for understanding the surface structure and shape changes of the part.

[0030] Step 1224: Perform texture similarity analysis on the denoised image data set, determine the connection points between texture repeating units and add them to the feature point set as texture key points.

[0031] Optionally, texture similarity analysis is performed on the denoised image data set, taking the image taken by camera B as an example. A method based on the grayscale co-occurrence matrix is ​​used to analyze the texture. The grayscale co-occurrence matrix can describe the frequency of occurrence of pixel pairs with different grayscale values ​​in an image at a certain distance and direction. The characteristics of the texture are measured by calculating the relevant features of the grayscale co-occurrence matrix, such as contrast, correlation, energy, and entropy. For the texture in the image, texture repeating units are found. For example, in the processed texture on the surface of a part, there may be a certain periodic pattern, which is the texture repeating unit. The connection points between the texture repeating units are determined. These connection points can reflect the distribution and direction of the texture. These connection points are added to the feature point set as texture key points. Texture key points play an important role in further analyzing the surface features and material properties of the part.

[0032] Step 123: Based on the spatial projection relationship of the feature point set between different image acquisition data, determine the three-dimensional coordinate mapping relationship of the feature point set, and generate initial three-dimensional point cloud data of the workpiece to be processed according to the three-dimensional coordinate mapping relationship.

[0033] After acquiring a set of feature points from each image acquisition data set, the 3D coordinate mapping relationship is determined based on the spatial projection relationships between these feature points across the different images. For multi-view images of the workpiece to be processed, different spatial projection relationships exist between the feature points at different viewpoints. By analyzing these relationships, the feature points in the 2D image can be mapped into 3D space, generating initial 3D point cloud data. This process involves multiple steps, including the use of camera parameters, cross-view matching of feature points, and the establishment of a projection ray model, to ensure the accurate generation of the initial 3D point cloud data.

[0034] As an optional embodiment, determining the three-dimensional coordinate mapping relationship of the feature point set based on the spatial projection relationship between the feature point set and different image acquisition data includes: Step 1231: Obtain a camera parameter set calibrated by each image acquisition device in a multi-view image data set, wherein the camera parameter set includes a focal length parameter, an imaging plane rotation matrix, and a camera space position coordinate.

[0035] In this embodiment of the present invention, it is necessary to obtain the calibrated camera parameter sets for each industrial camera placed around the workpiece to be processed. For example, the focal length parameter of camera A is 50 mm. The imaging plane rotation matrix is ​​determined by the camera's installation angle and posture. Its rotation matrix is ​​R1 (rotation matrix R1 is a 3×3 matrix whose elements are r11, r12, r13, r21, r22, r23, r31, r32, and r33, respectively. These elements are calculated based on the specific installation angle and posture of camera A). The camera's spatial position coordinates are (x1, y1, z1), which represent the position of camera A in the world coordinate system. Similarly, the focal length parameter of camera cameraB is 45mm, the imaging plane rotation matrix is ​​R2 (elements are r41, r42, r43, r51, r52, r53, r61, r62, r63), and the camera space position coordinates are (x2, y2, z2); the focal length parameter of camera cameraC is 55mm, the imaging plane rotation matrix is ​​R3 (elements are r71, r72, r73, r81, r82, r83, r91, r92, r93), and the camera space position coordinates are (x3, y3, z3). These camera parameter sets provide important basic data for the subsequent determination of the three-dimensional coordinate mapping relationship of the feature points.

[0036] Step 1232: Perform cross-view matching on each feature point in the feature point set, and extract the corresponding two-dimensional pixel coordinate set of the feature point in different image acquisition data, where each two-dimensional pixel coordinate includes horizontal and vertical coordinate values ​​in pixels.

[0037] Optionally, cross-view matching is performed for each feature point in the feature point set. For example, in the image captured by camera A, the 2D pixel coordinates of the feature point chP are determined to be (u1, v1) through the previous feature point extraction step. Then, points corresponding to this feature point are searched in the images captured by camera B and camera C. Feature point descriptor matching methods, such as the SIFT (Scale-Invariant Feature Transform) feature descriptor, are used to calculate the similarity of the feature point descriptors in different images. For feature point chP, a matching point is found in the image captured by camera B, with 2D pixel coordinates (u2, v2); a matching point is found in the image captured by camera C, with 2D pixel coordinates (u3, v3). Thus, the corresponding 2D pixel coordinates of feature point chP in the different image acquisition data are extracted: {(u1, v1), (u2, v2), (u3, v3)}. Each 2D pixel coordinate contains horizontal and vertical coordinate values ​​in pixel units, which provide the basis for subsequent coordinate transformation and 3D coordinate calculation.

[0038] Step 1233: Convert each two-dimensional pixel coordinate into a normalized plane coordinate according to the focal length parameter of the corresponding camera in the camera parameter set to eliminate the dimension mismatch caused by the size difference of the image sensor.

[0039] Furthermore, each 2D pixel coordinate is converted based on the focal length parameter of the corresponding camera in the camera parameter set. For example, the 2D pixel coordinates (u1, v1) of the feature point chP in the image captured by camera A are converted to normalized plane coordinates (xn1, yn1). In this embodiment of the present invention, the abscissa of the normalized plane coordinates is equal to the abscissa of the 2D pixel coordinate minus the abscissa of the image center, divided by the focal length. The same applies to the ordinate. For the 2D pixel coordinates (u2, v2) of the feature point chP in the image captured by camera B, the focal length parameter of camera B is f2. Similarly, the conversion is performed to obtain the normalized plane coordinates (xn2, yn2). For the 2D pixel coordinates (u3, v3) of the feature point chP in the image captured by camera C, the focal length parameter of camera C is f3. The conversion is performed to obtain the normalized plane coordinates (xn3, yn3). Through this conversion, the dimension mismatch problem caused by the size difference of image sensors is eliminated, so that the coordinates of feature points in images taken by different cameras can be subsequently processed at a unified scale.

[0040] Step 1234: Based on the imaging plane rotation matrix and camera space position coordinates of each camera, generate a projection ray model for each feature point, wherein the projection ray model is expressed by a spatial line parameter model determined by the camera optical center position coordinates and the normalized plane coordinates.

[0041] Optionally, a projection ray model is generated for each feature point based on the imaging plane rotation matrix and camera space coordinates of each camera. For example, for camera A, the optical center coordinates of camera A are (x1, y1, z1), and the normalized plane coordinates of feature point chP in camera A's image are (xn1, yn1). Based on this information, a projection ray model is generated, which can be expressed using a spatial line parameter model (a spatial line parameter model: a point on a line has the camera optical center coordinates (x1, y1, z1), and the line's direction vector is determined by the normalized plane coordinates and the camera optical center coordinates. Let the direction vector be (dx, dy, dz), then the parametric equation of the line is x = x1 + t * dx, y = y1 + t * dy, z = z1 + t * dz, where t is a parameter). Similarly, for cameras B and C, projection ray models are generated based on their optical center coordinates and the normalized plane coordinates of feature point chP in their respective images. These projection ray models provide an important basis for the subsequent calculation of the 3D coordinates of the feature points.

[0042] Step 1235: Calculate the spatial intersection point between multiple projection rays corresponding to the same feature point, and use the three-dimensional coordinate point with the minimum sum of the squares of the Euclidean distances from each projection ray to the spatial intersection point as the initial three-dimensional coordinate of the feature point.

[0043] For feature point chP, there are multiple projection rays from cameras cameraA, cameraB, and cameraC. Compute the spatial intersection of these projection rays. This intersection is found by solving the equations for the projection rays (the parametric equations for the three projection rays are combined to form a system of equations. Solving this system of equations yields the coordinates of the spatial intersection). There may be multiple intersections. In this case, the sum of the squared Euclidean distances from each projection ray to each intersection is calculated. (For one intersection point (x0, y0, z0), the squared Euclidean distance from camera A's projection ray to that intersection point is calculated as (x0-x1)^2+(y0-y1)^2+(z0-z1)^2, the squared Euclidean distance from camera B's projection ray to that intersection point is calculated as (x0-x2)^2+(y0-y2)^2+(z0-z3)^2, and the squared Euclidean distance from camera C's projection ray to that intersection point is calculated as (x0-x3)^2+(y0-y3)^2+(z0-z3)^2. These three square sums are added together to obtain the sum of the squared Euclidean distances. The 3D coordinate point with the smallest sum of the squared Euclidean distances is used as the initial 3D coordinate of the feature point chP. This calculation is performed for all feature points to obtain the initial 3D coordinates of all feature points. These coordinates constitute the initial 3D point cloud data of the workpiece to be processed.

[0044] Step 1236: Iteratively optimize the feature points whose projection residuals exceed the preset tolerance threshold, construct a regularization term based on the three-dimensional coordinate space continuity constraint of adjacent feature points, and use the nonlinear least squares method to adjust the initial three-dimensional coordinates to minimize the weighted sum of the projection residual and the regularization term.

[0045] After obtaining the initial three-dimensional coordinates of the feature points, check whether there are feature points whose projection residuals exceed the preset tolerance threshold. The preset tolerance threshold is set to 0.05, for example. For one of the feature points chQ, if its projection residual exceeds the threshold, iterative optimization is required. The regularization term is constructed based on the continuity constraint of the three-dimensional coordinate space of adjacent feature points. For example, the adjacent feature points of feature point chQ are Q1, Q2, and Q3, and their three-dimensional coordinates are (xq1, yq1, zq1), (xq2, yq2, zq2), and (xq3, yq3, zq3), respectively. The regularization term is constructed by calculating the distance and direction relationship between the feature point chQ and its adjacent feature points. (For example, the regularization term can be set to the sum of the squares of the distance differences between the feature point chQ and its adjacent feature points, i.e., [(xq - xq1)^2 + (yq - yq1)^2 + (zq - zq1)^2] + [(xq - xq2)^2 + (yq - yq2)^2 + (zq - zq2)^2] + [(xq - xq3)^2 + (yq - yq3)^2 + (zq - zq3)^2], where (xq, yq, zq) are the coordinates of the feature point chQ.) The initial 3D coordinates are adjusted using a nonlinear least squares method. By continuously adjusting the coordinate values, the weighted sum of the projection residual and the regularization term is minimized. For example, the weight of the projection residual is set to 0.8, the weight of the regularization term is set to 0.2, and the coordinate value is continuously optimized through iterative calculation until a certain convergence condition is met, such as the weighted sum of the projection residual and the regularization term is less than one of the minimum values ​​(for example, 0.001). At this time, the optimized three-dimensional coordinates of the feature point are obtained.

[0046] Step 1237: Generate a triangular patch topological connection structure based on the spatial adjacency relationship of the adjusted three-dimensional coordinates of all feature points, and determine a three-dimensional coordinate mapping relationship that matches the quantitative dimension of the image acquisition perspective based on the triangular patch topological connection structure.

[0047] For the three-dimensional coordinates of all feature points after iterative optimization, a triangular patch topological connection structure is generated according to the spatial adjacency relationship. Taking the feature points of the workpiece to be processed as an example, the feature points adjacent in space are connected to form triangular patches according to the relative position relationship between the feature points. By traversing all feature points, a series of interconnected triangular patches are constructed to form a complete triangular patch topological connection structure, which not only reflects the spatial relationship between the feature points, but also provides a basis for determining the three-dimensional coordinate mapping relationship. Based on the triangular patch topological connection structure, combined with the number and layout of viewpoints of the previously captured image, a three-dimensional coordinate mapping relationship that matches the dimension is determined. For example, if there are three viewpoints to capture images, the generated three-dimensional coordinate mapping relationship must be able to accurately integrate the feature point information under different viewpoints into the three-dimensional space, so that the generated initial three-dimensional point cloud data can accurately reflect the actual shape and position information of the workpiece to be processed, and prepare for subsequent surface fitting and three-dimensional model generation.

[0048] Step 124: performing surface fitting processing on the initial three-dimensional point cloud data to eliminate fitting errors caused by point cloud density differences and generate three-dimensional model data of a continuous surface structure.

[0049] Optionally, the initial 3D point cloud data may have differences in point cloud density due to factors such as the acquisition process, which can affect the quality and accuracy of the final 3D model data. Therefore, it is necessary to perform surface fitting processing on the initial 3D point cloud data to eliminate the fitting errors caused by differences in point cloud density and generate 3D model data with a continuous surface structure to more accurately represent the shape of the workpiece to be processed. In an embodiment of the present invention, the surface fitting process needs to take into account the complex shape of the part surface and the distribution of point clouds in different areas, and achieve the generation of a continuous surface through reasonable algorithms and steps.

[0050] In a preferred embodiment, performing surface fitting processing on the initial three-dimensional point cloud data to eliminate fitting errors caused by point cloud density differences and generate three-dimensional model data of a continuous surface structure includes: Step 1241: Divide the initial three-dimensional point cloud data into a plurality of sub-point cloud regions with uniform density, wherein the difference in point cloud density in each sub-point cloud region is less than a preset density threshold.

[0051] For the initial three-dimensional point cloud data of the workpiece to be processed, the area is first divided. The preset density threshold is, for example, that the difference in the number of points per cubic centimeter does not exceed 10 points. From the overall distribution of the point cloud data, it is divided according to the spatial position and density of the points. For example, for a relatively dense area in the point cloud data, it is divided into a sub-point cloud area through a spatial clustering algorithm (such as the DBSCAN algorithm, which divides the point cloud data into different clusters based on the concept of density connection) to ensure that the point cloud density in the sub-point cloud area is relatively uniform and the point cloud density difference is less than the preset density threshold. Similarly, similar divisions are performed on other areas, and finally the initial three-dimensional point cloud data is divided into multiple sub-point cloud areas with uniform density. These sub-point cloud areas provide a more stable data basis for subsequent local surface model fitting.

[0052] Step 1242: Perform local surface model fitting on each sub-point cloud region to generate multiple local surface patches, where the local surface patches include at least one of a plane, a cylindrical surface, and a spherical surface.

[0053] For each divided sub-point cloud region, a local surface model is fitted. For example, the point cloud distribution in one sub-point cloud region may exhibit certain geometric characteristics. If the point cloud distribution approximates a plane, a plane fitting algorithm (e.g., a least-squares-based plane fitting algorithm, which determines the plane equation by minimizing the sum of squared distances between points and the plane) is used to fit the plane, resulting in a local surface patch of the plane. If the point cloud distribution exhibits cylindrical characteristics, such as a point cloud distribution approximately cylindrical around an axis, a cylindrical surface fitting algorithm (which establishes a mathematical model of the cylinder and uses the point cloud data to determine the cylinder's parameters, such as radius and axis direction) is used to fit the cylindrical surface. If the point cloud distribution resembles a sphere, a spherical surface fitting algorithm (which matches the point cloud data with the spherical equation and uses an optimization algorithm to determine the spherical parameters, such as the center coordinates and radius) is used to obtain a local surface patch of the sphere. By performing this local surface model fitting on each sub-point cloud region, multiple local surface patches containing at least one of the following: plane, cylinder, and sphere.

[0054] Step 1243: According to the geometric continuity constraints between the local surface patches of adjacent sub-point cloud regions, the boundary parameters of the local surface patches are adjusted to generate a globally continuous target surface structure; and the target surface structure is converted into closed three-dimensional model data.

[0055] Optionally, local surface patches within adjacent sub-point cloud regions must satisfy geometric continuity constraints to generate a globally continuous target surface structure. For example, for adjacent planar and cylindrical local surface patches, a smooth transition must be ensured at their boundaries. This is achieved by adjusting the boundary parameters of the local surface patches. At the junction of the planar and cylindrical local surface patches, parameters such as the plane normal and the cylindrical generatrix direction are adjusted based on their spatial positional relationship and geometric characteristics to ensure continuous geometric changes at the junction. By adjusting the boundary parameters of all adjacent local surface patches, a globally continuous target surface structure is formed. This target surface structure is then converted into closed 3D model data. This may involve triangulating the surface (dividing the surface into multiple triangular patches to form a closed mesh structure) and adding necessary topological information. Ultimately, a complete, closed 3D model data is generated that accurately represents the shape of the workpiece to be machined.

[0056] Step 130: extracting a spatial feature set of the three-dimensional model data, wherein the spatial feature set includes surface curvature distribution features, contour topology features, and key geometric constraint features of the workpiece to be processed.

[0057] Optionally, after obtaining the 3D model data of the continuous surface structure, its spatial feature set needs to be extracted to further determine the positioning information of the workpiece to be machined in the machining coordinate system. This spatial feature set includes surface curvature distribution features, contour topology features, and key geometric constraint features, which can comprehensively describe the spatial geometric characteristics of the workpiece. Taking the workpiece to be machined as an example, these features are of great significance for analyzing the part's shape, structure, and matching relationship with the machining reference model, providing a key basis for subsequent feature matching and positioning information generation.

[0058] In an optional embodiment, extracting the spatial feature set of the three-dimensional model data includes: Step 131: Meshing the surface of the three-dimensional model data to generate a set of evenly distributed mesh nodes.

[0059] Optionally, for the three-dimensional model data of the workpiece to be processed, its surface is meshed. A suitable meshing algorithm is used, such as the octree meshing algorithm. The algorithm first divides the space containing the entire three-dimensional model data into a large cube, and then recursively divides the cube into smaller sub-cubes according to the shape and complexity of the model surface until a certain accuracy requirement is met. In this process, uniformly distributed mesh nodes are generated on the model surface. For example, for the relatively flat surface part of the workpiece to be processed, the distribution of mesh nodes is relatively sparse, but still remains uniform; for parts with complex shapes and large curvature changes, the distribution of mesh nodes will be denser to better capture surface details. Finally, a set of uniformly distributed mesh nodes is generated, which provides basic position information for the subsequent calculation of spatial features such as surface curvature distribution features.

[0060] Step 132: Calculate the curvature value at each grid node, and generate surface curvature distribution features according to the change trend of the curvature value between adjacent grid nodes.

[0061] For each mesh node in the generated mesh node set, calculate its curvature value. Taking one of the mesh nodes as an example, the curvature is calculated by performing a local analysis of the surface around the node. This can be done using a method based on the change in the surface normal vector (e.g., calculating the rate of change of the normal vector in different directions around the node, and obtaining the curvature value of the node according to a certain mathematical formula (e.g., calculating the derivative of the normal vector in the local coordinate system to obtain the curvature value)). After calculating the curvature values ​​of all mesh nodes, observe the changing trend of the curvature values ​​between adjacent mesh nodes. For example, on a mesh line, from node A to node B to node C, the curvature value may gradually increase, or the curvature value may remain relatively stable within a region and then suddenly change. Based on these changing trends, surface curvature distribution features are generated. The curvature values ​​can be visualized according to certain color codes or numerical ranges to facilitate more intuitive analysis of the surface curvature distribution. For example, areas with smaller curvature values ​​are represented in blue, and areas with larger curvature values ​​are represented in red. This allows for clear visibility of which parts of the part surface have gentle curvature changes and which parts have drastic changes, providing important information for subsequent feature matching with the benchmark model.

[0062] Step 133: extracting the contour edges of the three-dimensional model data, and generating contour topology structure features according to the connection relationship and branch structure of the contour edges.

[0063] To extract contour edges from three-dimensional model data, an algorithm based on boundary detection can be used. For example, contour edges can be determined by analyzing the connection relationship of mesh nodes and the direction change of surface normal vectors. For the three-dimensional model of the workpiece to be processed, the contour edges may include the outer edges of the part and the edges of some internal structures. After extracting these contour edges, their connection relationships and branching structures are analyzed. For example, some contour edges may be interconnected to form a closed loop, while some may have branches, where a contour edge branches into two or more edges in different directions. Based on these connection relationships and branching structures, contour topological structure features are generated. This topological structure can be represented by a graph or data structure. For example, a graph structure can be used, with nodes representing the endpoints of contour edges and edges representing contour edges. In this way, the topological relationship between contour edges is clearly displayed, providing a basis for topological isomorphism analysis with the preset workpiece processing benchmark model.

[0064] Step 134: Generate key geometric constraint features based on the constraint areas in the three-dimensional model data that meet preset geometric constraint conditions, where the geometric constraint conditions include at least one of parallelism, perpendicularity, and coaxiality.

[0065] In the three-dimensional model data, find the constraint area that meets the preset geometric constraint conditions. Taking parallelism as an example, preset a tolerance range for judging parallelism, for example, the angle deviation between two straight lines or two planes is within ±0.1 degrees and is considered to be parallel. Search for all straight lines or planes that may be parallel in the model data to determine the constraint area that meets the parallelism condition. For perpendicularity, also preset a tolerance range, such as the angle between two straight lines or planes is within 90°±0.1° and is considered to be perpendicular, find the area that meets the perpendicularity condition. For coaxiality, set an allowable deviation range, for example, the distance deviation between the axes of two cylindrical surfaces is within ±0.05mm and is considered to be coaxial, determine the area that meets the coaxiality condition. Based on these constraint areas that meet the preset geometric constraint conditions, generate key geometric constraint features. Furthermore, these features can be recorded through adaptive data structures or description methods, such as recording the identification of straight lines or planes that meet parallelism, element relationships that meet perpendicularity, and cylindrical surface information that meets coaxiality. These key geometric constraint features play an important role in verifying constraint compliance with the benchmark model and determining the positioning information of the workpiece.

[0066] Step 140: performing feature matching processing on the spatial feature set and a preset workpiece processing reference model to generate positioning information of the workpiece to be processed in a processing coordinate system, and the positioning information is used to drive the processing equipment to perform a workpiece processing path adjustment operation.

[0067] The extracted spatial feature set of the workpiece to be processed is matched against a preset workpiece machining benchmark model. The goal is to determine the workpiece's exact position and posture in the machining coordinate system. This allows the machining equipment to adjust the machining path based on this positioning information, ensuring machining accuracy and precision. In the machining scenario, the preset workpiece machining benchmark model represents the ideal state and standard for part machining. By matching the spatial features of the actual workpiece to be machined, differences between the two can be identified, thereby determining adjustment strategies.

[0068] As an optional embodiment, performing feature matching processing on the spatial feature set and a preset workpiece processing reference model to generate positioning information of the workpiece to be processed in a processing coordinate system includes: Step 141: Acquire a reference space feature set of the workpiece processing reference model, wherein the reference space feature set includes a reference surface curvature distribution, a reference contour topology structure, and a reference geometric constraint.

[0069] For the machining reference model of the workpiece to be machined, the curvature distribution of its reference surface is obtained by performing a curvature calculation on the surface of the ideal model similar to the previous one, but the calculation here is based on a precise mathematical model and design parameters. For example, one of the surfaces of the reference model is designed as a surface with a set curvature, and the curvature distribution of the surface is obtained through theoretical calculation. The reference contour topology is determined based on the design drawings, and the connection relationship and branch structure of the contour edges are clarified to form a standard topological structure. The reference geometric constraints are also determined based on the design requirements, such as stipulating that certain planes must be strictly parallel, certain axes must be coaxial, etc. These reference geometric constraints constitute part of the reference space feature set, which provides a reference standard for subsequent matching with the spatial features of the workpiece to be machined.

[0070] Step 142: performing a curvature similarity comparison between the surface curvature distribution characteristics of the workpiece to be processed and the reference surface curvature distribution to determine a first matching degree.

[0071] In this process, to account for differences in spatial position and scale between the two, certain alignment and normalization processes are required. For example, the mesh nodes of the two are first spatially aligned, so that the corresponding nodes are as close as possible in spatial position. Then, the curvature values ​​are normalized, mapping them to a uniform range, such as [0, 1], for a fair comparison. The difference in curvature values ​​for each corresponding mesh node is calculated. For example, for a mesh node node P1 in the surface curvature distribution of the workpiece to be machined, its curvature value is k1, and the curvature value of the corresponding node node P2 in the reference surface curvature distribution is k2. The difference between them is calculated as |k1-k2|. The proportion of mesh node pairs whose curvature value difference is less than a preset curvature tolerance threshold (e.g., 0.05) is counted as the initial matching degree, which reflects the initial similarity in curvature values ​​between the two.

[0072] In an alternative embodiment, comparing the surface curvature distribution characteristics of the workpiece to be processed with the reference surface curvature distribution for curvature similarity to determine the first matching degree includes: Step 1421: Grid alignment is performed on the surface curvature distribution feature and the reference surface curvature distribution to obtain multiple grid node pairs, each of which includes a grid node corresponding to the surface curvature distribution feature and a grid node corresponding to the reference surface curvature distribution, and the spatial positions of the two grid nodes in each grid node pair correspond one to one.

[0073] To perform grid alignment between the surface curvature distribution characteristics and the reference surface curvature distribution, a method based on the iterative closest point (ICP) algorithm can be used. First, a set of initial corresponding points is randomly selected. Then, through continuous iterative adjustment, the distance between the grid nodes of the surface curvature distribution characteristics of the workpiece to be processed and the grid nodes of the reference surface curvature distribution is minimized. In each iteration, the distance from each grid node to its nearest corresponding node is calculated, and the position of the node is adjusted based on these distances. After multiple iterations, multiple grid node pairs are obtained, and the two grid nodes in each pair correspond one-to-one in spatial position. For example, through multiple iterations of the ICP algorithm, one of the grid nodes nodeA on the surface of the workpiece to be processed forms a grid node pair with the grid node nodeA' on the reference surface that is closest in position and has similar geometric meaning. And so on, multiple pairs are obtained.

[0074] Step 1422: Calculate the difference in curvature values ​​between the two mesh nodes in each mesh node binary, and count the proportion of mesh node binary groups whose curvature value difference is less than a preset curvature tolerance threshold as the initial matching degree.

[0075] For each obtained mesh node binary, calculate the difference in curvature values ​​between the two mesh nodes. For example, for the mesh node binary (nodeA, nodeA'), the curvature value of nodeA is ka, and the curvature value of nodeA' is ka'. Calculate the curvature value difference |ka-ka'|. Preset a curvature tolerance threshold, such as 0.05. Count the number of binary pairs with a curvature value difference less than the threshold among all mesh node binary pairs, and calculate their proportion to the total number of binary pairs, and use this proportion as the initial matching degree. For example, if there are a total of 100 mesh node binary pairs, and the curvature value difference of 70 binary pairs is less than 0.05, then the initial matching degree is 70%. This initial matching degree reflects the initial similarity between the surface curvature distribution of the workpiece to be processed and the reference model.

[0076] Step 1423: According to the directional consistency of the curvature change gradient in the surface curvature distribution feature, the initial matching degree is corrected to generate a first matching degree.

[0077] After obtaining the initial matching degree, the surface curvature distribution features are corrected by considering the directional consistency of the curvature gradient. For the surface curvature distributions of the workpiece and the reference model, the curvature gradient direction is calculated at each grid node. For example, the gradient direction is determined by calculating the rate of change of the curvature value in different directions. (For a grid node, several adjacent directions are selected around it, the curvature value change in these directions is calculated, and the curvature gradient direction is determined based on the magnitude and direction of the change.) The curvature gradient directions at the corresponding grid nodes of the workpiece and the reference model are compared. If the curvature gradient directions of most corresponding nodes are consistent, it indicates that the two surfaces have similar surface shape trends, and the initial matching degree is appropriately increased. If the curvature gradient directions of many nodes are inconsistent, the initial matching degree is reduced. For example, if the curvature gradient directions of 80% of the corresponding grid nodes are consistent, the initial matching degree of 70% can be increased to 75%, resulting in a first matching degree. This first matching degree more accurately reflects the similarity between the surface curvature distributions of the workpiece and the reference model, providing more reliable data for the subsequent comprehensive calculation of positioning information.

[0078] Step 143: performing a topological isomorphism analysis on the contour topological structure features of the workpiece to be processed and the reference contour topological structure to determine a second matching degree.

[0079] A topological isomorphism analysis is performed between the contour topology of the workpiece to be machined and the topology of the reference contour. Topological isomorphism analysis determines whether two topologies are essentially identical, regardless of their specific shape and position. First, the contour topologies of both the workpiece to be machined and the reference model are represented as graphs, where nodes represent the endpoints of contour edges and edges represent contour edges.

[0080] A graph matching algorithm is then used to compare the two graph structures. For example, a graph matching method based on the Hungarian algorithm can be used. This algorithm seeks the optimal match between nodes in the two graphs, maximizing the sum of the weights of the matching edges (weights can be determined based on factors such as node attributes and edge lengths). During the matching process, topological properties such as node degree (the number of edges connected to a node) and edge connectivity are considered. If the two graphs can be completely overlapped by rearranging nodes and edges, they are considered topologically isomorphic. The proportion of successfully matched nodes and edges is calculated to determine the second degree of matching. For example, if the graph matching algorithm finds that 60% of the nodes and edges in the contour topology of the workpiece to be machined and the reference model can be successfully matched, the second degree of matching is 60%. This second degree of matching reflects the degree of similarity between the contour topology of the workpiece to be machined and the reference model, providing an important basis for comprehensive evaluation of workpiece positioning.

[0081] Step 144: performing constraint compliance verification on the key geometric constraint features of the workpiece to be processed and the reference geometric constraints to determine a third matching degree.

[0082] Detailed constraint compliance verification is performed on the key geometric constraint features of the workpiece to be machined and the baseline geometric constraints. For parallelism constraints, the parallelism deviation between the areas of the workpiece that meet the parallelism conditions and the corresponding areas in the baseline model is checked. For example, the baseline model requires that plane PA and plane PB are parallel, with a deviation within ±0.1 degrees. The corresponding planes PA' and PB' are found in the workpiece to be machined, and the angular deviation between them is calculated using a measurement algorithm (an algorithm that calculates the angle between plane normal vectors based on spatial vectors). If the deviation is within the allowable range, the parallelism constraint is met. The proportion of areas that meet the parallelism constraint to the total number of parallelism-constrained areas is calculated.

[0083] A similar check is performed for perpendicularity constraints. For example, if the baseline model specifies that line LC and plane PD are perpendicular, with a deviation within 90° ± 0.1°, find the corresponding line LC' and plane PD' in the workpiece to be machined. Verify perpendicularity by calculating the angle between the line's direction vector and the plane's normal vector. The percentage of cases where the perpendicularity constraint is met is calculated relative to the total number of cases where the constraint is met.

[0084] For coaxiality constraints, check the coaxiality deviation between cylindrical surfaces and corresponding structures in the reference model. For example, if the reference model requires cylindrical surfaces E and F to be coaxial within ±0.05 mm, find the corresponding cylindrical surfaces E' and F' in the workpiece and verify their coaxiality by measuring the distance deviation between the axes. Count the percentage of surfaces that meet the coaxiality constraint.

[0085] The third degree of fit is calculated by comprehensively verifying geometric constraints such as parallelism, perpendicularity, and coaxiality. For example, after detailed verification, if the parallelism, perpendicularity, and coaxiality meet the requirements at 70%, 80%, and 65%, respectively, a weighted calculation (0.3 for parallelism, 0.3 for perpendicularity, and 0.4 for coaxiality) is used to determine the third degree of fit. This reflects the degree of conformance between the workpiece to be machined and the reference model with respect to key geometric constraints.

[0086] Step 145: Calculate the position deviation of the workpiece to be processed relative to the workpiece processing reference model based on the weighted result of the first matching degree, the second matching degree, and the third matching degree to generate the positioning information.

[0087] In one embodiment, step 145 is implemented as follows: The first, second, and third matching degrees are weighted and summed according to preset weighting coefficients to generate a global matching degree. These weighting coefficients are adjusted based on the influence of surface curvature distribution, contour topology, and key geometric constraints on machining positioning. For example, surface curvature distribution, which has a greater impact on machining positioning, is assigned a weight of 0.4; contour topology is assigned a weight of 0.3; and key geometric constraints are assigned a weight of 0.3.

[0088] When the global matching degree exceeds a preset matching degree threshold, a set of matching feature point pairs corresponding to the surface curvature distribution features, the contour topology features, and the key geometric constraint features is extracted. The matching feature point pair set includes the three-dimensional coordinates of the feature points of the workpiece to be processed and the three-dimensional coordinates of the corresponding reference feature points in the workpiece processing reference model. For example, the preset matching degree threshold is 65%. When the global matching degree reaches 69.3% and exceeds this threshold, feature points in areas with high matching degrees are found from the surface curvature distribution features, such as areas with small differences in curvature values ​​and consistent curvature gradient directions. Feature points corresponding to successfully matched nodes are selected from the contour topology features based on the graph matching results. Feature points in areas that meet the geometric constraint conditions are selected from the key geometric constraint features. The three-dimensional coordinates of the feature points of the workpiece to be processed and the three-dimensional coordinates of the corresponding reference feature points in the reference model are combined to form a set of matching feature point pairs.

[0089] Coordinate transformation parameters are calculated for the set of matched feature point pairs. Based on the least squares spatial alignment error between the three-dimensional coordinates of the feature points and the three-dimensional coordinates of the reference feature points, the rotation matrix and translation vector of the workpiece to be machined relative to the workpiece machining reference model are calculated. Using a least squares algorithm, an error function is constructed (the error function is the sum of the squares of the differences between the three-dimensional coordinates of the feature points and the three-dimensional coordinates of the reference feature points in each coordinate axis direction) and the parameters of the rotation matrix and translation vector are optimized to minimize the error function. For example, through iterative calculations, the parameters of the rotation matrix and translation vector are adjusted until the error function converges to a small value, thereby obtaining a rotation matrix and translation vector that meet the requirements.

[0090] Based on the rotation matrix and the translation vector, the position deviation of the workpiece to be processed in the processing coordinate system is determined, and the position deviation includes the offset in the six degrees of freedom directions. The rotation matrix and the translation vector determine the rotation and displacement relationship of the workpiece to be processed relative to the reference model in space. By decomposing the rotation matrix (for example, using the Euler angle decomposition method to convert the rotation matrix into rotation angles around the three coordinate axes), the rotation angles around the X, Y, and Z axes can be obtained. These three angles represent the position deviation of the workpiece in terms of rotation; the three components of the translation vector represent the translation deviation in the X, Y, and Z axes respectively, and the offset in the six degrees of freedom directions is obtained in combination.

[0091] If the global match does not exceed the preset match threshold, the weight distribution coefficients are adjusted and the weighted summation operation is re-executed until the global match meets the preset match threshold or the maximum number of iterations is reached. For example, if the global match is 60% but does not reach the 65% threshold, the weight distribution coefficients are adjusted according to a specific strategy, such as increasing the weight of the surface curvature distribution feature to 0.5, adjusting the weight of the contour topology feature to 0.25, and adjusting the weight of the key geometric constraint feature to 0.25, and then recalculating the global match. This process is repeated until the global match meets the threshold or the maximum number of iterations is reached (for example, the maximum number of iterations is set to 10).

[0092] The positioning information including coordinate system conversion parameters is generated based on the posture deviation. The coordinate system conversion parameters are used to map the current posture of the workpiece to be processed to the target posture of the workpiece processing reference model. Based on the calculated offsets in the six degrees of freedom directions, the corresponding coordinate system conversion parameters are generated. These parameters can be represented as a transformation matrix or a set of parameters. Through this transformation matrix or parameter set, the posture of the workpiece to be processed in the current coordinate system can be accurately converted to a target posture consistent with the reference model.

[0093] Optionally, the method further includes: An inverse kinematic solution is performed on the coordinate system conversion parameters to generate a displacement adjustment amount for driving the movement of each axis of the processing equipment. The displacement adjustment amount is kinematically verified based on the mechanical structure parameters of the processing equipment to constrain the displacement adjustment amount to be within the travel range of the processing equipment. The verified displacement adjustment amount is encapsulated into a control instruction format to be recognized by the processing equipment to generate a data packet of the positioning information.

[0094] Perform an inverse kinematics calculation on the coordinate system transformation parameters. The processing equipment is a multi-axis CNC machine tool with three linear axes (LX, LY, and LZ) and three rotary axes (RA, RB, and RC). Based on the kinematic model of the processing equipment (which describes the relationship between the motion of each axis and the change in the workpiece posture), the coordinate system transformation parameters are substituted into the inverse kinematics algorithm. For example, for the rotation angle and translation distance in the posture deviation, the inverse kinematics algorithm calculates the required displacement for each axis.

[0095] The calculated displacement adjustment is verified using the kinematic forward solution based on the mechanical parameters of the machining equipment. For example, if the lead screw pitch of the machining equipment's LX axis is 5mm, the calculated LX axis displacement adjustment is substituted into the forward solution model using the kinematic forward solution algorithm to calculate the actual position change of the workpiece in space under this displacement adjustment. This actual position change is checked to ensure that it matches the expected position deviation. Furthermore, the displacement adjustment is checked to ensure that it is within the travel range of each axis of the machining equipment, such as the LX axis travel range of [0, 1000mm], the LY axis travel range of [0, 800mm], and the LZ axis travel range of [0, 600mm].

[0096] If the displacement adjustment passes verification, it is packaged into a control instruction format to be recognized by the processing equipment. For example, the control instruction format can be a code sequence in which the displacement adjustment of each axis is encoded according to a specific rule. This packaged control instruction format is sent to the processing equipment as a positioning information packet. The processing equipment adjusts the movement of each axis based on the information in the packet, thereby achieving accurate processing positioning of the workpiece.

[0097] As a non-limiting embodiment, after generating the positioning information of the workpiece to be processed in the processing coordinate system, it also includes: decomposing the preset processing path into multiple axial motion parameter sequences, scaling and interpolation compensation of each axial motion parameter according to the coordinate system conversion parameters in the positioning information; generating a multi-axis linkage kinematic model of the processing equipment, inputting the adjusted axial motion parameters into the multi-axis linkage kinematic model for reverse solution, and generating an actual displacement pulse sequence of each driving axis; generating a processing control instruction set including speed look-ahead control parameters based on the timing relationship and pulse equivalent of the displacement pulse sequence, and driving the processing axis to execute the processing path according to the optimized acceleration curve.

[0098] The preset machining path is determined based on the design requirements and machining process of the workpiece to be machined. For example, the machining path may include path planning for multiple operations such as drilling, milling, and boring. This machining path is decomposed into multiple axial motion parameter sequences. Taking the LX axis as an example, a series of axial motion parameters such as the starting position, target position, and motion speed are determined based on the movement distance and speed requirements in the LX axis direction of the machining path.

[0099] The coordinate system conversion parameters in the positioning information are used to scale and interpolate the parameters of each axial motion. For example, if the coordinate system conversion parameters indicate a certain displacement deviation of the workpiece in the LX direction, the axial motion parameters of the LX axis are scaled based on this deviation, adjusting the starting and target positions. To ensure smooth motion, interpolation compensation is also performed. For example, linear interpolation or spline interpolation is used to supplement the parameters of intermediate positions between two moving positions, making the motion more continuous.

[0100] Generate a multi-axis linkage kinematic model for the processing equipment. This model describes the relationship between the motion parameters of each axis and the final position and processing effect of the workpiece when multiple axes move simultaneously. For example, in milling processing, the linkage of the LX axis and the LY axis controls the motion trajectory of the milling cutter in the plane, the LZ axis controls the milling depth, and the RA axis controls the rotation angle of the milling cutter. The adjusted axial motion parameters are input into the multi-axis linkage kinematic model for reverse solution. Based on the known workpiece processing requirements and the adjusted axial motion parameters, the actual displacement pulse sequence required by each drive axis is calculated. For example, according to the processing path requirements, the workpiece needs to reach a set position at a certain moment. The number of displacement pulses required by each axis such as LX, LY, and LZ at that moment is determined through reverse solution of the multi-axis linkage kinematic model.

[0101] Based on the timing relationship and pulse equivalent of the displacement pulse sequence, a processing control instruction set containing speed look-ahead control parameters is generated. The pulse equivalent refers to the distance that each pulse drives the processing equipment axis to move. For example, the LX axis pulse equivalent is 0.01mm / pulse. According to the timing of the displacement pulse sequence, the movement speed of each axis at different times is determined. In order to achieve a smooth processing process, speed look-ahead control parameters are added, such as predicting the movement speed change in the next stage in advance, adjusting the acceleration curve, and avoiding the impact of speed mutations on processing quality. This information is integrated to generate a processing control instruction set, which is sent to the processing equipment to drive the processing axis to execute the processing path according to the optimized acceleration curve to ensure processing accuracy and quality.

[0102] As a non-limiting embodiment, after generating the positioning information of the workpiece to be processed in the processing coordinate system, it also includes: triggering the laser scanner to perform online point cloud acquisition on the processed surface after the processing equipment is started, and extracting the radial deviation vector between the actual processing contour and the theoretical model contour; performing a coordinate system rotation transformation on the radial deviation vector according to the rotation matrix in the positioning information, and generating a set of compensation vectors axially aligned with the processing coordinate system; using a Kalman filter to suppress noise on the compensation vector, and decomposing the filtered compensation amount into axial compensation components according to the processing feed direction, and superimposing it into the axial motion control instructions of the subsequent processing path.

[0103] After the machining equipment is started, the laser scanner begins to collect online point cloud data of the machined surface. The laser scanner generates point cloud data by emitting a laser beam and receiving reflected light to obtain distance information from the object surface. The laser scanner scans the workpiece along the machined surface, collecting a large amount of point cloud data that constitutes the actual machining contour.

[0104] Extract the radial deviation vector between the actual machining contour and the theoretical model contour. Compare the collected actual machining contour point cloud data with the theoretical model contour. For each point cloud data point, calculate the distance vector from its nearest point on the theoretical model contour. The radial component of this distance vector can be understood as the radial deviation vector.

[0105] The radial deviation vectors are transformed into a coordinate system rotation according to the rotation matrix in the positioning information. The rotation matrix in the positioning information describes the rotational relationship of the workpiece to be processed in the processing coordinate system. The radial deviation vectors are transformed according to the rotation matrix so that they are aligned with the axis of the processing coordinate system. For example, the direction of the radial deviation vector in the original coordinate system may not be consistent with the axis of the processing coordinate system. Through the transformation of the rotation matrix, the direction is converted to be consistent with the axis of the processing coordinate system, and a set of compensation vectors are generated that are aligned with the axis of the processing coordinate system.

[0106] A Kalman filter is used to suppress noise in the compensation vectors. The Kalman filter is an optimal linear filter that filters noisy signals based on the system's state equations and observation equations, yielding more accurate estimates. For each vector in the compensation vector set, the Kalman filter predicts the current state based on the previous state estimate and the current observation. By calculating parameters such as the covariance matrix, the filter continuously adjusts the estimate to remove the effects of noise.

[0107] The filtered compensation amount is decomposed into axial compensation components according to the machining feed direction and added to the axial motion control instructions of the subsequent machining path. The machining feed direction determines how the compensation amount is distributed in each axial direction. For example, in milling, the machining feed direction is mainly along the LX axis and the LY axis. The filtered compensation amount is decomposed into the LX, LY and other axes according to the machining feed direction to obtain the axial compensation components. These axial compensation components are added to the axial motion control instructions of the subsequent machining path. For example, in the subsequent machining path, the original LX axis motion control instruction requires a moving distance of W, which is now adjusted to W+ΔW according to the compensation component, where ΔW is the compensation amount of the LX axis, thereby correcting the machining path and improving the machining accuracy.

[0108] As a non-limiting embodiment, after generating the positioning information of the workpiece to be processed in the processing coordinate system, it also includes: periodically triggering the multi-view image acquisition device to capture local images of the workpiece at a preset frequency during the processing, and extracting the contour features of the remaining part of the workpiece in the current processing stage; real-time alignment of the contour features with the corresponding areas of the three-dimensional model data, and updating the actual posture data of the workpiece in the processing coordinate system; when it is detected that the posture offset exceeds the set ratio of the processing tool radius, activating the processing path emergency adjustment instruction and regenerating the positioning information including the tool compensation parameters.

[0109] During the machining process, the multi-view image acquisition device is periodically triggered to capture local images of the workpiece at a preset frequency. For example, the preset frequency is every 10 minutes, and the multi-view image acquisition device captures images of the workpiece from different angles. During the milling process, the image acquisition device captures local images of the part surrounding the milled portion of the part. These images contain information about the remaining portion of the workpiece during the current machining phase.

[0110] Extract the contour features of the remaining workpiece at the current processing stage. The captured partial image is processed, and algorithms such as edge detection and contour extraction are used to extract the contours of the remaining workpiece. For example, the Canny edge detection algorithm is used to find edges in the image, and then the contour tracking algorithm is used to extract the complete contour. These contour features can reflect the current shape and size of the workpiece.

[0111] Real-time registration of contour features with corresponding areas in the 3D model data. The 3D model data represents the ideal shape and position of the workpiece. The extracted contour features are matched and aligned with the corresponding areas in the 3D model data. Using some registration algorithms (such as an improved version of the ICP algorithm based on feature point matching, which takes into account partial shape changes of the workpiece during machining), the corresponding relationship between the two is found, the actual position change of the workpiece in the machining coordinate system is calculated, and the actual position data of the workpiece in the machining coordinate system is updated.

[0112] When it is detected that the posture offset exceeds the set ratio of the machining tool radius, the emergency adjustment instruction of the machining path is activated and the positioning information containing the tool compensation parameters is regenerated. For example, if the set ratio is 50%, if it is detected that the posture offset of the workpiece causes the deviation of the tool from the theoretical machining position to exceed 50% of the tool radius, it means that a large deviation has occurred in the machining process and the machining path needs to be adjusted urgently. At this time, the emergency adjustment instruction of the machining path is activated, and the workpiece is re-extracted and feature matched with the reference model. The required tool compensation parameters are calculated, and the positioning information containing the tool compensation parameters is regenerated and sent to the machining equipment. The machining path is adjusted to ensure the accuracy and quality of the machining, avoid further expansion of the machining error, and ensure that the final machining of the workpiece meets the requirements.

[0113] In actual applications, for the image acquisition link, the Zhang Zhengyou calibration method can be used to pre-calibrate the intrinsic parameters (focal length, distortion coefficient) and extrinsic parameters (rotation matrix, spatial position) of the industrial camera, and a synchronous trigger can be used to ensure the timing consistency of multi-camera acquisition.

[0114] In the feature extraction stage, the high and low thresholds of Canny edge detection can be adaptively set based on the image grayscale histogram. The Hough transform uses accumulator peak detection to determine the line segment parameters. The curvature calculation can be achieved by fitting the local surface to a quadratic surface and then taking the derivative.

[0115] Furthermore, bundle adjustment can be used to optimize projection residuals during 3D reconstruction. Point cloud coordinates are iteratively solved using the Levenberg-Marquardt algorithm, with convergence conditions set to a residual change rate of less than 1e-5 or 200 iterations. For surface fitting, DBSCAN clustering is set with a neighborhood radius of ε = 2 mm and a minimum number of points minPts = 15. Local surface types are selected by principal component analysis to determine the eigenvalue distribution.

[0116] Optionally, the RANSAC algorithm is used to eliminate abnormal matching points during feature matching, the maximum and minimum linear mapping is used for curvature normalization, the robot kinematic model is established through the Denavit-Hartenberg parameters for inverse kinematics solution, and the homogeneous coordinate transformation matrix is ​​used for pose inversion verification.

[0117] In an embodiment of the present invention, processing compensation can be combined with adaptive particle filtering to suppress point cloud noise, and path adjustment can achieve smooth transition through B-spline curve interpolation. It can be understood that the parameters of each link are based on the unified dimension of the International System of Units, and the algorithm module is implemented through open source libraries such as OpenCV and PCL.

[0118] The embodiment of the present invention first obtains a multi-view image data set that comprehensively covers different surface areas of the workpiece to be processed, providing rich and complete original materials for subsequent processing; then the multi-view image data set is subjected to 3D reconstruction processing to generate 3D model data that can intuitively and accurately present the three-dimensional shape of the workpiece to be processed; then a spatial feature set is extracted from the 3D model data, which can deeply characterize the spatial characteristics of the workpiece from multiple dimensions, providing a comprehensive and detailed basis for precise positioning; finally, the spatial feature set is feature-matched with a preset workpiece processing reference model, and the generated positioning information can effectively drive the processing equipment to perform workpiece processing path adjustment operations, can dynamically adapt to the actual shape of the workpiece, optimize the processing path, improve the processing accuracy and efficiency of the processing equipment, and ensure high-quality completion of workpiece processing.

[0119] Furthermore, Figure 2 The structural block diagram of the irregular workpiece processing positioning system 300 is shown, which includes: a memory 310 for storing program instructions and data; a processor 320 for coupling with the memory 310 and executing the instructions in the memory 310 to implement the above method.

[0120] Furthermore, a computer storage medium is provided, comprising instructions, which implement the above method when executed on a processor.

[0121] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A method for machining and positioning irregular workpieces based on machine vision, characterized in that: include: Acquire a multi-view image data set of a workpiece to be processed, wherein the multi-view image data set includes image acquisition data covering different areas of a surface of the workpiece to be processed; Performing three-dimensional reconstruction processing on the multi-view image data set to generate three-dimensional model data of the workpiece to be processed; Extracting a spatial feature set of the three-dimensional model data, wherein the spatial feature set includes surface curvature distribution features, contour topological structure features, and key geometric constraint features of the workpiece to be processed; The spatial feature set is subjected to feature matching processing with a preset workpiece processing reference model to generate positioning information of the workpiece to be processed in a processing coordinate system, and the positioning information is used to drive the processing equipment to perform a workpiece processing path adjustment operation.

2. The method according to claim 1, characterized in that The performing three-dimensional reconstruction processing on the multi-view image data set to generate three-dimensional model data of the workpiece to be processed includes: performing noise suppression processing on each image acquisition data in the multi-view image data set to obtain a denoised image data set; Extracting a feature point set of each image acquisition data in the denoised image data set, wherein the feature point set includes edge intersection points, curvature mutation points and texture key points; Determining a three-dimensional coordinate mapping relationship of the feature point set based on a spatial projection relationship between different image acquisition data, and generating initial three-dimensional point cloud data of the workpiece to be processed according to the three-dimensional coordinate mapping relationship; The initial three-dimensional point cloud data is subjected to surface fitting processing to eliminate fitting errors caused by point cloud density differences, thereby generating three-dimensional model data of a continuous surface structure.

3. The method according to claim 2, characterized in that The step of extracting a feature point set of each image acquisition data from the denoised image data set comprises: Invoking an edge detection algorithm to perform gradient calculation on the denoised image data set to determine pixel regions in the image data whose pixel gradient amplitudes exceed a preset gradient threshold; Detecting intersections between edge line segments within the pixel area, and adding the intersections as edge intersections to the feature point set; Performing curvature analysis on the denoised image data set, identifying a target area whose curvature change amplitude exceeds a preset curvature change threshold, and extracting a center point of the target area as a curvature mutation point to be added to the feature point set; A texture similarity analysis is performed on the denoised image data set to determine the connection points between texture repeating units and add them to the feature point set as texture key points.

4. The method according to claim 2, characterized in that The determining of the three-dimensional coordinate mapping relationship of the feature point set based on the spatial projection relationship between different image acquisition data includes: Obtaining a camera parameter set calibrated by each image acquisition device in a multi-view image data set, wherein the camera parameter set includes a focal length parameter, an imaging plane rotation matrix, and a camera space position coordinate; Performing cross-view matching on each feature point in the feature point set, and extracting a set of two-dimensional pixel coordinates corresponding to the feature point in different image acquisition data, where each two-dimensional pixel coordinate includes horizontal and vertical coordinate values ​​in pixels; Convert each two-dimensional pixel coordinate into a normalized plane coordinate according to a focal length parameter of a corresponding camera in the camera parameter set to eliminate dimensional mismatch caused by size differences of image sensors; Based on the imaging plane rotation matrix and camera space position coordinates of each camera, a projection ray model of each feature point is generated, wherein the projection ray model is expressed by a spatial line parameter model determined by the camera optical center position coordinates and the normalized plane coordinates; Calculate the spatial intersection of multiple projection rays corresponding to the same feature point, and take the 3D coordinate point with the minimum sum of the squares of the Euclidean distances from each projection ray to the spatial intersection as the initial 3D coordinate of the feature point; Iteratively optimize feature points whose projection residuals exceed a preset tolerance threshold, construct a regularization term based on the spatial continuity constraint of the three-dimensional coordinates of adjacent feature points, and use a nonlinear least squares method to adjust the initial three-dimensional coordinates to minimize the weighted sum of the projection residual and the regularization term; The adjusted three-dimensional coordinates of all feature points are used to generate a triangular facet topological connection structure according to a spatial adjacency relationship, and a three-dimensional coordinate mapping relationship that matches the number dimension of the image acquisition viewing angle is determined based on the triangular facet topological connection structure.

5. The method according to claim 2, characterized in that The performing surface fitting processing on the initial three-dimensional point cloud data to eliminate fitting errors caused by point cloud density differences and generate three-dimensional model data of a continuous surface structure includes: Dividing the initial three-dimensional point cloud data into a plurality of sub-point cloud regions with uniform density, wherein the difference in point cloud density in each sub-point cloud region is less than a preset density threshold; Performing local surface model fitting on each sub-point cloud region to generate a plurality of local surface patches, wherein the local surface patches include at least one of a plane, a cylindrical surface, and a spherical surface; According to the geometric continuity constraints between the local surface patches of adjacent sub-point cloud regions, the boundary parameters of the local surface patches are adjusted to generate a globally continuous target surface structure; The target surface structure is converted into closed three-dimensional model data.

6. The method according to claim 1, characterized in that The extracting of the spatial feature set of the three-dimensional model data includes: Meshing the surface of the three-dimensional model data to generate a set of evenly distributed mesh nodes; Calculating the curvature value at each grid node, and generating surface curvature distribution characteristics according to the change trend of the curvature value between adjacent grid nodes; Extracting the contour edges of the three-dimensional model data, and generating contour topological structure features according to the connection relationship and branch structure of the contour edges; generating key geometric constraint features based on a constraint area in the three-dimensional model data that satisfies a preset geometric constraint condition, wherein the geometric constraint condition includes at least one of parallelism, perpendicularity, and coaxiality; The step of performing feature matching processing on the spatial feature set and a preset workpiece processing reference model to generate positioning information of the workpiece to be processed in a processing coordinate system includes: Acquire a reference space feature set of the workpiece machining reference model, wherein the reference space feature set includes a reference surface curvature distribution, a reference profile topology structure, and a reference geometric constraint; performing a curvature similarity comparison between the surface curvature distribution feature of the workpiece to be processed and the reference surface curvature distribution to determine a first matching degree; Performing a topological isomorphism analysis on the contour topological structure features of the workpiece to be processed and the reference contour topological structure to determine a second matching degree; Performing constraint compliance verification on the key geometric constraint features of the workpiece to be processed and the reference geometric constraint to determine a third matching degree; According to a weighted result of the first matching degree, the second matching degree, and the third matching degree, a posture deviation of the workpiece to be processed relative to the workpiece processing reference model is calculated to generate the positioning information.

7. The method according to claim 6, characterized in that The comparing the surface curvature distribution characteristics of the workpiece to be processed with the reference surface curvature distribution for curvature similarity to determine a first matching degree includes: Performing grid alignment processing on the surface curvature distribution feature and the reference surface curvature distribution to obtain a plurality of grid node pairs, each grid node pair comprising a grid node corresponding to the surface curvature distribution feature and a grid node corresponding to the reference surface curvature distribution, and the spatial positions of the two grid nodes in each grid node pair corresponding one to one; Calculate the difference in curvature values ​​between the two grid nodes in each grid node binary, and count the proportion of grid node binary groups whose curvature value difference is less than the preset curvature tolerance threshold as the initial matching degree; The initial matching degree is corrected according to the directional consistency of the curvature change gradient in the surface curvature distribution feature to generate a first matching degree.

8. The method according to claim 6, characterized in that The step of calculating the position deviation of the workpiece to be processed relative to the workpiece processing reference model based on a weighted result of the first matching degree, the second matching degree, and the third matching degree to generate the positioning information includes: Performing a weighted summation of the first matching degree, the second matching degree, and the third matching degree according to a preset weight distribution coefficient to generate a global matching degree, wherein the weight distribution coefficient is adjusted according to the influencing factors of the surface curvature distribution characteristics, the contour topological structure characteristics, and the key geometric constraint characteristics in the processing positioning; When the global matching degree exceeds a preset matching degree threshold, extracting a set of matching feature point pairs corresponding to the surface curvature distribution features, the contour topological structure features, and the key geometric constraint features, wherein the set of matching feature point pairs includes the three-dimensional coordinates of the feature points of the workpiece to be processed and the three-dimensional coordinates of the corresponding reference feature points in the workpiece processing reference model; Solving coordinate transformation parameters for the set of matching feature point pairs, and calculating a rotation matrix and a translation vector of the workpiece to be processed relative to the workpiece processing reference model based on a least squares spatial alignment error between the three-dimensional coordinates of the feature points and the three-dimensional coordinates of the reference feature points; Determining a position deviation of the workpiece to be processed in the processing coordinate system based on the rotation matrix and the translation vector, wherein the position deviation includes offsets in six degrees of freedom directions; When the global matching degree does not exceed the preset matching degree threshold, adjusting the weight distribution coefficient and re-performing the weighted summation operation until the global matching degree meets the preset matching degree threshold or reaches a maximum number of iterations; generating the positioning information including coordinate system conversion parameters according to the posture deviation, wherein the coordinate system conversion parameters are used to map the current posture of the workpiece to be processed to the target posture of the workpiece processing reference model; The method further comprises: Performing an inverse kinematic solution on the coordinate system transformation parameters to generate a displacement adjustment for driving the motion of each axis of the processing equipment, and performing a forward kinematic solution verification on the displacement adjustment based on the mechanical structure parameters of the processing equipment to constrain the displacement adjustment within the travel range of the processing equipment; The verified displacement adjustment amount is encapsulated into a control instruction format to be recognized by the processing equipment to generate a data packet of the positioning information.

9. A positioning system for processing irregular workpieces, characterized in that: include: Memory, used to store program instructions and data; A processor, coupled to a memory, and configured to execute instructions in the memory to implement the method according to any one of claims 1 to 8.

10. A computer storage medium, characterized in that The method comprises instructions which, when executed on a processor, implement the method according to any one of claims 1 to 8.

Images