Deburring method combining 2D image and 3D point cloud

Abstract

The invention discloses a deburring method combining a 2D image and a 3D point cloud, and relates to the technical field of image processing, and the method comprises the steps: carrying out the point cloud registration under the verification of the surface orientation consistency of a standard point cloud and an actual point cloud; constructing a pixel mapping matrix between the workpiece image and the RGB image; semantic-level edge coarse extraction is carried out on the workpiece image, and extraction of a sub-pixel-level actual edge contour is carried out on the basis of coarse extraction in combination with a traditional edge detection algorithm; aligning the actual edge contour to the actual point cloud through the pixel mapping matrix; and the target polishing surfaces are aligned to the actual point cloud, to-be-polished edge contours are screened based on the distances between the contours of all the target polishing surfaces in the actual point cloud and all the actual edge contours, and a polishing track is obtained in the workpiece image. According to the method, the sub-pixel edge positioning capability of the 2D image and the spatial topology information of the 3D point cloud are fused, and the inherent defect of a single mode is overcome.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a deburring method combining 2D images and 3D point clouds. Background Art

[0002] In the industrial manufacturing field, burrs, as excess residues generated during the cutting, stamping, or casting of workpieces, can significantly reduce the assembly accuracy and service life of products. Traditional manual deburring methods are not only inefficient and inconsistent, but also difficult to meet the dual requirements of precision and efficiency in modern production lines. In recent years, automated deburring technology based on machine vision has gradually become a mainstream solution, but existing technologies still have significant drawbacks: On the one hand, while systems relying on single 3D point cloud processing can capture the workpiece's three-dimensional topography, they are susceptible to point cloud sparsity and noise interference in edge areas. Missing points in the point cloud reduce registration accuracy, while outliers can lead to false or missed burr detections. For example, on thin-walled parts or workpieces with complex curves, occlusions in structured light cameras can cause point cloud fragmentation, distorting edge contour reconstruction and ultimately causing the robot arm's grinding path to deviate or even run empty.

[0003] On the other hand, while traditional 2D vision-based solutions can achieve precise edge positioning through high-resolution cameras (micron-level pixel size) and sub-pixel algorithms, they lack depth information and cannot distinguish between real burrs and texture interference. When oil, scratches, or reflections are present on the workpiece surface, gradient calculations can be misleading, misjudging non-burr structures as targets to be polished. Furthermore, such systems typically require customized detection algorithms for each part, preventing users from flexibly defining differentiated process parameters (such as local polishing force and path density) directly on the 3D digital model. This results in inefficient production line switching and makes it difficult to adapt to the flexible manufacturing needs of high-variety, small-batch production.

[0004] A more fundamental contradiction lies in the fact that existing technologies have yet to effectively integrate the complementary advantages of 2D and 3D data. The spatial topological information of 3D point clouds and the sub-pixel edge accuracy of 2D images are disconnected, and multi-sensor data is simply superimposed rather than collaboratively optimized. This disconnect makes it impossible for the system to overcome the inherent flaws of point clouds in edge areas, while also making it difficult to precisely locate burrs using the positioning capabilities of 2D vision. Furthermore, the rigid binding of process parameters to machining paths further limits the system's adaptability, forcing manual intervention to correct trajectories when faced with complex, irregularly shaped workpieces. Summary of the Invention

[0005] In order to effectively integrate 2D and 3D data to achieve complementary advantages and reduce trajectory correction requiring manual intervention, the present invention proposes a deburring method combining 2D images and 3D point clouds, comprising the following steps: S1: Extracts and visualizes the pre-recorded standard point cloud of the workpiece, and stores the configuration information and target polishing surface adjusted by the user based on the standard point cloud; S2: The actual point cloud and corresponding RGB image of the workpiece are acquired through a 3D structured light camera, and the standard point cloud and the actual point cloud are registered under the surface orientation consistency verification; S3: Acquire the workpiece image through the 2D area array camera and construct a pixel mapping matrix between the workpiece image and the RGB image; S4: Based on the DexiNed edge detection network, the workpiece image is subjected to semantic-level coarse edge extraction. In combination with the traditional edge detection algorithm, the actual edge contour is extracted at the sub-pixel level based on the coarse extraction. S5: Align the actual edge contour extracted from the workpiece image to the RGB image through the pixel mapping matrix, and then align it to the actual point cloud based on the camera parameters of the 3D structured light camera; S6: Align the target polishing surface to the actual point cloud based on the point cloud registration result, and screen the edge contours to be polished using the contour distance metric for the contours of each target polishing surface in the actual point cloud and all actual edge contours; S7: Mapping the screening results and configuration information back to the workpiece image through an inverse process, and obtaining a grinding trajectory based on the configuration information adjustment for the target grinding surface corresponding to the edge contour to be ground in the workpiece image.

[0006] The present invention uses multimodal data collaborative processing to fuse the sub-pixel edge positioning capability of 2D images with the spatial topology information of 3D point clouds, projecting 2D contours into the 3D point cloud coordinate system, effectively overcoming the inherent defects of traditional single-modality technology. That is, the contour distortion problem caused by the sparsity of 3D point clouds in the edge areas is compensated by 2D sub-pixel precision, and the weakness of 2D images that are susceptible to texture interference is corrected through 3D spatial relationships, so that the solution can effectively reduce missed detection and false detection rates.

[0007] Furthermore, in step S2, the point cloud registration between the standard point cloud and the actual point cloud under surface orientation consistency verification specifically includes: The KFPCS coarse registration algorithm is used to perform the initial pose transformation of the actual point cloud relative to the standard point cloud. Based on the initial pose transformation, the PICP fine registration algorithm is used to perform sub-millimeter level point cloud registration, and the normal vector consistency verification is introduced to avoid the flipping situation after registration.

[0008] Furthermore, in the step S3, constructing a pixel mapping matrix between the workpiece image and the RGB image specifically includes: The SIFT feature matching algorithm is used to extract the feature sets of the RGB image and the workpiece image respectively, and the feature sets between the RGB image and the workpiece image are matched through the k-nearest neighbor cross validation of BFMather to construct the pixel mapping matrix.

[0009] Furthermore, in step S4, the extraction of actual edge contour at sub-pixel level based on coarse extraction in combination with a traditional edge detection algorithm specifically includes: The traditional edge detection algorithm performs discrete noise elimination and non-maximum suppression on the edge probability map obtained by coarse extraction, and then repairs the broken edge points connected to each other for strong edge points with weak edge points. The sub-pixel level actual edge contour is extracted by fitting a quadratic surface model with the repaired edge points as the center.

[0010] Furthermore, the traditional edge detection algorithm is the Canny algorithm.

[0011] Furthermore, in the step S5, the camera parameters based on the 3D structured light camera are aligned to the actual point cloud, specifically including: After the actual edge contour is aligned to the RGB image, the pixel coordinates of the actual edge contour in the RGB image coordinate system are converted into normalized camera coordinate system coordinates based on the camera intrinsic parameter matrix and distortion coefficient in the camera parameters. Combined with the pose relationship between the 3D structured light camera and the world coordinate system defined by the camera extrinsic parameter matrix in the camera parameters, the normalized coordinates are mapped to the three-dimensional space coordinates of the actual point cloud through the perspective projection model to complete the alignment of the actual edge contour to the actual point cloud.

[0012] Furthermore, in the step S6, the contour screening of the edge to be polished under the contour distance metric is specifically as follows: The Hausdorff distance between the contour of each target polishing surface in the actual point cloud and all actual edge contours is calculated, and the actual edge contours with a Hausdorff distance less than a preset threshold are selected as the edge contours to be polished.

[0013] Furthermore, in the step S7, the polishing trajectory acquisition based on the configuration information adjustment is specifically: based on the configuration information, corresponding morphological changes, smoothing and sampling operations are performed on the target polishing surface, and the polishing trajectory is acquired under burr interference removal.

[0014] Furthermore, in step S7, the grinding trajectory adjusted based on the configuration information is surrounded by a number of grinding segments configured with corresponding configuration information to form the edge contour to be ground, and the transition segments between adjacent grinding segments are generated by the interpolation algorithm to completely surround the contour of the target grinding surface.

[0015] Compared with the prior art, the present invention has at least the following beneficial effects: (1) The present invention proposes a deburring method that combines 2D images and 3D point clouds. Through multimodal data collaborative processing, the sub-pixel edge positioning capability of 2D images is integrated with the spatial topology information of 3D point clouds, so that the 2D contour is projected into the 3D point cloud coordinate system. This effectively overcomes the inherent defects of traditional single-modality technology. That is, the contour distortion problem caused by the sparsity of 3D point clouds in the edge area is compensated by 2D sub-pixel accuracy, and the weakness of 2D images that are easily disturbed by texture is corrected by 3D spatial relationships, thereby reducing the missed detection and false detection rates. (2) The point cloud registration process adopts the KFPCS and PICP dual-stage optimization strategy combined with the normal vector consistency verification mechanism, which completely solves the registration flip risk caused by initial pose deviation or noise in industrial scenes; (3) At the edge extraction level, the DexiNed network prioritizes identifying potential edge areas under texture interference and outputs an edge probability map. Traditional algorithms focus on high-response areas to perform Gaussian noise reduction, non-maximum suppression, and dual-threshold connection to repair contour breaks caused by surface pores, and ultimately achieve sub-pixel positioning through quadratic surface fitting. This design significantly improves the weak edge detection rate while reducing edge jitter, ensuring the integrity and geometric accuracy of the burr contour. (4) Supports user-driven flexible process configuration. Users can directly define differentiated grinding areas and parameter libraries (such as high pressure and low speed in sharp angle areas) on the standard point cloud. The target grinding surface information is screened by edge similarity matching and mapped to the 2D execution coordinate system synchronously with the process parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A step-by-step diagram of a deburring method combining 2D images and 3D point clouds. DETAILED DESCRIPTION

[0017] The following are specific embodiments of the present invention and the accompanying drawings to further describe the technical solutions of the present invention, but the present invention is not limited to these embodiments.

[0018] In the field of industrial automation deburring, traditional single-modality vision systems have long faced the dual dilemma of contour distortion caused by the sparsity of the edges of three-dimensional point clouds and misjudgment caused by the interference of two-dimensional image textures. Especially for castings with complex structures such as engine cylinders and gearbox housings, existing technologies are difficult to achieve stable detection of sub-millimeter burrs under strong reflections, oil stains and air holes, and are even unable to support users to flexibly define process parameters according to differentiated processing requirements. In order to break through this technical bottleneck, the present invention proposes an innovative method that integrates multimodal perception and intelligent decision-making: by collaboratively utilizing the spatial topology reconstruction capabilities of 3D structured light cameras and the sub-pixel edge resolution advantages of 2D area array cameras, a full closed-loop processing link of "point cloud registration-pixel mapping-edge fusion-process configuration" is constructed. As Figure 1 As shown, the present invention proposes a deburring method combining 2D images and 3D point clouds, comprising the steps of: S1: Extracts and visualizes the pre-recorded standard point cloud of the workpiece, and stores the configuration information and target polishing surface adjusted by the user based on the standard point cloud; S2: The actual point cloud and corresponding RGB image of the workpiece are acquired through a 3D structured light camera, and the standard point cloud and the actual point cloud are registered under the surface orientation consistency verification; S3: Acquire the workpiece image through the 2D area array camera and construct a pixel mapping matrix between the workpiece image and the RGB image; S4: Based on the DexiNed edge detection network, the workpiece image is subjected to semantic-level coarse edge extraction. In combination with the traditional edge detection algorithm, the actual edge contour is extracted at the sub-pixel level based on the coarse extraction. S5: Align the actual edge contour extracted from the workpiece image to the RGB image through the pixel mapping matrix, and then align it to the actual point cloud based on the camera parameters of the 3D structured light camera; S6: Align the target polishing surface to the actual point cloud based on the point cloud registration result, and screen the edge contours to be polished using the contour distance metric for the contours of each target polishing surface in the actual point cloud and all actual edge contours; S7: Mapping the screening results and configuration information back to the workpiece image through an inverse process, and obtaining a grinding trajectory based on the configuration information adjustment for the target grinding surface corresponding to the edge contour to be ground in the workpiece image.

[0019] During the deburring system's initialization phase, the system first extracts a standard point cloud model of the workpiece from a pre-existing CAD design library (typically converted from a 3D digital model generated by industrial software such as NX and CATIA). This model is then rendered using an Open3D or PCL point cloud processing engine, dynamically displaying the workpiece's 3D topology within an interactive interface. Within this visual interface, users can perform in-depth process configuration: selecting specific edge areas (such as the engine block parting line and transmission case oil passage openings) as target grinding surfaces, and defining a library of process parameters for each surface—including, but not limited to, key-value pairs such as grinding force (adjustable from 200-500N), tool speed (gradient settings from 8,000 to 20,000 rpm), and feed rate (0.1-0.5 m / s). For complex cavity structures, the system supports drawing topological constraints for machining paths directly on the 3D digital model, for example, requiring tangential continuity around corners or avoiding interference with locating pins. All configuration information (geometric boundary definition + process parameters + path constraints) is stored in a structured data format in a local database. The process parameter library is automatically linked to the material property library (such as the hardness coefficient of cast iron and aluminum alloy), providing a decision basis for subsequent adaptive processing.

[0020] In automated deburring systems, the robotic arm's grinding trajectory must strictly adhere to the theoretical design model of the workpiece (a standard point cloud generated by CAD). However, the actual workpiece's 3D topography can differ from the theoretical model due to factors such as clamping deviations and machining deformation. Directly applying the preset trajectory to the actual workpiece can lead to two serious problems due to spatial mismatch: First, trajectory deviation can lead to missed burrs (such as residual burrs on the edge of a cylinder's mold line); second, interference between the tool and the workpiece can cause collisions (such as puncturing a thin-walled part due to trajectory deviation).

[0021] To eliminate this hidden danger, the present invention uses a high-resolution 3D structured light camera (resolution ≥ 0.05mm) to scan the actual workpiece, synchronously obtains the actual point cloud model containing three-dimensional coordinate information and its corresponding RGB texture image, and performs point cloud registration to transform the standard point cloud into the actual point cloud coordinate system, so that all subsequent processing (such as target surface screening and trajectory mapping) are based on a unified spatial reference, and dynamically compensates for the dimensional deformation of the actual workpiece during processing and clamping (such as ±2mm deformation caused by thermal expansion and contraction) through the registration process, to avoid the failure of the preset process parameters.

[0022] For point cloud registration, the system first uses the KFPCS coarse registration algorithm to establish the initial transformation, which includes the following steps: (1) Based on the ISS algorithm, each coordinate point in the standard point cloud Calculate its neighborhood covariance matrix , perform key point extraction, the formula is expressed as: , in, is the coordinate point The neighborhood point set of Neighborhood point set Random coordinate points in are the coordinates of the neighborhood centroid, and T is the matrix transpose.

[0023] (2) Retain the key points that meet the eigenvalue ratio threshold. The formula is expressed as: , in, is the covariance matrix The eigenvalues of is the threshold parameter for suppressing interference in Pingtan area.

[0024] (3) Perform similarity measurement of four-point basis set (4PCS) by selecting a set of coplanar points , not all collinear points, calculate affine invariant ratio , the formula is: , in, For a straight line With straight line The intersection of .

[0025] (4) Matching basis sets , decomposed by SVD to find : Obtain the three-dimensional coordinates of the key points of the actual point cloud ( , is the i-th coordinate point in the actual point cloud), used to compare with the standard point cloud centroid coordinates in the SVD solution Calculate the translation vector together (N is the number of key points matched). Then the covariance matrix of the key point set is Perform singular value decomposition (SVD), , in, is an orthogonal matrix, Is a diagonal matrix with singular values. Solve the rotation matrix and translation vectors , , Final output initial transformation matrix , and preliminarily compress the posture deviation.

[0026] The PICP fine registration algorithm is then used to calibrate the initial transformation matrix to sub-millimeter level, including the following steps: (1) Solve the coordinate points The vector corresponding to the minimum eigenvalue of the neighborhood covariance matrix , , By traversing all possible directions, find the quadratic form (Neighborhood points are on the unit vector The unit vector with the smallest projection variance in the direction .

[0027] (2) The optimization goal is to minimize the distance from the coordinate point to the tangent plane. , in, The coordinate points in the actual point cloud and the standard point cloud The corresponding coordinate points, K is the number of coordinate point pairs that successfully establish corresponding relationships (the number of matching coordinate point pairs between the standard point cloud and the actual point cloud).

[0028] On the basis of the above-mentioned coarse registration + fine registration, the present invention further performs anti-flip verification by calculating the isotropy of the normal vector.

[0029] After registering the actual point cloud with the standard point cloud in 3D data, the present invention then uses a 2D area scan camera to capture an image of the actual workpiece. This image is then preprocessed using adaptive Gaussian filtering to suppress ambient light noise, and Otsu threshold segmentation combined with connected component analysis to extract the main workpiece area and eliminate background interference from the conveyor belt. SIFT (Scale-Invariant Feature Transform) is then used to perform feature matching between the workpiece image and the RGB image simultaneously acquired by the 3D structured light camera.

[0030] Specifically, SIFT constructs a Gaussian pyramid of images to detect local extreme points at different scales. These points are typically key points in the image, such as edges and corners. It then precisely locates the position and scale of the feature points in scale space, excluding points with low contrast and strong edge responses to ensure the stability of the key points. Finally, the orientation of each key point is calculated to ensure rotational invariance. A 16×16 window is generated around the key point, divided into 4×4 blocks, and the orientation histogram of each block is calculated to form a 128-dimensional feature vector.

[0031] In this way, we only need to use SIFT algorithm to extract key points and feature descriptors for RGB image and workpiece image respectively, and use the similarity between feature descriptors to perform matching. Here, the present invention uses BFMatcher to perform bidirectional k-nearest neighbor cross matching and applies ratio test to filter out unstable matches. Specifically, for each feature descriptor of the workpiece image, , in the RGB image feature description subset Search for the closest European distance and the next closest , when the distance is Less than the nearest neighbor / next nearest neighbor distance ratio threshold Candidate matches are retained when ; further reverse verification is performed only when Nearest neighbor reverse matching in RGB images When For a valid match. Finally, based on the successful matching Point pair Solving the homography matrix , realize the coordinate transformation with low pixel error between two image coordinate systems The formula is as follows: Assume the workpiece image feature set: , is the total number of feature point sets of the workpiece image after detection by SIFT algorithm, is the spatial coordinate of the i-th feature point in the workpiece image, Assume the RGB image feature set: , is the total number of feature point sets of the RGB image after detection by the SIFT algorithm, is the spatial coordinate of the jth feature point in the RGB image, Perform a k-nearest neighbor search: , Minimum number of neighbors to search for to satisfy the distance ratio test.

[0032] Distance ratio test: , Two-way cross validation: forward matching , reverse match , the final matching set .

[0033] In addition, the present invention also improves the reliability of matching by adopting the RANSAC algorithm to eliminate erroneous matching pairs, and realizes the acquisition of a pixel mapping matrix with low pixel error between the two image coordinate systems.

[0034] While constructing the pixel mapping matrix, the system also uses the pretrained DexiNed edge detection network to perform semantic-level coarse edge extraction on the preprocessed workpiece image. This network utilizes a multi-level encoder-decoder architecture, fine-tuned based on ImageNet pretrained weights, and maintains a fixed input image resolution of 1024×768 pixels. The network output is an edge probability map, where each pixel value represents the probability of an edge at that location (in the range of 0-1). This stage leverages the feature fusion capabilities of deep learning to identify weak edge responses in complex textures. For example, it can maintain edge continuity even in areas of oil stains or reflective interference on the casting surface, thus resolving the edge breakage problem caused by texture interference in traditional algorithms.

[0035] After completing the coarse extraction of semantic-level edges, the system performs refined processing using traditional edge detection algorithms. First, the edge probability map undergoes discrete noise elimination: a morphological erosion operation is used to remove isolated noise points (iterated once using a 3×3 square kernel), followed by a dilation operation to restore the morphological integrity of the valid edges (iterated twice using the same kernel). Non-maximum suppression is then performed, retaining local maxima along the edge gradient and suppressing non-peak responses. For edge fractures that persist after this processing (common in workpiece porosity), the system initiates an edge repair mechanism: dual thresholds for strong and weak edges are set (the strong edge threshold is 70% of the maximum gradient, and the weak edge threshold is 30%). A search is performed for weak edge points within a 3×3 neighborhood around the strong edge point. If the gradient directions of the two points align, a connecting channel is established, effectively repairing contour interruptions caused by the loss of local features.

[0036] After obtaining a complete set of connected edge points, the system performs sub-pixel edge localization. A 7×7 pixel neighborhood window is established around each edge point, and a quadratic surface model is fitted using the least squares method. This model determines sub-pixel coordinates by finding the extreme points where the partial derivatives of the surface equations are zero. The model calculates the second-order derivative characteristics of the pixel gradient distribution within the window and accurately infers the edge position based on the maximum point of the quadratic surface curvature change, thereby extracting the actual edge contour at the sub-pixel level.

[0037] After completing sub-pixel edge contour extraction from the workpiece image, the system initiates the cross-modal spatial alignment process. First, based on the pixel mapping matrix constructed in step S3, the actual edge contour points in the workpiece image coordinate system are mapped to the RGB image coordinate system. Once the contours are aligned to the RGB image, the system loads the camera parameters of the 3D structured light camera to perform 3D spatial mapping. The key operation consists of two core transformation stages: (1) Distortion correction and normalization conversion: Based on the camera intrinsic parameter matrix (focal length and Optical Center ) and radial-tangential distortion coefficients, and perform distortion correction on the contour points in the RGB image. Specifically, the distortion model is solved by an iterative approximation algorithm: Assume that the initial coordinates of the pixel point are , calculate its normalized offset relative to the optical center, introduce distortion polynomial to adjust radial offset and tangential offset, and obtain the corrected coordinates after 3 iterations The corrected coordinates are then converted to normalized camera coordinates using an inverse perspective transformation. : .

[0038] (2) 3D space mapping: Combined with the camera external parameter matrix (including the rotation matrix and translation vectors ) defines the position relationship between the camera and the world coordinate system, and recovers the depth information through the perspective projection model. Read the depth value corresponding to the RGB image pixel in the actual point cloud , the normalized coordinates Convert to camera coordinate system 3D coordinates : , Finally, the camera coordinate system coordinates are converted to the world coordinate system coordinates through the external parameter matrix : .

[0039] This process achieves millimeter-level precision alignment of sub-pixel edge contours to the actual point cloud.

[0040] Then, based on the point cloud registration results obtained in step S2, the target polished surface is aligned to the actual point cloud, and the system initiates the edge similarity matching process. The core processing first performs a parametric reconstruction of the boundary contour of each target polished surface: the plane equation is fitted using the normal vector constrained RANSAC algorithm, the boundary point set is extracted, and a B-spline curve is applied to fit it into a closed contour to form the target contour curve. At this point, the actual edge contour converted from step S5 to the actual point cloud exists in the form of a discrete point set. At this point, since the contour of the target polished surface (hereinafter referred to as the target contour curve) and the actual edge contour are both in the actual point cloud, the Hausdorff distance can be used to match the two.

[0041] First, calculate the target contour curve To the actual edge contour point set Directed Hausdorff distance of ,Right now Each point to The maximum value of the closest distance; synchronous calculation of the reverse distance The maximum value of the two is defined as the bidirectional Hausdorff distance, which physically represents the distance deviation between the most dissimilar parts of the two contours. To improve computational efficiency, the system establishes a three-dimensional KD-Tree to accelerate the nearest neighbor search.

[0042] Then, based on the preset distance threshold (Dynamically adjusted according to the workpiece processing tolerance), select Hausdorff distance ≤ The actual edge profile is used as the edge profile to be polished.

[0043] After completing the screening of the edge contours to be polished, the system initiates the polishing trajectory generation process. First, configuration-driven morphological adjustments are performed: Based on the user's preset configuration information (e.g., a polishing amount of 0.2-0.5 mm), the contour of the target polished surface in the workpiece image is adaptively deformed. The core mechanism is to establish a functional relationship between the structure element radius and the polishing amount (structure element radius r = 1.5 polishing amount + 0.3 mm), using an elliptical structure kernel for image expansion. The physical equivalent of this operation in two-dimensional image space is to simulate the material removal range under the action of the tool envelope surface, ensuring that the projection area of the polishing trajectory on the workpiece surface covers all burr areas.

[0044] Then smoothing optimization is performed: non-uniform B-spline curve fitting is applied to the expanded contour point set. The curve controls the point spacing. Feed rate set by the user (mm / s) and tool speed (rpm) is determined dynamically, and the specific relationship is (The proportional coefficient The range of 0.6-0.9 is adjusted based on the tool diameter. By solving the chord-length parameterization matrix for the coordinates of the curve's control vertices, a C²-continuous space curve is generated using least-squares fitting constraints. This curve automatically inserts additional control points at points where the curvature changes (exceeding a set threshold of 150 m⁻¹) to smoothly transition the local curvature to a safe range.

[0045] In the sampling stage, based on the principle of equal arc length segmentation (arc length segment number , is the total length of the contour) to discretize the fitting curve. Each sampling point carries three process attributes: spatial posture (calculated by curve differential geometry tangent vector , normal vector ), machining parameters (mapping the force and speed configured by the user) and error compensation (based on the curvature difference between adjacent points). Calculating the speed adjustment factor ).

[0046] Finally, because the contour of a polished surface can be divided into multiple segments with different parameter configurations, which can result in a break after processing, the present invention uses an interpolation algorithm to generate transition segments at the disconnected points between adjacent contours, connecting the adjacent polished segments. For example, a square surface consists of four edges forming a complete closed contour. Because the user can configure different parameters for each of the four edges, the four edges cannot be connected after processing. However, by using an interpolation algorithm to form transition segments, the edges can be connected and the complete contour of the surface can be reconstructed.

[0047] In summary, the present invention proposes a deburring method that combines 2D images and 3D point clouds. Through collaborative multimodal data processing, the sub-pixel edge positioning capability of 2D images is integrated with the spatial topology information of 3D point clouds, so that the 2D contour is projected into the 3D point cloud coordinate system, effectively overcoming the inherent defects of traditional single-modality technology. That is, the contour distortion problem caused by the sparsity of 3D point clouds in the edge area is compensated by 2D sub-pixel accuracy, and the weakness of 2D images that are susceptible to texture interference is corrected through 3D spatial relationships, thereby reducing missed detection and false detection rates.

[0048] The point cloud registration process utilizes a dual-stage optimization strategy of KFPCS and PICP, combined with a normal vector consistency verification mechanism, to completely eliminate the risk of registration flipping caused by initial pose deviation or noise in industrial scenarios. Furthermore, at the edge extraction level, the DexiNed network prioritizes identifying potential edge regions despite texture interference and outputs an edge probability map. Traditional algorithms focus on high-response areas, performing Gaussian noise reduction, non-maximum suppression, and dual-threshold connection to repair contour breaks caused by surface porosity, ultimately achieving sub-pixel positioning through quadratic surface fitting. This design significantly improves the detection rate of weak edges while minimizing edge jitter, ensuring the integrity and geometric accuracy of burr contours.

[0049] The present invention can also support user-driven flexible process configuration. Users can directly define differentiated polishing areas and parameter libraries (such as high pressure and low speed in acute-angle areas) on the standard point cloud. The target polishing surface information is screened by edge similarity matching and then mapped to the 2D execution coordinate system synchronously with the process parameters.

[0050] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0051] In addition, in the present invention, descriptions such as "first," "second," and "one" are for descriptive purposes only and should not be understood to indicate or imply their relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include at least one of such features. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0052] In the present invention, unless otherwise specified or limited, the terms "connection" and "fixation" should be understood in a broad sense. For example, "fixation" can mean fixed connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; internal communication between two elements or interaction between two elements, unless otherwise specified. Those skilled in the art will be able to understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0053] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in this field can implement it. When the combination of technical solutions is mutually contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

Claims

1. A deburring method combining 2D images and 3D point clouds, characterized in that: Including steps: S1: Extracts and visualizes the pre-recorded standard point cloud of the workpiece, and stores the configuration information and target polishing surface adjusted by the user based on the standard point cloud; S2: The actual point cloud and corresponding RGB image of the workpiece are acquired through a 3D structured light camera, and the standard point cloud and the actual point cloud are registered under the surface orientation consistency verification; S3: Acquire the workpiece image through the 2D area array camera and construct a pixel mapping matrix between the workpiece image and the RGB image; S4: Based on the DexiNed edge detection network, the workpiece image is subjected to semantic-level coarse edge extraction. In combination with the traditional edge detection algorithm, the actual edge contour is extracted at the sub-pixel level based on the coarse extraction. S5: Align the actual edge contour extracted from the workpiece image to the RGB image through the pixel mapping matrix, and then align it to the actual point cloud based on the camera parameters of the 3D structured light camera; S6: Align the target polishing surface to the actual point cloud based on the point cloud registration result, and screen the edge contours to be polished using the contour distance metric between the contours of each target polishing surface in the actual point cloud and all actual edge contours; S7: Mapping the screening results and configuration information back to the workpiece image through an inverse process, and obtaining a grinding trajectory based on the configuration information adjustment for the target grinding surface corresponding to the edge contour to be ground in the workpiece image.

2. The deburring method combining 2D images and 3D point clouds according to claim 1, wherein: In the step S2, the point cloud registration between the standard point cloud and the actual point cloud under surface orientation consistency verification specifically includes: The KFPCS coarse registration algorithm is used to perform the initial pose transformation of the actual point cloud relative to the standard point cloud. Based on the initial pose transformation, the PICP fine registration algorithm is used to perform sub-millimeter level point cloud registration, and the normal vector consistency verification is introduced to avoid the flipping situation after registration.

3. The deburring method combining 2D images and 3D point clouds according to claim 1, wherein: In the step S3, constructing a pixel mapping matrix between the workpiece image and the RGB image specifically includes: The SIFT feature matching algorithm is used to extract the feature sets of the RGB image and the workpiece image respectively, and the feature sets between the RGB image and the workpiece image are matched through the k-nearest neighbor cross validation of BFMather to construct the pixel mapping matrix.

4. The deburring method combining 2D images and 3D point clouds according to claim 1, wherein: In step S4, the extraction of actual edge contour at sub-pixel level based on coarse extraction in combination with a traditional edge detection algorithm specifically includes: The traditional edge detection algorithm performs discrete noise elimination and non-maximum suppression on the edge probability map obtained by coarse extraction, and then repairs the broken edge points connected to each other for strong edge points with weak edge points. The sub-pixel level actual edge contour is extracted by fitting a quadratic surface model with the repaired edge points as the center.

5. The deburring method combining 2D images and 3D point clouds according to claim 4, characterized in that: The traditional edge detection algorithm is the Canny algorithm.

6. The deburring method combining 2D images and 3D point clouds according to claim 1, wherein: In step S5, the camera parameters of the 3D structured light camera are aligned to the actual point cloud, specifically including: After the actual edge contour is aligned to the RGB image, the pixel coordinates of the actual edge contour in the RGB image coordinate system are converted into normalized camera coordinate system coordinates based on the camera intrinsic parameter matrix and distortion coefficient in the camera parameters. Combined with the pose relationship between the 3D structured light camera and the world coordinate system defined by the camera extrinsic parameter matrix in the camera parameters, the normalized coordinates are mapped to the three-dimensional space coordinates of the actual point cloud through the perspective projection model to complete the alignment of the actual edge contour to the actual point cloud.

7. The deburring method combining 2D images and 3D point clouds according to claim 1, wherein: In step S6, the contour screening of the edge to be polished under the contour distance metric is specifically as follows: The Hausdorff distance between the contour of each target polishing surface in the actual point cloud and all actual edge contours is calculated, and the actual edge contours with a Hausdorff distance less than a preset threshold are selected as the edge contours to be polished.

8. The deburring method combining 2D images and 3D point clouds according to claim 1, wherein: In the step S7, the polishing trajectory acquisition based on the configuration information adjustment is specifically: based on the configuration information, corresponding morphological changes, smoothing and sampling operations are performed on the target polishing surface, and the polishing trajectory is acquired under burr interference removal.

9. The deburring method combining 2D images and 3D point clouds according to claim 1, wherein: In step S7, the grinding trajectory adjusted based on the configuration information is surrounded by a number of grinding segments configured with corresponding configuration information to form the edge contour to be ground, and the transition segments between adjacent grinding segments are generated by the interpolation algorithm to completely surround the target grinding surface.

Images