A deburring method combining 2D images and 3D point clouds

Abstract

The application discloses a deburring method combining 2D images and 3D point clouds, and relates to the technical field of image processing, and comprises the following steps: registering point clouds between a standard point cloud and an actual point cloud under surface orientation consistency verification; constructing a pixel mapping matrix between a workpiece image and an RGB image; performing rough extraction of a semantic-level edge on the workpiece image, and combining a traditional edge detection algorithm to perform extraction of an actual edge contour at a sub-pixel level on the basis of the rough extraction; aligning the actual edge contour to the actual point cloud through the pixel mapping matrix; aligning a target polishing surface to the actual point cloud, screening an edge contour to be polished based on the distance between the contour of each target polishing surface in the actual point cloud and all actual edge contours, and acquiring a polishing track in the workpiece image. The application fuses the sub-pixel edge positioning capability of the 2D image and the spatial topological information of the 3D point cloud, and overcomes the inherent defects of a single mode.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a deburring method combining 2D images and 3D point clouds. BACKGROUND

[0002] In the field of industrial manufacturing, burrs, as excess residues generated in the process of workpiece cutting, stamping or casting, can significantly reduce the assembly accuracy and service life of products. Traditional manual deburring methods are not only inefficient and inconsistent, but also difficult to meet the dual requirements of precision and efficiency of modern production lines. In recent years, automatic deburring technology based on machine vision has gradually become the mainstream solution, but the existing technology still has significant defects:

[0003] On the one hand, systems relying on single 3D point cloud processing can obtain three-dimensional topographic information of the workpiece, but are easily affected by point cloud sparsity and noise in the edge area. Point cloud missing leads to decreased registration accuracy, while outliers cause false positives or false negatives of burrs. For example, on thin-walled parts or complex curved workpieces, the point cloud generated by the structured light camera due to the blocking effect is broken, causing distortion of the edge profile reconstruction, and ultimately leading to deviation of the polishing path of the mechanical arm or empty running.

[0004] On the other hand, traditional solutions based on 2D vision can achieve accurate positioning of edges through high-resolution cameras (micron-level pixel size) and sub-pixel algorithms, but lack depth information, making it difficult to distinguish between real burrs and texture interference. When the workpiece surface is contaminated with oil, scratches or reflections, gradient calculation is easily misled, mistaking non-burr structures as targets for polishing. In addition, such systems usually require custom development of detection algorithms for each type of part, and users cannot directly define differentiated process parameters (such as local polishing intensity and path density) on three-dimensional models, resulting in low efficiency of production line switching and difficulty in adapting to flexible manufacturing requirements for multiple varieties and small batches.

[0005] The more fundamental contradiction is that existing technologies have not effectively integrated the complementary advantages of 2D and 3D data. The spatial topological information of 3D point clouds and the sub-pixel edge accuracy of 2D images are in a state of fragmentation, and multi-sensor data is simply superimposed rather than synergistically optimized. This fragmentation makes it difficult for the system to overcome the inherent defects of point clouds in the edge area and to utilize the positioning capabilities of 2D vision to achieve accurate locking of burrs. At the same time, the rigid binding of process parameters and processing paths further limits the adaptability of the system, resulting in the need for manual intervention to correct the trajectory when faced with complex-shaped workpieces. SUMMARY

[0006] In order to effectively integrate 2D and 3D data to achieve complementary advantages and reduce manual intervention in trajectory correction, the present application proposes a deburring method combining 2D images and 3D point clouds, comprising the steps of:

[0007] S1: extract and visualize the standard point cloud of the workpiece pre-registration, store the configuration information adjusted by the user based on the standard point cloud and the target polishing surface;

[0008] S2: obtain the actual point cloud of the workpiece and the corresponding RGB image through the 3D structured light camera, and perform point cloud registration between the standard point cloud and the actual point cloud under surface orientation consistency verification;

[0009] S3: obtain the workpiece image through the 2D area array camera, and construct a pixel mapping matrix between the workpiece image and the RGB image;

[0010] S4: perform semantic-level edge rough extraction on the workpiece image based on the DexiNed edge detection network, and perform sub-pixel-level actual edge contour extraction on the basis of the rough extraction combined with the traditional edge detection algorithm;

[0011] S5: align the actual edge contour extracted from the workpiece image to the RGB image through the pixel mapping matrix, and then align to the actual point cloud based on the camera parameters of the 3D structured light camera;

[0012] S6: based on the registration result between the point clouds, align the target polishing surface to the actual point cloud, and perform edge contour screening under contour distance measurement for each target polishing surface in the actual point cloud and all actual edge contours;

[0013] S7: map the screening result and the configuration information back to the workpiece image through the inverse process, and obtain the polishing trajectory of the target polishing surface corresponding to the edge contour to be polished in the workpiece image based on the configuration information adjustment.

[0014] The present application fuses the sub-pixel edge positioning capability of 2D image and the spatial topological information of 3D point cloud through multi-modal data collaborative processing, projects the 2D contour to the 3D point cloud coordinate system, effectively overcomes the inherent defects of traditional single modal technology, that is, the contour distortion problem caused by the sparsity of 3D point cloud in the edge area is compensated by 2D sub-pixel accuracy, and the weakness of 2D image vulnerable to texture interference is corrected through 3D spatial relationship, so that the scheme can effectively reduce the miss detection and false detection rate.

[0015] Further, in the S2 step, the point cloud registration between the standard point cloud and the actual point cloud under surface orientation consistency verification specifically includes:

[0016] The KFPCS coarse registration algorithm is used for initial pose transformation of the actual point cloud relative to the standard point cloud, the PICP fine registration algorithm is used for sub-millimeter level point cloud registration based on the initial pose transformation, and the normal vector consistency verification is introduced to avoid the turning situation after registration.

[0017] Further, in the S3 step, the pixel mapping matrix between the constructed workpiece image and the RGB image specifically comprises:

[0018] The SIFT feature matching algorithm is used to extract the feature sets of the RGB image and the workpiece image, and the BFMather k-neighbor cross-validation is used to match the feature sets between the RGB image and the workpiece image to construct the pixel mapping matrix.

[0019] Further, in the S4 step, the traditional edge detection algorithm is used to extract the actual edge contour at the sub-pixel level on the basis of the coarse extraction, and specifically comprises:

[0020] The traditional edge detection algorithm is used to perform discrete noise elimination and non-maximum suppression on the edge probability map obtained through the coarse extraction, repair the broken edges of the strong edge points connected with the weak edge points, extract the actual edge contour at the sub-pixel level based on the fitted quadratic surface model, and take the repaired edge points as the center.

[0021] Further, the traditional edge detection algorithm is the Canny algorithm.

[0022] Further, in the S5 step, the camera parameters based on the 3D structured light camera are aligned to the actual point cloud, and specifically comprises:

[0023] After the actual edge contour is aligned to the RGB image, the pixel coordinates of the actual edge contour in the RGB image coordinate system are converted into the normalized camera coordinate system coordinates based on the camera intrinsic matrix and the distortion coefficient in the camera parameters, the pose relationship between the 3D structured light camera and the world coordinate system is defined based on the camera extrinsic matrix in the camera parameters, the normalized coordinates are mapped to the three-dimensional space coordinates of the actual point cloud through the perspective projection model, and the alignment of the actual edge contour to the actual point cloud is completed.

[0024] Further, in the S6 step, the screening of the edge contour to be polished under the profile distance measurement specifically comprises:

[0025] The Hausdorff distance between the profile of each target polishing surface in the actual point cloud and all actual edge contours is calculated, and the actual edge contour with a Hausdorff distance less than a preset threshold is selected as the edge contour to be polished.

[0026] Further, in the S7 step, the polishing trajectory acquisition based on the configuration information adjustment specifically comprises: based on the configuration information, performing morphological changes, smoothing and sampling operations on the target polishing surface, and acquiring the polishing trajectory after removing the burr interference.

[0027] Further, in the S7 step, the polishing trajectory based on the configuration information is composed of a plurality of polishing segments configured with corresponding configuration information to enclose the edge profile to be polished, and a transition segment between adjacent polishing segments is generated by an interpolation algorithm to completely enclose the target polishing surface.

[0028] Compared with the prior art, the present application has at least the following beneficial effects:

[0029] (1) The deburring method combining 2D images and 3D point clouds proposed in the present application cooperates with multi-modal data processing to fuse the sub-pixel edge positioning ability of 2D images and the spatial topological information of 3D point clouds, so as to project the 2D contour to the 3D point cloud coordinate system, effectively overcoming the inherent defects of traditional single modal technology, i.e. the contour distortion problem caused by the sparsity of 3D point clouds in the edge region is compensated by 2D sub-pixel accuracy, and the weakness of 2D images vulnerable to texture interference is corrected through 3D spatial relationship, thereby reducing the missed detection and false detection rate;

[0030] (2) The KFPCS and PICP two-stage optimization strategy combined with the normal vector consistency verification mechanism solves the registration flip risk caused by initial pose deviation or noise in the industrial scene;

[0031] (3) In the edge extraction layer, the DexiNed network preferentially identifies the potential edge region under texture interference and outputs an edge probability map; the traditional algorithm focuses on the high response area to perform Gaussian noise reduction, non-maximum suppression and double threshold connection, and repairs the contour break caused by surface pores, and finally realizes sub-pixel positioning through quadratic surface fitting. This design greatly improves the weak edge detection rate, reduces edge jitter, and ensures the integrity and geometric accuracy of the burr contour;

[0032] (4) Support for user-driven flexible process configuration, users can directly define differentiated polishing areas and parameter libraries (such as high pressure and low speed in sharp corner areas) on standard point clouds, and after target polishing surface information is screened through edge similarity matching, it is mapped to the 2D execution coordinate system synchronously with the process parameters. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 A step diagram of a deburring method combining 2D images and 3D point clouds. DETAILED DESCRIPTION

[0034] The following is a specific embodiment of the present application combined with the drawings, which further describes the technical solutions of the present application, but the present application is not limited to these embodiments.

[0035] In the field of industrial automation deburring, the traditional single modal vision system has long been faced with the double dilemma of contour distortion caused by the sparsity of three-dimensional point cloud edges and misjudgment caused by two-dimensional image texture interference. Especially for engine cylinder, gearbox shell and other complex structural castings, the existing technology is difficult to realize the stable detection of sub-millimeter burrs under the interference of strong light, oil stains and air hole, and it is also difficult to support users to define process parameters flexibly according to differentiated processing needs. In order to break through this technical bottleneck, the present application proposes an innovative method combining multi-modal perception and intelligent decision-making: by synergistically using the spatial topology reconstruction capability of the 3D structured light camera and the sub-pixel edge analysis advantage of the 2D area array camera, a full closed-loop processing link of "point cloud registration-pixel mapping-edge fusion-process configuration" is constructed. As shown in Figure 1 The present application proposes a deburring method combining 2D image and 3D point cloud, including the steps of:

[0036] S1: extracting and visualizing the standard point cloud pre-entered by the workpiece, storing the configuration information and target polishing surface adjusted by the user based on the standard point cloud;

[0037] S2: acquiring the actual point cloud of the workpiece and the corresponding RGB image through the 3D structured light camera, and performing point cloud registration between the standard point cloud and the actual point cloud under surface orientation consistency verification;

[0038] S3: acquiring the workpiece image through the 2D area array camera, and constructing the pixel mapping matrix between the workpiece image and the RGB image;

[0039] S4: performing semantic-level edge rough extraction on the workpiece image based on the DexiNed edge detection network, and performing sub-pixel-level actual edge contour extraction based on the traditional edge detection algorithm on the basis of rough extraction;

[0040] S5: aligning the actual edge contour extracted from the workpiece image to the RGB image through the pixel mapping matrix, and then aligning it to the actual point cloud based on the camera parameters of the 3D structured light camera;

[0041] S6: based on the point cloud registration result, aligning the target polishing surface to the actual point cloud, and performing edge contour screening under contour distance measurement for each target polishing surface in the actual point cloud and all actual edge contours;

[0042] S7: mapping the screening result and the configuration information back to the workpiece image through the inverse process, and obtaining the polishing trajectory of the target polishing surface corresponding to the actual edge contour in the workpiece image based on the configuration information adjustment.

[0043] In the initialization stage of the deburring system, the system first extracts the standard point cloud model of the workpiece from the pre-stored CAD design library (usually converted from the three-dimensional model generated by industrial software such as NX, CATIA, etc.), visualizes the rendering through the Open3D or PCL point cloud processing engine, and dynamically displays the three-dimensional topology of the workpiece in the interactive interface. Based on this visualization interface, the user performs deep process configuration: can frame the specific edge area (such as the mold line of the engine cylinder body, the oil channel port of the gearbox shell, etc. which are high-risk areas of burrs) as the target polishing surface, and independently define the process parameter library for each surface - including but not limited to polishing force (200-500N adjustable), tool speed (8000-20000rpm gradient setting), feed speed (0.1-0.5m / s) and other key-value parameter sets. For complex cavity structures, the system supports directly drawing topological constraint lines of the machining path on the three-dimensional model, such as requiring the polishing trajectory to maintain tangential continuity at the corner or avoid interference with the positioning pin. All configuration information (geometric boundary definition + process parameters + path constraints) is stored in a structured data format to the local database, and the process parameter library is automatically associated with the material property library (such as the hardness coefficient of cast iron, aluminum alloy), providing decision basis for subsequent adaptive machining.

[0044] In the automated deburring system, the polishing trajectory of the mechanical arm needs to strictly follow the theoretical design model of the workpiece (standard point cloud generated by CAD), but the actual workpiece has pose differences with the theoretical model due to factors such as clamping deviation and machining deformation. If the pre-set trajectory is directly applied to the actual workpiece, it will cause two serious problems due to spatial mismatch: one is that the trajectory deviation causes burr missed polishing (such as residual burr on the edge of the cylinder body mold line), and the other is that the tool and the workpiece interfere with each other, causing collision accidents (such as thin-walled parts being punched due to trajectory deviation).

[0045] To eliminate this hidden danger, the invention selects a high-resolution 3D structured light camera (resolution ≥0.05mm) to scan the actual workpiece, synchronously acquires the actual point cloud model containing three-dimensional coordinate information and its corresponding RGB texture image, and performs point cloud registration to transform the standard point cloud to the actual point cloud coordinate system, so that all subsequent processing (such as target surface screening, trajectory mapping) is based on a unified spatial reference, and the actual workpiece is dynamically compensated during machining and clamping (such as ±2mm deformation caused by thermal expansion and contraction), avoiding the invalidation of pre-set process parameters.

[0046] For point cloud registration, the system first uses the KFPCS coarse registration algorithm to establish an initial transformation, including the following steps:

[0047] (1) Based on the ISS algorithm, the covariance matrix of each coordinate point in the standard point cloud is calculated , and the key points are extracted, which is expressed by the formula: ​

[0048] ,

[0049] wherein, is a neighborhood point set of coordinate point , is a random coordinate point in neighborhood point set , is a neighborhood centroid coordinate, and T is a matrix transpose.

[0050] (2) The key points satisfying the eigenvalue ratio threshold are reserved, and the formula is expressed as:

[0051] ,

[0052] wherein, is an eigenvalue of covariance matrix , is a threshold parameter for suppressing Pingtan area interference.

[0053] (3) Similarity measurement of four-point basis (4PCS) is performed, and an affine invariant ratio is calculated by selecting a group of coplanar points , which are not all collinear points, and the formula is expressed as:

[0054] ,

[0055] wherein, is the intersection of straight line and straight line .

[0056] (4) For the matched basis , solve by SVD decomposition:

[0057] The three-dimensional coordinates of the actual point cloud key points are solved ( , is the i-th coordinate point in the actual point cloud), which are used to calculate the translation vector together with the standard point cloud centroid coordinates (N is the number of matched key points). Then, the singular value decomposition (SVD) is performed on the covariance matrix of the key point set,

[0058] ,

[0059] wherein, is an orthogonal matrix, is a singular value diagonal matrix. The rotation matrix and the translation vector are solved,

[0060] ,

[0061] Final output initial transformation matrix , the pose deviation is preliminarily compressed.

[0062] Then, the PICP fine registration algorithm is used to calibrate the initial transformation matrix to the sub-millimeter level, including the following steps:

[0063] (1) solving coordinate points The minimum eigenvalue of the neighborhood covariance matrix corresponds to the vector ,

[0064] ,

[0065] By traversing all possible directions, find the unit vector (the projection variance of the neighborhood point in the unit vector direction) that minimizes the quadratic form .

[0066] (2) taking the minimum coordinate point to the tangent plane distance as the optimization objective,

[0067] ,

[0068] Wherein, is the coordinate point corresponding to the coordinate point in the actual point cloud and the standard point cloud, and K is the number of coordinate point pairs successfully establishing the corresponding relationship (the number of matching coordinate point pairs between the standard point cloud and the actual point cloud).

[0069] On the basis of the above coarse registration + fine registration, the invention further verifies the anti-reversing through the calculation of the homotopy of the normal vector.

[0070] After the actual point cloud and the standard point cloud are registered on the 3D data, the invention further collects the workpiece image of the actual workpiece through the 2D area array camera, and pre-processes: adopts adaptive Gaussian filtering to suppress environmental light noise, and extracts the workpiece main body area through Otsu threshold segmentation combined with connected domain analysis, eliminating the background interference of the conveying belt. Then, through SIFT (Scale-Invariant Feature Transform), the feature matching of the workpiece image and the RGB image synchronously acquired by the 3D structured light camera is carried out.

[0071] Specifically, SIFT is to detect local extreme points of an image at different scales by constructing a Gaussian pyramid of the image, which are usually the key points in the image such as edges, corners, etc. Then the positions and scales of the feature points are accurately located in the scale space, and the points with low contrast and strong edge response are excluded to ensure the stability of the key points. Finally, the direction of each key point is calculated to make the feature rotationally invariant, and a 16x16 window is generated around the key point, which is divided into 4x4 small blocks, and the direction histogram of each small block is calculated to form a 128-dimensional feature vector.

[0072] In this way, as long as the key points and feature descriptors are extracted from the RGB image and the workpiece image respectively using the SIFT algorithm, and the similarity between the feature descriptors is used for matching. Here, the BFMatcher is selected for bidirectional k-nearest neighbor cross matching, and the ratio test is applied to filter out unstable matches. Specifically, for each feature descriptor of the workpiece image , the Euclidean distance closest to and the second closest to in the feature descriptor set of the RGB image are searched . When the distance ratio is smaller than the nearest neighbor / second nearest neighbor distance ratio threshold , the candidate match is retained; further reverse verification is performed, and only when the nearest neighbor reverse match in the RGB image is , it is determined that is an effective match. Finally, based on the successfully matched point pairs , a homography matrix is solved , realizing low pixel error coordinate conversion between the coordinate systems of the two images . The formula is expressed as follows:

[0073] Let the feature set of the workpiece image be: , , the total number of feature points detected by the SIFT algorithm for the workpiece image, , the spatial coordinates of the i-th feature point in the workpiece image,

[0074] Let the feature set of the RGB image be: , , the total number of feature points detected by the SIFT algorithm for the RGB image, , the spatial coordinates of the j-th feature point in the RGB image,

[0075] Perform k-nearest neighbor search: , , the minimum number of nearest neighbor searches that meet the distance ratio test.

[0076] Distance ratio test: ,

[0077] Two-way cross-validation: forward matching , reverse matching Final matching set .

[0078] In addition, the application also improves the reliability of matching by using the RANSAC algorithm to eliminate incorrect matching pairs, and realizes the acquisition of a pixel mapping matrix with low pixel error between the coordinate systems of two images.

[0079] While constructing the pixel mapping matrix, the system also calls the pre-trained DexiNed edge detection network to perform semantic-level edge rough extraction on the preprocessed workpiece image. The network uses a multi-level coding and decoding architecture and is fine-tuned based on the ImageNet pre-trained weights. The input image resolution is fixed at 1024x768 pixels. The network output is an edge probability map, where each pixel value represents the likelihood of the position belonging to an edge (0-1 interval). This stage utilizes the feature fusion capability of deep learning to identify weak edge responses under complex textures, such as maintaining edge continuity in areas with oil stains or glare interference on the surface of castings, solving the edge breakage problem caused by texture interference in traditional algorithms.

[0080] After completing the semantic-level edge rough extraction, the system performs fine processing of the traditional edge detection algorithm. First, the edge probability map is subjected to discrete noise elimination: morphological erosion operation is used to remove isolated noise points (3x3 square structure kernel, 1 iteration), and then inflation operation is used to restore the morphological integrity of valid edges (same structure kernel, 2 iterations). Subsequently, non-maximum suppression processing is performed to retain local maximum points along the edge gradient direction and suppress non-peak responses. For edge breakage phenomena that still exist after the above processing (commonly found in workpiece porosity areas), the system starts the edge repair mechanism: set strong and weak edge double thresholds (strong edge threshold is 70% of the maximum gradient value, weak edge threshold is 30%), search for weak edge points in the 3x3 neighborhood of strong edge points, if the gradient directions of the two are consistent, establish a connection channel, effectively repair the contour interruption caused by local feature loss.

[0081] After obtaining the complete connected edge point set, the system implements sub-pixel level edge positioning. A 7x7 pixel neighborhood window is established around each edge point, and a quadratic surface model is fitted by least squares method. The model determines the sub-pixel coordinates by solving the extreme points of the partial derivatives of the surface equation: calculate the second derivative characteristics of the pixel gradient distribution in the window, accurately calculate the edge position according to the maximum value point of the quadratic surface curvature change, and thus realize the extraction of the actual edge contour at the sub-pixel level.

[0082] After the sub-pixel edge contour extraction of the workpiece image is completed, the system starts the cross-modal spatial alignment process. First, based on the pixel mapping matrix constructed in step S3, the actual edge contour points in the workpiece image coordinate system are mapped to the RGB image coordinate system. After the contour is aligned to the RGB image, the system loads the camera parameters of the 3D structured light camera to perform three-dimensional space mapping. The key operations include two core conversion stages:

[0083] (1) Distortion correction and normalization conversion: based on the camera intrinsic matrix (focal length and optical center ) and radial-tangential distortion coefficients, the contour points in the RGB image are corrected for distortion. Specifically, the distortion model is solved by an iterative approximation algorithm: assuming that the initial coordinates of the pixel points are , the normalized offset of the pixel points relative to the optical center is calculated, the radial and tangential offset is adjusted by introducing a distortion polynomial, and after 3 iterations, the corrected coordinates are obtained. Then the corrected coordinates are converted to normalized camera coordinate system coordinates by inverse perspective transformation:

[0084] .

[0085] (2) Three-dimensional space mapping: combined with the pose relationship between the camera and the world coordinate system defined by the camera extrinsic matrix (including the rotation matrix and the translation vector ), the depth information is recovered by the perspective projection model. Read the depth value corresponding to the RGB image pixel in the actual point cloud, convert the normalized coordinates to camera coordinate system three-dimensional coordinates :

[0086] ,

[0087] Finally, the camera coordinate system coordinates are converted to world coordinate system coordinates by the extrinsic matrix:

[0088] .

[0089] Through this process, the sub-pixel edge contour is aligned to the actual point cloud with millimeter-level precision.

[0090] Then, according to the point cloud registration result obtained in S2, the target polishing surface is aligned to the actual point cloud, and the system starts the edge similarity matching process. The core processing first parameterizes the boundary contour of each target polishing surface: a plane equation is fitted by a normal vector constrained RANSAC algorithm, the boundary point set is extracted and a B-spline curve is applied to fit a closed contour to form a target contour curve. At this time, the actual edge contour converted from the target contour curve in S5 exists in the form of a discrete point set. At this time, since the target contour curve (hereinafter referred to as the target contour curve) and the actual edge contour are both in the actual point cloud, the Hausdorff distance can be used to match the two.

[0091] First, the directed Hausdorff distance between the target contour curve and the actual edge contour point set is calculated , that is, the maximum value of the nearest distance of each point in to ; the reverse distance is calculated at the same time. The maximum value of the two defines the bidirectional Hausdorff distance, which represents the distance deviation of the most dissimilar parts of the two contours. In order to improve the calculation efficiency, the system establishes a three-dimensional KD-Tree to accelerate the nearest neighbor search.

[0092] Then, based on the preset distance threshold (dynamically adjusted according to the workpiece machining tolerance), the actual edge contour with a Hausdorff distance ≤ is selected as the edge contour to be polished.

[0093] After completing the selection of the edge contour to be polished, the system starts the polishing trajectory generation process. First, perform morphological adjustment driven by configuration information: according to the user's preset configuration information (such as polishing amount 0.2-0.5mm), perform adaptive deformation operation on the contour of the target polishing surface in the workpiece image. The core mechanism is to establish a functional relationship between the structure element radius and the polishing amount (structure element radius r = 1.5 polishing amount + 0.3 mm), and to perform image dilation operation using an elliptical structure kernel. This operation has a physical equivalent meaning in two-dimensional image space, which simulates the material removal range under the action of the tool envelope surface, ensuring that the projection area of the polishing trajectory on the workpiece surface covers all the burr areas.

[0094] Then, perform smoothing processing optimization: apply a non-uniform B-spline curve fitting to the contour point set after inflation. The curve control point distance is determined dynamically by the user's set feed speed (mm / s) and tool rotation speed (rpm), and the specific relationship is (where the proportionality coefficient According to the tool diameter in the 0.6-0.9 interval floating). By solving the chord length parameterization matrix of the curve control vertex coordinates, the C2 continuous space curve is generated by using the least square fitting constraint condition. The curve automatically inserts additional control points at the curvature mutation point (more than the set threshold 150 m-1), so that the local curvature is smoothly transitioned to the safe range.

[0095] In the sampling stage, the fitting curve is discretized based on the equal arc length segmentation principle (the number of arc length segments , is the total length of the contour). Each sampling point carries three process attributes: spatial pose (the tangent vector , normal vector is calculated by the differential geometry of the curve), machining parameter (mapping the user-configured force and rotation speed), and error compensation (the speed adjustment factor is calculated based on the curvature difference of adjacent points).

[0096] Finally, since the profile of a polishing surface can be divided into multiple segments to configure different parameters, the processing will be broken off, therefore, the invention generates a transition segment by using an interpolation algorithm for the disconnected part between adjacent profiles, and connects the adjacent two polishing segments. For example, a square surface is composed of four edges to form a complete closed contour, since the user is allowed to configure different parameters for the four edges respectively, the four edges cannot be connected after processing, and the transition segment is formed by the interpolation algorithm, so that the four edges can be connected to form a complete contour of the surface.

[0097] To sum up, the deburring method combining 2D image and 3D point cloud proposed by the invention fuses the sub-pixel edge positioning ability of 2D image and the spatial topological information of 3D point cloud through multi-modal data collaborative processing, so that the 2D contour is projected to the 3D point cloud coordinate system, effectively overcoming the inherent defects of traditional single modal technology, that is, the contour distortion problem caused by the sparsity of 3D point cloud in the edge region is compensated by 2D sub-pixel accuracy, and the weakness of 2D image vulnerable to texture interference is corrected through 3D spatial relationship, thereby reducing the missed detection and false detection rate.

[0098] The point cloud registration link adopts a KFPCS and PICP two-stage optimization strategy combined with a normal vector consistency verification mechanism, completely solving the registration flip risk caused by initial pose deviation or noise in the industrial scene. At the same time, at the edge extraction level, the DexiNed network preferentially identifies the potential edge region under texture interference and outputs an edge probability map; the traditional algorithm focuses on the high response area to perform Gaussian noise reduction, non-maximum suppression and double threshold connection, and repairs the contour break caused by surface pores, and finally realizes sub-pixel positioning through quadratic surface fitting. This design greatly improves the weak edge detection rate, reduces edge jitter, and ensures the integrity and geometric accuracy of the burr contour.

[0099] The application can also support user-driven flexible process configuration, and the user can directly define a differentiated polishing area and a parameter library (such as a high-pressure low-speed area of an acute angle) on a standard point cloud. After target polishing surface information is screened by edge similarity matching, the target polishing surface information is mapped to a 2D execution coordinate system synchronously with process parameters.

[0100] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present application are only used to explain the relative positional relationship, movement condition, etc. between components in a certain specific posture (as shown in the drawings), and if the specific posture changes, the directional indications also change accordingly.

[0101] In addition, the descriptions such as "first", "second", "one", etc. in the present application are only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise specifically limited.

[0102] In the present application, unless otherwise specifically defined and limited, the terms "connection", "fixing", etc. should be understood broadly, for example, "fixing" can be fixed connection, or detachable connection, or integral; can be mechanical connection, or electrical connection; can be directly connected, or indirectly connected through an intermediate medium; can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise specifically limited. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0103] In addition, the technical solutions of each embodiment of the present application can be combined with each other, but it must be based on the fact that a person skilled in the art can realize it, and when the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist, nor is it within the scope of protection required by the present application.

Claims

1. A deburring method combining 2D images and 3D point clouds, characterized in that, The method comprises the steps of: S1: extracting and visualizing the standard point cloud pre-entered by the workpiece, and storing the configuration information adjusted by the user based on the standard point cloud and the target polishing surface; S2: acquiring the actual point cloud of the workpiece and the corresponding RGB image through a 3D structured light camera, and performing point cloud registration between the standard point cloud and the actual point cloud under surface orientation consistency verification; S3: acquiring the workpiece image through a 2D area array camera, and constructing a pixel mapping matrix between the workpiece image and the RGB image; S4: performing semantic-level edge rough extraction on the workpiece image based on a DexiNed edge detection network, and performing sub-pixel-level actual edge contour extraction on the basis of the rough extraction combined with a traditional edge detection algorithm; S5: aligning the actual edge contour extracted from the workpiece image to the RGB image through the pixel mapping matrix, and then aligning to the actual point cloud based on the camera parameters of the 3D structured light camera; S6: aligning the target polishing surface to the actual point cloud based on the point cloud registration result, and performing profile distance measurement on the profile of each target polishing surface in the actual point cloud and all actual edge contours to screen the to-be-polished edge contour; S7: mapping the screening result and the configuration information back to the workpiece image through an inverse process, and obtaining the polishing track of the target polishing surface corresponding to the to-be-polished edge contour in the workpiece image based on the configuration information adjustment.

2. The deburring method of claim 1, wherein, In the S2 step, the point cloud registration between the standard point cloud and the actual point cloud under surface orientation consistency verification specifically comprises: performing initial pose transformation of the actual point cloud relative to the standard point cloud by using a KFPCS coarse registration algorithm, performing sub-millimeter-level point cloud registration through a PICP fine registration algorithm on the basis of the initial pose transformation, and avoiding the turning situation after registration by introducing normal vector consistency verification.

3. The deburring method of claim 1, wherein, In the S3 step, the construction of the pixel mapping matrix between the workpiece image and the RGB image specifically comprises: extracting the feature sets of the RGB image and the workpiece image by using a scale-invariant feature transform (SIFT) feature matching algorithm, and matching the feature sets between the RGB image and the workpiece image through k-nearest neighbor cross-validation of BFMather to construct the pixel mapping matrix.

4. The deburring method of claim 1, wherein, In the S4 step, the sub-pixel-level actual edge contour extraction on the basis of the rough extraction combined with the traditional edge detection algorithm specifically comprises: the traditional edge detection algorithm performs discrete noise elimination and non-maximum suppression processing on the edge probability graph obtained through rough extraction, performs edge point mutual connection-based fracture repair on strong edge points connected with weak edge points, takes the repaired edge points as the center to fit a quadratic surface model, and extracts the sub-pixel-level actual edge contour.

5. The deburring method of claim 4, wherein, The traditional edge detection algorithm is a Canny algorithm.

6. The deburring method of claim 1, wherein, In the S5 step, the alignment to the actual point cloud based on the camera parameters of the 3D structured light camera specifically comprises: After the actual edge profile is aligned to the RGB image, based on the camera intrinsic matrix and distortion coefficient in the camera parameters, the actual edge profile pixel coordinates in the RGB image coordinate system are converted into normalized camera coordinate system coordinates, the pose relationship between the 3D structured light camera and the world coordinate system defined by the camera extrinsic matrix in the camera parameters is combined, the normalized coordinates are mapped to the three-dimensional space coordinates of the actual point cloud through the perspective projection model, and the alignment of the actual edge profile to the actual point cloud is completed.

7. The deburring method of claim 1, wherein, In the S6 step, the screening of the edge profile to be polished in the profile distance metric is specifically: Calculate the Hausdorff distance between the profile of each target polishing surface in the actual point cloud and all actual edge profiles, and screen the actual edge profiles with a Hausdorff distance less than a preset threshold as the edge profile to be polished.

8. The deburring method of claim 1, wherein, In the S7 step, the polishing trajectory acquisition based on the configuration information adjustment is specifically: based on the configuration information, performing corresponding morphological changes, smoothing and sampling operations on the target polishing surface, and performing polishing trajectory acquisition under the removal of burr interference.

9. The deburring method of claim 1, wherein, In the S7 step, the polishing trajectory based on the configuration information adjustment is composed of a plurality of polishing sections configured with corresponding configuration information to surround the edge profile to be polished, and a transition section between adjacent polishing sections is generated through an interpolation algorithm to completely surround the target polishing surface.

Images