Industrial intelligent spraying tracing method

Abstract

The invention relates to the technical field of vision, in particular to an industrial intelligent spraying tracing method, which comprises the following steps of: firstly, acquiring an original depth map of a curved surface workpiece by utilizing a depth camera, and obtaining original data containing depth information; next, edge feature enhancement preprocessing is carried out on the original depth image, edge pixels are accurately extracted through morphological gradient operation and a user-defined anisotropic diamond kernel, and an edge mask image is constructed; then, curvature region division and self-adaptive interpolation restoration are carried out, quadratic polynomial curved surface fitting is carried out on a hole pixel neighborhood, a local curvature value is calculated, a curved surface region is divided according to the curvature value, and corresponding interpolation strategies are adopted for different regions; afterwards, edge detail sharpening post-processing is carried out on the repaired depth image, and the edge contrast is enhanced by using a non-sharpening mask algorithm; and the repaired and sharpened depth map is converted into a three-dimensional point cloud, workpiece pose information is determined based on three-dimensional point cloud data, and spraying track planning is carried out.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of industrial automation, robot spraying and machine vision, and in particular to an industrial intelligent spraying tracking method capable of solving the problem of spraying precision of curved workpieces. BACKGROUND

[0002] In industrial production, the industrial intelligent spraying tracking method aims to realize the automatic and accurate spraying of various workpieces to improve production efficiency and spraying quality. The general process covers multiple key links from image acquisition to final spraying operation.

[0003] Image acquisition link: The workpiece image information is collected by the camera of the vision module. For planar workpieces, a color camera can be used to obtain a two-dimensional color image, as the shape is regular and the features are relatively simple, and the two-dimensional image is sufficient to meet the subsequent processing requirements. For curved workpieces, considering the complexity of the surface geometry, a camera capable of outputting a depth map is needed to obtain an image containing depth information, providing multi-dimensional data support for subsequent three-dimensional model reconstruction and trajectory planning.

[0004] Image preprocessing and feature extraction link: The data processing module performs a series of preprocessing operations on the collected original image. Through histogram equalization, the original image with known gray probability is converted into a new image with uniform gray probability distribution, enhancing the image contrast and making the image details clearer, which is convenient for subsequent processing. Median filtering is to sort the pixels in the image neighborhood by brightness, and replace the original image gray value with the median value, effectively eliminating the burr noise in the image and making the image smoother. Laplace algorithm is used for image sharpening, which highlights the edge part of the image through second-order differential operation, enhances the contrast between the edge contour and other parts, and lays a foundation for subsequent image segmentation and edge detection, thereby obtaining key feature information such as the edge contour of the workpiece. For the depth map of curved workpieces, if the edge and image center have a hollow area with pixel value of 0, the fast marching method is used for repair, which estimates the pixel value of the pixel points on the boundary of the hollow area, continuously narrows the boundary, and finally completes the pixel value estimation of the area to be repaired.

[0005] Workpiece size measurement and pose determination link: Based on the edge detection result, the data processing module uses the Hough transform straight line detection method to measure the size of the workpiece. Select an appropriate straight line parameter equation and establish an accumulator, traverse the entire image, and accumulate the detected edge points. After calculation, find the maximum value of the accumulator to determine the straight line features of the workpiece edge, and then obtain the shape size information of the workpiece. At the same time, combined with two-dimensional image (for planar workpieces) or three-dimensional point cloud data (for curved workpieces), the pose information of the workpiece is determined to provide accurate workpiece position and attitude parameters for subsequent spraying path planning.

[0006] The spraying trajectory planning stage involves planning the spraying trajectory based on the workpiece type and the acquired workpiece dimensions and orientation information. For planar workpieces, the bottom of the spray gun is kept perpendicular to the workpiece plane, and a zigzag method is used for path planning within a plane parallel to the workpiece plane. In this method, the spray guns move in opposite directions between adjacent spraying paths. Paths with odd-numbered sequences move from the left end to the right end of the workpiece, with the lateral coordinate value continuously increasing; paths with even-numbered sequences move from the right end to the left end of the workpiece, with the lateral coordinate value continuously decreasing. Transition paths are perpendicular to the spraying paths. Except for the starting point of the first path, the lateral coordinate of the starting point of other paths is equal to the lateral coordinate of the ending point of the previous path. The starting point of the first path is determined by combining the workpiece boundary information. Furthermore, the segmented path is planned to be parallel to the long side of the minimum bounding rectangle of the workpiece to be sprayed, so as to reduce the number of path inflection points, improve spraying efficiency and robot operation stability; for curved workpieces, the posture change of the spray gun during the spraying process is determined according to the surface shape information, and a coating deposition model for curved surface spraying is constructed in combination with the planar coating deposition model, and a spraying trajectory adapted to the surface shape is planned so that the spray gun posture changes with the surface, ensuring uniform coating.

[0007] Trajectory data transmission and spraying operation execution: The communication module transmits the processed image data to the data processing module via an Ethernet RJ45 interface. After completing the spraying trajectory planning, the position and attitude data of the spraying points are transmitted to the position register of the spraying robot's controller via CH232 serial communication. When the workpiece is conveyed to the robot's spraying area by the conveyor, the spraying robot reads the spraying trajectory data from the controller's position register according to the robot program, and controls the spray gun to perform the spraying operation according to the planned trajectory, completing the spraying task on the workpiece.

[0008] Although existing industrial intelligent spraying tracking methods have achieved certain application results in industrial production, they still have obvious technical defects when dealing with curved workpieces, especially in depth map hole repair and subsequent related processes, which seriously affect spraying accuracy and coating quality.

[0009] The problem of depth map hole repair: Existing technologies mainly employ the fast traversal method when repairing depth map holes at the edges of curved workpieces. However, this method only estimates pixel values ​​based on the distance values ​​of neighboring pixels, failing to fully consider the complex geometric features of the curved surface edges, such as curvature and normal vector variations. This leads to a deviation between the repaired edge region depth value and the actual surface shape; originally clear edge details become blurred. When converted to point cloud data, edge blurring or distortion occurs, making the point cloud data unable to accurately reflect the true shape of the curved workpiece.

[0010] Impact on 3D point cloud reconstruction and spraying trajectory planning: Due to the distortion of edge details in the depth map, the calculation of the normal vector of the surface boundary is inaccurate during the conversion of the depth map into a 3D point cloud. The normal vector is crucial for determining the attitude of the spray gun during spraying; inaccurate normal vectors prevent the spraying trajectory planning from accurately matching the surface curvature changes. In actual spraying, the spray gun attitude, such as tilt angle and height from the surface, deviates from actual requirements, ultimately leading to uneven coating thickness. In industries with extremely high coating quality requirements, such as aerospace, uneven coating thickness can affect the service life and performance of parts, and even pose safety hazards. Summary of the Invention

[0011] To address the aforementioned technical problems, this invention provides an industrial intelligent spraying tracking method, comprising the following steps:

[0012] Acquire the original depth map of the curved surface workpiece;

[0013] By using morphological gradient operations and a custom anisotropic rhombic kernel, edge feature enhancement preprocessing is performed on the original depth map to extract edge pixels and construct an edge mask map.

[0014] The hole pixel neighborhood of the original depth map is fitted with a quadratic polynomial surface to calculate the local curvature value. Based on the curvature value, multiple surface regions are divided, and corresponding interpolation strategies are used to repair the depth map for different regions.

[0015] The repaired depth map is post-processed by sharpening edge details using an unsharpened mask algorithm to enhance edge contrast.

[0016] The repaired and sharpened depth map is then converted into a 3D point cloud. Based on the 3D point cloud data, the workpiece pose information is determined and the spraying trajectory is planned. The spraying robot controls the spray gun according to the trajectory data, so that the spray gun posture changes with the curvature of the surface to complete the spraying operation.

[0017] Furthermore, after the curved workpiece to be sprayed is transferred to the vision system area, the depth camera acquires images of the workpiece in the shooting area to obtain an original depth map containing depth information. The depth map contains empty areas with a pixel value of 0 at the edges and the center of the image, providing raw data for subsequent processing.

[0018] Furthermore, edge feature enhancement preprocessing is performed on the original depth map. Specifically, morphological gradient operations are first performed on the original depth map to preserve the surface edge features while identifying hole boundaries to guide subsequent repair directions. To avoid edge blurring caused by isotropic filtering kernels, a custom anisotropic rhombic kernel is selected as the structuring element. This kernel responds differently in the horizontal and vertical directions, which can enhance gradient changes along specific directions and improve edge preservation. The gradient value of each pixel is calculated using the maximum value of the four-neighbor difference, expressed as follows:

[0019]

[0020] Here, f(x,y) represents the depth value of pixel (x,y) in the depth map, and G(x,y) is the gradient response value of that point. A larger value indicates a stronger depth variation around the point, meaning it's more likely to be located at the edge of a curved surface. To effectively distinguish between edge and non-edge regions, a fixed gradient threshold is set. The judgment condition is:

[0021]

[0022] If the condition is met, the pixel (x,y) is marked as an edge pixel, and a value of 1 is assigned to the corresponding position in the edge mask image, while the other positions are 0, forming a binary edge mask image M(x,y).

[0023] Furthermore, in the edge feature enhancement preprocessing stage, the data processing module performs morphological gradient operations on the original depth map to extract structurally clear edge information. The structuring element used is an anisotropic rhombic kernel, whose major axis direction is dynamically adjusted according to the geometric trend of the curved workpiece in the image, ensuring that it remains consistent with the potential edge direction during morphological operations, thereby enhancing edge response and suppressing noise interference. The anisotropic rhombic kernel possesses direction selectivity, adapting to edge variation characteristics in regions with different curvatures. The morphological gradient is defined as the difference between dilation and erosion operations, expressed as...

[0024]

[0025] in This represents the gradient intensity of a pixel, f is the original depth map, and S is the diamond structure element in the current direction. This represents the result of an expansion operation with S as the structuring element. The result of the erosion operation is represented by the difference between the two values, reflecting the degree of edge change in the local area. In the actual calculation, the module calculates the grayscale difference between each pixel and its neighboring pixels based on the four-neighborhood, and takes the maximum absolute value as the final gradient response. This method enhances regions with abrupt depth changes in the image, ensuring that edge pixels can be accurately identified. Subsequently, by setting a gradient threshold, all pixels are binarized to generate an edge mask map. This mask map identifies all regions with strong gradient responses, which are then used as edge regions in subsequent processing. The edge mask map acts as a directional constraint in the interpolation restoration stage, preventing interpolation weights from crossing edge pixels, ensuring that interpolation only propagates within the continuous surface, avoiding edge blurring and structural distortion, thereby improving the structural consistency between the restored area and the real boundary.

[0026] Furthermore, the curvature region segmentation and adaptive interpolation repair specifically involve the data processing module extracting a 5×5 neighborhood pixel region centered on each missing pixel during the hole repair stage. A quadratic polynomial surface fitting is then performed using known depth values ​​to accurately reconstruct the local surface geometry. The fitting function adopts the form...

[0027]

[0028] Where z(x,y) is the depth value, (x,y) is the local coordinate, and the coefficients a, b, c, d, e, f are obtained by least squares fitting and are used to characterize the shape and variation trend of the surface in the neighborhood. Based on the fitted surface, the local Gaussian curvature of each pixel is further calculated using differential geometry methods, and its expression is:

[0029]

[0030] in , For the first-order partial derivative, , , , , represent the second-order partial derivatives, indicating the degree of curvature of the surface in different directions. , , are parameters obtained by fitting a quadratic polynomial surface to the neighborhood of the hole pixels, used to calculate the local Gaussian curvature to divide the surface region (smooth / convex / concave) and guide the interpolation strategy.

[0031] Based on the calculated numerical characteristics of K(x,y), the system divides the surface into three regions: when K≈0, the curvature of the region is gentle, belonging to an approximate plane; when K>0, the surface exhibits an outward convex shape; and when K<0, the surface has an inward concave structure. After the surface type identification is completed, the data processing module combines the gradient vector of the fitted surface for each region. The unit normal vector is calculated to guide the subsequent interpolation direction. Matching interpolation strategies are used for different curvature regions to ensure that the interpolation process faithfully reproduces the original surface shape, thereby improving the geometric consistency between the repaired area and the real surface and avoiding the generation of structural artifacts.

[0032] Furthermore, a matching interpolation strategy is adopted for different curvature regions, specifically:

[0033] For smooth regions, the interpolation strategy is as follows: When performing depth value restoration in smooth regions, the data processing module employs an improved bilinear interpolation method. This method introduces an edge mask map to constrain the interpolation direction, preventing structural aliasing caused by crossing edge regions during interpolation. This method introduces a combined weighting strategy of gradient response and spatial distance on top of traditional bilinear interpolation, allowing pixels that are far from the edge but spatially close to it to have a greater impact on the interpolation result. The interpolation calculation formula is as follows:

[0034]

[0035] in Let N represent the depth value of the pixel (x, y) to be repaired, and N be the set of pixels with known depth values ​​in its non-edge neighborhood. For pixels in the neighborhood The depth value, Spatial distance weights are defined as follows:

[0036]

[0037] Indicates spatial proximity to the target pixel. Let G(i,j) be the gradient value of pixel (i,j), and λ be the edge suppression coefficient, used to control the rate at which the gradient decays with respect to the weights. When a neighboring pixel is close to an edge region, its gradient value G(i,j) is larger, causing the corresponding exponential term to... This significantly reduces the contribution of that point to the interpolation result.

[0038] By introducing an edge response suppression term into the weights, the interpolation in the edge direction is weakened, making the interpolation process more inclined to preserve the structural smoothness and regional consistency, avoiding depth abrupt changes and error diffusion caused by crossing the edge, and improving the geometric continuity and realism of the repaired area.

[0039] The interpolation strategy used for convex / concave surface regions is as follows: When handling depth voids in convex and concave surface regions, the data processing module employs a radial basis function interpolation method based on the local normal vector direction to enhance the interpolation's adaptability to complex surface structures. This method first calculates the unit normal vector direction for each pixel based on the quadratic polynomial surface fitting results from the previous stage, and then selects a set of pixels with known depth values ​​distributed along the normal vector direction in the void boundary region as interpolation base points. The interpolation process uses a thin-plate spline function as the kernel function, and its interpolation expression is:

[0040]

[0041] in Let be the depth value of the point (x, y) to be repaired, and ∂Ω represent the set of pixels with known depths on the boundary of the hole. Let be the depth value of the boundary point (i,j). This represents the Euclidean distance between the repair point and the boundary point. This is a thin-plate spline function used to establish a smooth relationship between distance and weights. Because this function has high resolution at close range and gradually transitions smoothly at long range, it can naturally simulate the physical continuity of surface convexity and concavity. Simultaneously, when constructing interpolation weights, local normal vector directions are introduced for projection matching. Only boundary points with an angle less than a set threshold to the current normal vector direction are retained for interpolation, ensuring that the interpolation path is consistent with the main curvature direction of the surface and suppressing error propagation across surface directions. This strategy, by integrating spatial structural information and curvature-guided interpolation constraints, achieves depth recovery of convex and concave regions, improving the accuracy of the interpolation results in reproducing the surface morphology.

[0042] Furthermore, the repaired depth map undergoes edge detail sharpening post-processing. Specifically,

[0043] For the area covered by the edge mask map, an unsharpened mask algorithm is used to generate a low-frequency background by applying Gaussian blur to the repaired depth map. Then, the low-frequency background is subtracted from the original repaired depth map. The sharpening intensity of the edge area is higher than that of the non-edge area, thus enhancing the edge contrast.

[0044] Furthermore, in the post-processing stage of edge detail sharpening, the data processing module employs differentiated strategies for different regions based on the edge mask image to improve the edge sharpness of the depth map and the contour accuracy of the 3D point cloud reconstruction. First, the repaired depth map is blurred using Gaussian filtering to construct a low-frequency background image. The filter kernel size is dynamically adjusted according to the edge complexity; regions with significant changes in edge curvature use smaller Gaussian kernels, such as 3×3, to retain more detailed structure, while stable edge regions use larger Gaussian kernels, such as 5×5, to suppress background noise. The image generated by the blurring operation is denoted as... The formula for calculating the sharpened image is then:

[0045]

[0046] in This is the final sharpened depth value. The depth value is the result of the previous stage of restoration, and α is the sharpening coefficient. Its value is adaptively assigned based on the local state of the edge mask image. A higher sharpening coefficient, such as 2.0, is set for edge regions, while it is reduced to between 0.5 and 1.0 for non-edge regions to control over-enhancement. This strategy enhances the depth gradient at the edges by strengthening the high-frequency components in the image, compensating for the over-smoothing edge problem that may occur during interpolation restoration, and improving the fidelity of the subsequent point cloud in boundary curvature recovery. This results in higher response accuracy of the spraying trajectory planning process to the contour structure.

[0047] Furthermore, in the 3D point cloud reconstruction step, the data processing module, based on the repaired and sharpened depth map, uses the camera's intrinsic parameter model to precisely map the depth value of each pixel in the image domain to point coordinates in 3D space, thereby constructing complete point cloud data. The position of each pixel in the depth map is represented by coordinates... This indicates that its corresponding depth value is Z, obtained through the camera optical center coordinates in the intrinsic parameter matrix. and focal length parameters The transformation relationship between projecting two-dimensional pixels into three-dimensional space is as follows:

[0048]

[0049] Where X, Y, and Z represent the spatial coordinates of the pixel in the 3D point cloud, and Z is the depth distance represented by the grayscale intensity value in the depth map. , The coordinates of the principal point in the camera imaging coordinate system. , These are the focal lengths in the horizontal and vertical directions, respectively. After conversion, a dense point cloud is formed. By fitting local surfaces to the neighborhoods of each point in the point cloud and calculating the normal vector, information on surface curvature changes can be further obtained, providing a geometric basis for subsequent spraying trajectory planning. During the spraying path generation process, the system guides the real-time adjustment of the spray gun posture with the normal vector direction, ensuring that the spraying angle is consistent with the surface normal, thereby ensuring uniform coating thickness, high adhesion accuracy, and improving the spraying consistency and surface quality of curved workpieces.

[0050] Beneficial effects:

[0051] Accuracy Improvement: This invention effectively solves the accuracy problem of spraying curved workpieces in existing technologies through an innovative curvature adaptive hole repair algorithm based on morphological edge preservation. During the depth map hole repair process, edge pixels are accurately extracted through morphological gradient calculation and a custom anisotropic rhombic kernel to construct an edge mask map, avoiding excessive smoothing of edge regions during the repair process and preserving key edge details. Quadratic polynomial surface fitting is performed on the neighborhood of the hole pixels to calculate local curvature values. The surface regions are divided based on these curvature values, and corresponding interpolation strategies are adopted for different regions, improving the accuracy of point cloud reconstruction. Precise point cloud data makes the calculation of the surface boundary normal vector more accurate, providing accurate surface geometry information for spray trajectory planning, thus enabling the spray gun attitude to be precisely adjusted according to changes in surface curvature. In actual spraying, it ensures that the spray gun maintains the optimal distance and angle with the surface of the curved workpiece, achieving uniform coating coverage and avoiding excessively thick or thin coatings. This significantly improves coating uniformity and product quality, meeting the needs of industries with extremely high coating quality requirements, such as aerospace and high-end automotive manufacturing.

[0052] Efficiency Guarantee: From an algorithm complexity perspective, the algorithm of this invention is based on local neighborhood operations and fast gradient calculation, with a complexity of O(N·M) (where N×M is the image size). This means that the computational load is relatively small when processing images, enabling the processing and analysis of depth maps to be completed in a short time. In real-time industrial vision processing, this low-complexity algorithm can quickly perform operations such as hole repair and edge enhancement on the acquired depth map without affecting the production cycle due to complex image processing algorithms. On industrial production lines, the processed image data can be transmitted to subsequent modules in a timely manner, ensuring that the painting robot can quickly obtain accurate painting trajectory data, achieving efficient painting operations, improving the overall operating efficiency of the industrial intelligent painting system, and meeting the high efficiency requirements of industrial production.

[0053] Enhanced Adaptability: The method of this invention has broad adaptability to curved workpieces with different curvatures and shapes. In the curvature region segmentation stage, the surface is divided into smooth regions, convex regions, and concave regions by calculating local curvature values, and different interpolation strategies are adopted for different regions. Whether it is a surface with gentle curvature changes, a free-form surface with complex curvature changes, or a multi-curvature splicing region, effective depth map void repair can be performed. In practical applications, whether it is a workpiece with complex curved surfaces such as aero-engine blades or some curved surface parts of automobile bodies, the method of this invention can accurately process their depth maps, providing reliable data support for subsequent spraying trajectory planning. This ensures uniform coating coverage during the spraying process of various complex curved workpieces, meeting diverse industrial spraying needs and improving the applicability of industrial intelligent spraying systems in different production scenarios. Attached Figure Description

[0054] Fig. 1 This is a flowchart illustrating the present invention;

[0055] Fig. 2 This is a flowchart of the edge feature enhancement preprocessing process;

[0056] Fig. 3 This is a schematic diagram of curvature region division and interpolation strategy. Detailed Implementation

[0057] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0058] refer to Figs. 1-3 This invention provides an industrial intelligent spraying tracking method, which innovatively designs the process for repairing and processing voids in depth maps of curved workpieces, and mainly includes the following key steps:

[0059] The original depth map of the curved workpiece is acquired using a depth camera to obtain raw data containing depth information. Then, the original depth map is preprocessed with edge feature enhancement. By using morphological gradient operation and a custom anisotropic rhombus kernel, edge pixels are accurately extracted and an edge mask map is constructed.

[0060] Curvature region segmentation and adaptive interpolation repair are performed. Quadratic polynomial surface fitting is performed on the neighborhood of hole pixels to calculate local curvature values. The surface region is segmented based on the curvature values, and corresponding interpolation strategies are adopted for different regions.

[0061] The repaired depth map is post-processed by sharpening the edge details and using an unsharpened mask algorithm to enhance the edge contrast. The repaired and sharpened depth map is then converted into a 3D point cloud. Based on the 3D point cloud data, the workpiece pose information is determined and the spraying trajectory is planned.

[0062] The communication module transmits data via Ethernet RJ45 interface and CH232 serial port, and the painting robot controls the spray gun to complete the painting operation based on the trajectory data.

[0063] Curved workpiece depth map acquisition: After the workpiece transfer module transmits the curved workpiece to be sprayed to the vision system area, the depth camera of the vision module acquires images of the workpiece in the shooting area to obtain the original depth map containing depth information. The depth map may contain empty areas with a pixel value of 0 at the edges and the center of the image, providing raw data for subsequent processing.

[0064] Edge Feature Enhancement Preprocessing: The data processing module first performs morphological gradient calculations on the original depth map. The aim is to identify hole boundaries while preserving surface edge features in the depth map to guide subsequent repair. To avoid edge blurring caused by isotropic filtering kernels, a custom anisotropic rhombic kernel is selected as the structuring element. This kernel responds differently in the horizontal and vertical directions, enhancing gradient changes along specific directions and improving edge preservation. The gradient value of each pixel is calculated as the maximum value of the four-neighbor difference, expressed as follows:

[0065]

[0066] Here, f(x,y) represents the depth value of pixel (x,y) in the depth map, and G(x,y) is the gradient response value of that point. A larger value indicates a stronger depth variation around the point, meaning it's more likely to be located at the edge of a curved surface. To effectively distinguish between edge and non-edge regions, a fixed gradient threshold is set. The judgment condition is

[0067]

[0068] If this condition is met, the pixel (x, y) is marked as an edge pixel, and a value of 1 is assigned to the corresponding position in the edge mask image, while the remaining positions are set to 0, forming a binary edge mask image M(x, y). This mask image is used to constrain the subsequent interpolation repair direction, ensuring that interpolation operations are performed only in non-edge directions in regions near the edge, thereby avoiding edge blurring due to overly smooth interpolation. This strategy preserves the structural integrity of high-gradient regions, achieving edge detail retention in the depth map during hole repair, and improving the accuracy and stability of subsequent 3D reconstruction.

[0069] Curvature Region Segmentation and Adaptive Interpolation Repair: During the hole repair stage, the data processing module extracts a 5×5 neighborhood pixel region centered on each missing pixel. Using known depth values, a quadratic polynomial surface fitting is performed to accurately restore the local surface geometry. The fitting function adopts the form...

[0070]

[0071] Where z(x,y) is the depth value, (x,y) is the local coordinate, and the coefficients a, b, c, d, e, f are obtained by least squares fitting and are used to characterize the shape and variation trend of the surface in the neighborhood. Based on the fitted surface, the local Gaussian curvature of each pixel is further calculated using differential geometry methods, and its expression is:

[0072]

[0073] in, , For the first-order partial derivative, , , These are the second-order partial derivatives, representing the curvature of the surface in different directions. They are parameters obtained after fitting a quadratic polynomial surface to the neighborhood of the hole pixels, used to calculate the local Gaussian curvature to classify surface regions (smooth / convex / concave) and guide the interpolation strategy. Based on the calculated K(x,y) numerical characteristics, the system divides the surface into three types of regions: when K≈0, the curvature is smooth, belonging to an approximate plane; when K>0, the surface exhibits an outward convex shape; when K<0, the surface has an inward concave structure. After surface type identification, the data processing module combines the gradient vectors of the fitted surfaces for each region. The unit normal vector is calculated to guide the subsequent interpolation direction. Matching interpolation strategies are used for different curvature regions to ensure that the interpolation process faithfully reproduces the original surface shape, thereby improving the geometric consistency between the repaired area and the real surface and avoiding the generation of structural artifacts.

[0074] Different interpolation strategies are used for different regions:

[0075] For smooth regions: When performing depth value restoration in smooth regions, the data processing module employs an improved bilinear interpolation method. This method introduces an edge mask map to constrain the interpolation direction, preventing structural aliasing caused by crossing edge regions during interpolation. This method introduces a combined weighting strategy of gradient response and spatial distance on top of traditional bilinear interpolation, allowing pixels that are far from the edge but spatially close to it to have a greater impact on the interpolation result. The interpolation calculation formula is as follows:

[0076]

[0077] in Let N represent the depth value of the pixel (x, y) to be repaired, and N be the set of pixels with known depth values ​​in its non-edge neighborhood. For pixels in the neighborhood The depth value, Spatial distance weights are defined as follows:

[0078]

[0079] Indicates spatial proximity to the target pixel. Let G(i,j) be the gradient value of pixel (i,j), and λ be the edge suppression coefficient, used to control the rate at which the gradient decays with respect to the weights. When a neighboring pixel is close to an edge region, its gradient value G(i,j) is larger, causing the corresponding exponential term to... This significantly reduces the contribution of that point to the interpolation result. By introducing an edge response suppression term into the weights, the interpolation in the edge direction is weakened, making the interpolation process more inclined to preserve structural smoothness and regional consistency, avoiding depth abrupt changes and error propagation caused by crossing edges, and improving the geometric continuity and realism of the repaired area.

[0080] For convex / concave surface regions: When handling deep voids in convex and concave surface regions, the data processing module employs a radial basis function interpolation method based on local normal vector directions to enhance the interpolation's adaptability to complex surface structures. This method first calculates the unit normal vector direction for each pixel based on the quadratic polynomial surface fitting results from the previous stage, and then selects a set of pixels with known depth values ​​distributed along the normal vector direction in the void boundary region as interpolation base points. The interpolation process uses a thin-plate spline function as the kernel function, and its interpolation expression is as follows:

[0081]

[0082] in Let be the depth value of the point (x, y) to be repaired, and ∂Ω represent the set of pixels with known depths on the boundary of the hole. Let be the depth value of the boundary point (i,j). This represents the Euclidean distance between the repair point and the boundary point. This is a thin-plate spline function used to establish a smooth relationship between distance and weights. Because this function has high resolution at close range and gradually transitions smoothly at long range, it can naturally simulate the physical continuity of surface convexity and concavity. Simultaneously, when constructing interpolation weights, local normal vector directions are introduced for projection matching, retaining only boundary points with an angle less than a set threshold with the current normal vector direction for interpolation. This ensures that the interpolation path is consistent with the main curvature direction of the surface, suppressing error propagation across surface directions. This strategy, by fusing spatial structural information with curvature-guided interpolation constraints, achieves depth recovery of convex and concave regions, improving the accuracy of the interpolation results in reproducing the surface morphology.

[0083] Post-processing for edge detail sharpening: The data processing module uses an unsharpened mask algorithm to generate a low-frequency background by applying Gaussian blur to the repaired depth map for the area covered by the edge mask map. Then, the low-frequency background is subtracted from the original repaired depth map. The sharpening intensity of the edge area is higher than that of the non-edge area, thus enhancing the edge contrast.

[0084] 3D point cloud reconstruction and subsequent processing: The repaired and sharpened depth map is converted into a 3D point cloud. Based on the 3D point cloud data, the pose information of the workpiece is determined, and the spraying trajectory is planned so that the spray gun posture changes with the curvature of the surface to ensure uniform coating.

[0085] Trajectory data transmission and spraying operation execution: The communication module transmits the processed image data to the data processing module via the Ethernet RJ45 interface. After completing the spraying trajectory planning, the position and attitude data of the spraying point are transmitted to the controller of the spraying robot via CH232 serial communication. The spraying robot controls the spray gun to complete the spraying operation according to the trajectory data.

[0086] In the edge feature enhancement preprocessing stage, the data processing module performs morphological gradient operations on the original depth map to extract structurally clear edge information. The structuring element used is an anisotropic rhombic kernel, whose major axis direction is dynamically adjusted according to the geometric trend of the curved workpiece in the image, ensuring consistency with the potential edge direction during morphological operations. This enhances edge response and suppresses noise interference. The anisotropic rhombic kernel possesses direction selectivity, adapting to edge variation characteristics in regions with different curvatures. The morphological gradient is defined as the difference between dilation and erosion operations, expressed as...

[0087]

[0088] in This represents the gradient intensity of a pixel, f is the original depth map, and S is the diamond structure element in the current direction. This represents the result of an expansion operation with S as the structuring element. The result of the erosion operation is represented by the difference between the two values, reflecting the degree of edge change in the local area. In the actual calculation, the module calculates the grayscale difference between each pixel and its neighboring pixels based on the four-neighborhood, and takes the maximum absolute value as the final gradient response. This method enhances regions with abrupt depth changes in the image, ensuring that edge pixels can be accurately identified. Subsequently, by setting a gradient threshold, all pixels are binarized to generate an edge mask map. This mask map identifies all regions with strong gradient responses, which are then used as edge regions in subsequent processing. The edge mask map acts as a directional constraint in the interpolation restoration stage, preventing interpolation weights from crossing edge pixels, ensuring that interpolation only propagates within the continuous surface, avoiding edge blurring and structural distortion, thereby improving the structural consistency between the restored area and the real boundary.

[0089] In the post-processing stage of edge detail sharpening, the data processing module employs differentiated strategies for different regions based on the edge mask image to improve the edge sharpness of the depth map and the contour accuracy of the 3D point cloud reconstruction. First, the repaired depth map is blurred using Gaussian filtering to construct a low-frequency background image. The filter kernel size is dynamically adjusted according to the edge complexity; regions with significant changes in edge curvature use smaller Gaussian kernels, such as 3×3, to retain more detailed structure, while regions with stable edges use larger Gaussian kernels, such as 5×5, to suppress background noise. The image generated by the blurring operation is denoted as... The formula for calculating the sharpened image is then:

[0090]

[0091] in This is the final sharpened depth value. The depth value is the result of the previous stage of restoration, and α is the sharpening coefficient. Its value is adaptively assigned based on the local state of the edge mask image. A higher sharpening coefficient, such as 2.0, is set for edge regions, while it is reduced to between 0.5 and 1.0 for non-edge regions to control over-enhancement. This strategy enhances the depth gradient at the edges by strengthening the high-frequency components in the image, compensating for the over-smoothing edge problem that may occur during interpolation restoration, and improving the fidelity of the subsequent point cloud in boundary curvature recovery. This results in higher response accuracy of the spraying trajectory planning process to the contour structure.

[0092] In the 3D point cloud reconstruction step, the data processing module, based on the repaired and sharpened depth map, uses the camera's intrinsic parameter model to precisely map the depth value of each pixel in the image domain to point coordinates in 3D space, thereby constructing complete point cloud data. The position of each pixel in the depth map is represented by coordinates. This indicates that its corresponding depth value is Z, obtained through the camera optical center coordinates in the intrinsic parameter matrix. and focal length parameters The transformation relationship between projecting two-dimensional pixels into three-dimensional space is as follows:

[0093]

[0094] Where X, Y, and Z represent the spatial coordinates of the pixel in the 3D point cloud, and Z is the depth distance represented by the grayscale intensity value in the depth map. , The coordinates of the principal point in the camera imaging coordinate system. , These are the focal lengths in the horizontal and vertical directions, respectively. After conversion, a dense point cloud is formed. By fitting local surfaces to the neighborhoods of each point in the point cloud and calculating the normal vector, information on surface curvature changes can be further obtained, providing a geometric basis for subsequent spraying trajectory planning. During the spraying path generation process, the system guides the real-time adjustment of the spray gun posture with the normal vector direction, ensuring that the spraying angle is consistent with the surface normal, thereby ensuring uniform coating thickness, high adhesion accuracy, and improving the spraying consistency and surface quality of curved workpieces.

[0095] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An industrial intelligent spraying tracking method, characterized in that, Includes the following steps, Acquire the original depth map of the curved surface workpiece; By using morphological gradient operations and a custom anisotropic rhombic kernel, edge feature enhancement preprocessing is performed on the original depth map to extract edge pixels and construct an edge mask map. The hole pixel neighborhood of the original depth map is fitted with a quadratic polynomial surface to calculate the local curvature value. Based on the curvature value, multiple surface regions are divided, and an interpolation strategy is used to repair the depth map for different regions. The repaired depth map is post-processed by sharpening edge details using an unsharpened mask algorithm. The repaired and sharpened depth map is then converted into a 3D point cloud. Based on the 3D point cloud data, the workpiece pose information is determined and the spraying trajectory is planned.

2. The industrial intelligent spraying tracking method as described in claim 1, characterized in that, After the curved workpiece to be sprayed is transferred to the vision system area, the workpiece is image acquired by the depth camera to obtain the original depth map containing depth information. The depth map contains empty areas with a pixel value of 0 at the edges and the center of the image, which provide raw data for subsequent processing.

3. The industrial intelligent spraying tracking method as described in claim 1, characterized in that, The original depth map undergoes edge feature enhancement preprocessing. Specifically, morphological gradient operations are first performed on the original depth map to identify hole boundaries while preserving surface edge features, thus guiding subsequent repair directions. A custom anisotropic rhombic kernel is selected as the structuring element. This kernel responds differently in the horizontal and vertical directions, enhancing gradient changes along specific directions and improving edge preservation. The gradient value of each pixel is calculated using the maximum value of the four-neighbor difference, expressed as follows: Where f(x,y) represents the depth value of pixel (x,y) in the depth map, and G(x,y) is the gradient response value of that point; a fixed gradient threshold is set. To distinguish between edge regions and non-edge regions, the judgment condition is: If the condition is met, the pixel (x,y) is marked as an edge pixel, and a value of 1 is assigned to the corresponding position in the edge mask image, while the other positions are 0, forming a binary edge mask image M(x,y).

4. The industrial intelligent spraying tracking method as described in claim 1, characterized in that, In the edge feature enhancement preprocessing stage, morphological gradient operations are performed on the original depth map to extract edge information with clear structure. The structural element used is an anisotropic rhombic kernel, whose major axis direction is dynamically adjusted according to the geometric trend of the curved workpiece in the image, so that it can be consistent with the potential edge direction during morphological operations, thereby enhancing edge response and suppressing noise interference. The anisotropic rhombic kernel has direction selectivity and can adapt to the edge change features of different curvature regions. The morphological gradient is defined as the difference between the dilation and erosion operations, expressed as: in, This represents the gradient intensity of a pixel, f is the original depth map, and S is the diamond structure element in the current direction. This represents the result of an expansion operation with S as the structuring element. The result of the erosion operation is represented by the difference between the two values, which reflects the degree of edge change in the local area. In the actual operation, the gray level difference between each pixel and its neighboring pixels is calculated based on the four-neighborhood, and the maximum absolute value is taken as the final gradient response. This method strengthens the areas with abrupt changes in depth in the image and ensures that edge pixels can be accurately identified. Subsequently, by setting a gradient threshold, all pixels are binarized to generate an edge mask map. This mask map identifies all areas with strong gradient responses and uses them as edge regions for subsequent processing.

5. The industrial intelligent spraying tracking method as described in claim 1, characterized in that, Curvature region segmentation and adaptive interpolation repair, specifically, In the hole repair stage of the original depth map, for each missing pixel, a 5×5 neighboring pixel region centered on that point is extracted, and a quadratic polynomial surface fitting is performed using known depth values ​​to accurately restore the local surface geometry. The fitting function takes the form: Where z(x,y) is the depth value, (x,y) is the local coordinate, and the coefficients a, b, c, d, e, f are obtained by least squares fitting and are used to characterize the shape and variation trend of the surface in the neighborhood. Based on the fitted surface, the local Gaussian curvature of each pixel is further calculated using differential geometry, and its expression is: in, , For the first-order partial derivative, , , These are the second-order partial derivatives, representing the degree of curvature of the surface in different directions; Based on the calculated numerical characteristics of K(x,y), the system divides the surface into three types of regions: When K≈0, it indicates that the curvature of the region is gentle, and it is a flat region; When K>0, it indicates that the surface has an outward convex shape, which is a convex surface region; When K < 0, it indicates that the surface has a concave structure and is a concave surface region; After the surface type is identified, the data processing module combines the gradient vectors of the fitted surface in each region. Calculate its unit normal vector to guide subsequent interpolation direction; use matching interpolation strategies for different curvature regions.

6. The industrial intelligent spraying tracking method as described in claim 5, characterized in that, Different curvature regions employ a matching interpolation strategy, specifically: The interpolation strategy for smooth regions is as follows: When performing depth value restoration in smooth regions, an improved bilinear interpolation method is used, which constrains the interpolation direction by introducing an edge mask map; specifically, a composite weighting strategy of gradient response and spatial distance is introduced on the basis of traditional bilinear interpolation, so that pixels far from the edge but spatially close have a greater impact on the interpolation result; the interpolation calculation formula is: in, Let N represent the depth value of the pixel (x, y) to be repaired, and N be the set of pixels with known depth values ​​in its non-edge neighborhood. For pixels in the neighborhood The depth value, Spatial distance weights are defined as follows: Spatial distance weight This indicates the spatial proximity between the neighboring pixel (i,j) and the target pixel (x,y) to be repaired; the larger the value, the closer the two are in space. Let G(i,j) be the gradient value of pixel (i,j), and λ be the edge suppression coefficient, used to control the rate at which the gradient decays with respect to the weights. When a neighboring pixel is close to an edge region, its gradient value G(i,j) is larger, causing the corresponding exponential term to... This significantly reduces the contribution of that point to the interpolation result; The interpolation strategy used for convex / concave surface regions is as follows: When handling depth holes in convex and concave surface regions, a radial basis function interpolation method based on the local normal vector direction is adopted. Specifically, firstly, the unit normal vector direction of each pixel is calculated based on the quadratic polynomial surface fitting results of the previous stage, and a set of pixels with known depth values ​​distributed along the normal vector direction are selected as interpolation base points in the hole boundary region. The interpolation process uses a thin-plate spline function as the kernel function, and its interpolation expression is: in, Let be the depth value of the point (x, y) to be repaired, and ∂Ω represent the set of pixels with known depths on the boundary of the hole. Let be the depth value of the boundary point (i,j). This represents the Euclidean distance between the repair point and the boundary point. For thin plate spline functions, it is used to establish a smooth relationship between distance and weight; at the same time, when constructing interpolation weights, local normal vector directions are introduced for projection matching, and only boundary points with an angle less than a set threshold with the current normal vector direction are retained for interpolation, ensuring that the interpolation path is consistent with the main curvature direction of the surface and suppressing error propagation across the surface direction.

7. The industrial intelligent spraying tracking method as described in claim 1, characterized in that, The repaired depth map undergoes edge detail sharpening post-processing. Specifically, For the area covered by the edge mask map, an unsharpened mask algorithm is used to generate a low-frequency background by applying Gaussian blur to the repaired depth map. Then, the low-frequency background is subtracted from the original repaired depth map. The sharpening intensity of the edge area is higher than that of the non-edge area, thus enhancing the edge contrast.

8. The industrial intelligent spraying tracking method as described in claim 7, characterized in that, In the post-processing stage of edge detail sharpening, a differentiated strategy is adopted for different regions based on the edge mask image to improve the edge clarity of the depth map and the contour accuracy of the 3D point cloud reconstruction. First, the repaired depth map is blurred using Gaussian filtering to construct a low-frequency background image. The filter kernel size is dynamically adjusted according to the edge complexity; regions with large changes in edge curvature use smaller Gaussian kernels, such as 3×3, to retain more detailed structure, while regions with stable edges use larger Gaussian kernels, such as 5×5, to suppress background noise. The image generated by the blurring operation is denoted as... The formula for calculating the sharpened image is then: in, This is the final sharpened depth value. The depth value is the result of the previous stage of repair. α is the sharpening factor, which is adaptively assigned according to the state of the edge mask in the local location. A higher sharpening factor, such as 2.0, is set for the edge area, while the sharpening factor is reduced to between 0.5 and 1.0 for the non-edge area to control over-enhancement.

9. The industrial intelligent spraying tracking method as described in claim 1, characterized in that, In the 3D point cloud reconstruction step, based on the repaired and sharpened depth map, the camera's intrinsic parameter model is used to accurately map the depth value of each pixel in the image domain to point coordinates in 3D space, thereby constructing complete point cloud data; the position of each pixel in the depth map is represented by coordinates. This indicates that its corresponding depth value is Z, obtained through the camera optical center coordinates in the intrinsic parameter matrix. and focal length parameters The transformation relationship between projecting two-dimensional pixels into three-dimensional space is as follows: Where X, Y, and Z represent the spatial coordinates of the pixel in the 3D point cloud, and Z is the depth distance represented by the grayscale intensity value in the depth map. , The coordinates of the principal point in the camera imaging coordinate system. , These are the focal lengths in the horizontal and vertical directions, respectively. After conversion, a dense point cloud is formed. By fitting local surfaces to the neighborhoods of each point in the point cloud and calculating the normal vectors, information on surface curvature changes can be further obtained, providing a geometric basis for subsequent spraying trajectory planning.

Images