Positioning method and positioning device

Abstract

The application discloses a positioning method and a positioning device. The method comprises the following steps: collecting a depth map and preprocessing, removing outliers based on depth gradient smoothness analysis, and inferring and interpolating the object contour through gradient direction; extracting accurate single-pixel edges based on gradient features; converting the depth map into a point cloud and performing denoising correction; on the basis of traditional geometric centers, introducing multi-objective optimization of grabbing stability, upgrading the positioning center to an optimal grabbing target point considering contact area, gravity center load, surface flatness and clamping force constraints, and more suitable for high-precision stable grabbing of small objects without surface features. The whole process does not depend on object surface texture, gray or contour features, and only uses depth gradient geometric properties as the positioning basis. The application realizes high-precision and high-robustness three-dimensional positioning of objects without surface features, effectively improves the industrial grabbing success rate, and is suitable for automatic sorting and assembly scenes of micro electronic components, precision mechanical parts and the like.

Description

Technical Field

[0001] This invention relates to the field of industrial robot grasping technology, and in particular to a positioning method and positioning device. Background Technology

[0002] In industrial automation grasping scenarios, 3D grasping technology is a core component for achieving flexible manufacturing. Traditional grasping and positioning methods often employ template matching algorithms, which pre-construct feature templates of objects and compare them with visually acquired images to determine the object's position.

[0003] However, in practical applications, there are many small objects to be grasped with limited surface features (such as small metal parts, ceramic gaskets, and miniature injection molded parts). These objects share common characteristics: simple surface texture, simple geometric contours, and a lack of significant feature points. Traditional template matching methods struggle to effectively extract stable matching features, leading to the following problems: High matching failure rate: Due to insufficient feature points, template matching is prone to mismatches or matching failures; Low positioning accuracy: Even if the match is successful, the positioning is easily offset due to changes in lighting and attitude. Poor robustness: Sensitive to noise, obstruction, and background interference, making it difficult to meet the stability requirements of industrial production. In addition to traditional template matching algorithms, some people have tried to use deep learning methods to train neural networks for target detection and localization. However, this requires a large number of labeled samples, and it is difficult to obtain samples of small-sized precision parts. Furthermore, it still relies on surface feature mapping, has poor adaptability to objects without features, and lacks real-time performance. As a result, the success rate of robot grasping will drop significantly, and it will not be able to meet the stability requirements of industrial production.

[0004] To address the shortcomings of existing technologies, this invention provides a positioning method that achieves accurate positioning without relying on surface features, thereby improving the success rate of grasping small objects. Summary of the Invention

[0005] Therefore, the purpose of this invention is to provide a positioning method that achieves high-precision and robust positioning without the need for surface features by using depth map gradient smoothness analysis, gradient direction inference and completion, and point cloud center calculation, thereby improving the success rate of automated grasping.

[0006] To achieve the above objectives, the present invention provides a positioning method comprising the following steps: S1. Acquire the original depth map of the target object and its surrounding scene; the target object is an object with a small area and no significant surface features; S2. Preprocess the original depth map by using depth gradient continuity analysis to remove discrete noise points from the original depth map and obtain an effective depth map. S3. Based on the effective depth map, the complete outline of the target object is restored by gradient direction inference and interpolation to obtain the completed depth map; S4. Based on the completed depth map and the target object type, perform gradient feature-based edge extraction to obtain the single-pixel level precise edge of the target object and extract the target object. S5. Convert the extracted target object into a three-dimensional point cloud, and perform noise reduction and correction on the point cloud; S6. Based on the corrected point cloud computing, the optimal grasping target point of the target object is used as the robot's grasping target point.

[0007] More preferably, in S2, the preprocessing of the original depth map includes: invalid value masking and median filtering of the original depth map, specifically including the following steps: Construct a mask matrix with the same size as the original depth map and mark invalid pixels; For the regions in the original depth map that are not marked as invalid pixels, a median filter is used to filter them to obtain a preliminary denoised depth map. The filtered depth map and the mask matrix are retained as inputs for depth gradient continuity analysis.

[0008] More preferably, in S2, the depth gradient continuity analysis includes the following steps: The depth gradient magnitude G(x,y) of each pixel in the original depth map is calculated based on the first-order central difference of the discrete domain. ; ; Based on the area constraint of the target object ; simplified to ; ; Depth gradient magnitude ; in, This indicates the preprocessed depth map at the pixel level ( The depth value at ( ); () represents the pixel coordinates in the depth map; and These are the pixel spacings in the x and y directions, respectively. and Pixels Depth gradients in the x and y directions; Set a low gradient threshold when three or more consecutive adjacent pixels satisfy the depth gradient magnitude. If the gradient value is greater than or equal to the low gradient threshold and the gradient amplitude fluctuation is less than or equal to 2 pixels, it is determined to be a valid pixel of the target object; otherwise, it is marked as noise. The mask matrix is ​​updated and the noise is removed. The updated mask matrix contains only valid pixels or valid pixel markings.

[0009] Further preferably, in S3, based on the effective depth map, gradient direction inference and interpolation completion include: Locate the missing pixels at the edge of the target object in the effective depth map. The missing pixels are pixels marked as invalid and adjacent to the effective pixels in the updated mask matrix. The gradient direction of the effective pixels in the neighborhood of the missing pixel is calculated based on discrete domain numerical differentiation. : ; in The gradient direction is represented by radians. and Pixels Depth gradients in the x and y directions; Gradient direction radian value Convert to gradient direction angle value ; Based on the gradient direction angle value The indicated target object extension trend is used to perform linear interpolation using the depth values ​​of adjacent valid pixels to fill in the depth values ​​of missing pixels; after completion, the gradient continuity between the completed area and the original target object area is verified to ensure that the gradient amplitude fluctuation is ≤ 1 pixel.

[0010] More preferably, the step based on the gradient direction angle value The indicated object extension trend is obtained by linear interpolation using the depth values ​​of adjacent valid pixels, including: If the gradient direction angle value If the value is ∈[0°, 45°], then interpolate along the horizontal direction; if the gradient direction angle value If ∈ (45°, 135°], then interpolate along the vertical direction; if the gradient direction angle value If ∈ (135°, 225°], then interpolate in the horizontal direction; if the gradient direction angle value If ∈ (225°, 315°), then interpolate in the reverse direction along the vertical.

[0011] More preferably, in S4, the step of performing gradient feature-based edge extraction on the target object based on the completed depth map to obtain the single-pixel-level precise edge of the target object includes: calculating the depth gradient magnitude of the completed depth map using the first-order central difference in the discrete domain. And by linear normalization, the depth gradient magnitude is mapped to the range of [0,255] to obtain the normalized gradient map; Set an edge gradient threshold to initially filter candidate edge pixels from the normalized gradient map; Non-maximum suppression is applied to candidate edge pixels to refine wide edges into single-pixel edges, thus obtaining the precise single-pixel edges of the object. The non-maximum suppression includes: reducing the gradient direction angle value. The gradient magnitude is simplified to four main directions (0°, 45°, 90°, 135°). Based on the main directions, two neighboring pixels of the current pixel in the gradient direction are selected, and the gradient magnitude of the current pixel is calculated. If the gradient magnitude of the current pixel is the maximum value in the neighborhood, it is retained as an edge pixel; otherwise, it is suppressed to 0.

[0012] Further preferred embodiments include, S7, distinguishing multiple different target object types based on gradient features, specifically: Analyze the distribution characteristics of the normalized depth gradient magnitude, gradient direction characteristics, and edge morphology after non-maximum suppression; For sheet-like objects, the criteria for judgment include: the depth gradient magnitude has two symmetrical peak regions, the gradient directions of the two peak regions are opposite, the gradient magnitude of the middle region is close to 0, and the edges after non-maximum suppression are two parallel single-pixel straight lines. For cube-shaped objects, the criteria for judgment include: the depth gradient magnitude has 4 obvious gradient magnitude peaks, the peak value after normalization is ≥180, and the gradient magnitude in other areas is ≤80; the gradient direction has only 4 fixed directions, corresponding to the 4 edges of the cube, and the gradient direction of adjacent edge pixels changes abruptly. For conical objects, the criteria for determination include: the original depth gradient magnitude increases gradually from the bottom of the cone to the top, and the normalized depth gradient magnitude gradually increases, showing an asymmetrical distribution; the gradient direction is radially distributed around the center of the cone, but the coverage of the direction angle converges with the top.

[0013] More preferably, in S5, converting the extracted target object into a three-dimensional point cloud includes: Based on the pre-calibrated camera intrinsic parameter matrix K and depth value Z, the 3D coordinates corresponding to each pixel are calculated using the inverse projection formula: in, Focal length Principal point coordinates; For each pixel in the depth map; The three-dimensional coordinates of each pixel are aggregated to obtain the final three-dimensional point cloud.

[0014] Further preferably, in S6, the optimal grasping target point of the target object based on the corrected point cloud computing includes: Based on the single-pixel level precise edge obtained by S4 and the target object type determined by S7, the effective grasping area Ω is defined, the easily damaged area of ​​the object edge is eliminated, and the safe clamping range is retained. Construct a multi-target grasping stability evaluation function that includes clamping contact area, optimal clamping force, centroid projection stability, and depth gradient smoothness: in, For the weighting coefficients, satisfying ; For clamping contact area factor; The optimal clamping force factor; The centroid projection stability factor; It is the depth gradient smoothing factor; In the effective crawling domain The evaluation function is optimized by finding the extreme value, and the three-dimensional coordinates corresponding to the maximum value of the function are taken as the optimal target point for capture. .

[0015] Furthermore, the output positioning information for grasping includes: converting the three-dimensional center coordinates in the camera coordinate system to the world coordinate system, and then converting them to the grasping coordinates in the robot arm base coordinate system; Generate gripping instructions that include gripping position, posture, and force, and output them to the robotic arm actuator.

[0016] The present invention also provides a positioning device, implemented based on the positioning method described above, comprising: The data acquisition module is used to collect the original depth map of the target object and its surrounding scene; the target object is an object whose surface does not have significant features. The data preprocessing module preprocesses the original depth map, performs depth gradient continuity analysis, removes discrete noise points from the original depth map, and obtains an effective depth map. The depth map completion module, based on the effective depth map, restores the complete outline of the target object through gradient direction inference and interpolation to obtain the completed depth map; The target extraction module extracts the target object by performing gradient feature-based edge extraction based on the completed depth map and the target object. This yields a single-pixel-level accurate edge of the target object. The point cloud correction module converts the extracted target object into a three-dimensional point cloud and performs noise reduction and correction on the point cloud. The target localization module calculates the optimal grasping target point of the target object based on the calibrated point cloud computing, and uses this as the robot's grasping target point.

[0017] The localization method disclosed in this application, through a complete process of "depth map acquisition - preprocessing - point cloud fitting - 3D center calculation," focuses on depth information and geometric position, completely eliminating dependence on surface features. Depth map acquisition and alignment ensure accurate spatial mapping, preprocessing and planar correction eliminate environmental interference, point cloud fitting and filtering focus on geometric shape, and finally, outputs commands through 3D coordinate calculation. Compared with traditional techniques, it does not rely on texture contours to construct templates, avoiding localization failures caused by missing features; compared with deep learning, it does not require sample training, has better real-time performance, and is not limited by the lack of surface features. This solution accurately solves the problem of localizing miniature featureless parts, expands the application scenarios of machine vision localization, and improves the stability and efficiency of precision part grasping. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating a positioning method according to the present invention.

[0019] Figure 2 This is a schematic diagram of a positioning device provided by the present invention. Detailed Implementation

[0020] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] This application addresses the technical pain point of existing positioning technologies that rely on surface texture and contour features, leading to positioning failures for objects with low texture or no surface features (such as miniature metal pads and ceramic capacitors). It provides a technical solution based on depth information and geometric position as the core positioning basis. Completely eliminating reliance on object surface features, it achieves precise positioning of target objects through a series of standardized processes. The entire process requires no extraction of any texture or contour features from the object's surface, effectively solving the positioning problem for the aforementioned special objects. It offers high positioning accuracy, strong stability, and wide adaptability, and can be applied to automated grasping and positioning in scenarios such as microelectronic components and precision mechanical parts.

[0022] like Figure 1 As shown, one embodiment of the present invention provides a positioning method including the following steps: S1. Acquire the original depth map of the target object and its surrounding scene; the target object is an object with no significant surface features; before acquiring the original depth map, prioritize completing the camera parameter calibration preset, accurately setting core configurations such as internal and external parameters and distortion correction coefficients, eliminating inherent equipment errors, and providing a reliable accuracy benchmark for subsequent depth map acquisition and data processing, ensuring data accuracy throughout the entire process. Acquiring the depth map ensures that image resolution and depth information integrity meet standards, and coordinate calibration technology achieves precise pixel alignment, eliminating spatial offset and ensuring information consistency.

[0023] S2. Preprocess the original depth map by performing depth gradient continuity analysis and removing discrete noise from the original depth map to obtain an effective depth map. The effective depth map here only removes noise; object completion and edge extraction are still needed to remove the background and extract the target object.

[0024] The preprocessing primarily involves noise removal. In depth maps of small-area objects with few features, noise (isolated pixels caused by sensor noise and environmental interference) is easily confused with object pixels—both are "small-area pixel clusters" with no obvious texture differences. Accurate removal requires a combination of discrete-domain numerical differentiation and gradient magnitude analysis. The core idea is that noise lacks continuous depth variation and exhibits abnormal gradient magnitudes, while object pixels, although small in area, possess continuous depth gradient features. Planar correction is performed based on a preset reference plane to correct image shifts caused by shooting angle and device posture, laying a solid foundation for subsequent segmentation and extraction.

[0025] Specifically, the preprocessing of the original depth map includes: Invalid value masking and median filtering are applied to the original depth map; Construct a mask matrix with the same size as the original depth map. Mark invalid pixels; For the regions in the original depth map that are not marked as invalid pixels, a median filter is used to filter them to obtain a preliminary denoised depth map. The filtered depth map and the mask matrix are retained as inputs for depth gradient continuity analysis.

[0026] First, create an empty invalid value mask ( Invalid pixels are marked, and then subjected to median filtering. Initially remove salt and pepper noise, retain the core pixels of small objects, and select the appropriate filter kernel. This avoids excessive filtering that could lead to the loss of object pixels.

[0027] The depth gradient continuity analysis includes the following steps: The depth gradient magnitude G(x,y) of each pixel in the original depth map is calculated based on the first-order central difference of the discrete domain. ; ; Based on the area constraint of the target object ; simplified to ; ; Depth gradient magnitude ; Set a low gradient threshold when three or more consecutive adjacent pixels satisfy the depth gradient magnitude. If the gradient value is greater than or equal to the low gradient threshold and the gradient amplitude fluctuation is less than or equal to 2, it is determined to be a valid pixel of the target object; otherwise, it is marked as noise. The mask matrix is ​​updated and removed. The updated mask matrix only contains the identifiers of valid pixels and / or valid pixels. That is, if the number 0 represents invalid pixels and 1 represents valid pixels, the updated mask matrix will include 0 or 1. Therefore, noise marked as 0 can be removed.

[0028] Noise detection is based on the following conditions: isolated pixels (without 3 or more adjacent pixels) or gradient magnitudes of adjacent pixels. Gradient fluctuation > 2 pixels (no continuous depth change, judged as noise).

[0029] Pixels identified as noise are marked as invalid values, and the mask matrix is ​​updated. This yields a depth map containing only the core pixels of small-area objects. .

[0030] S3. Based on the effective depth map, the complete outline of the target object is restored by gradient direction inference and interpolation to obtain the completed depth map; Specifically, it includes: Locate the missing pixels at the edge of the target object in the effective depth map. The missing pixels are pixels marked as invalid and adjacent to the effective pixels in the updated mask matrix. The gradient direction angle value of the effective pixels in the neighborhood of the missing pixel is calculated based on discrete domain numerical differentiation. : ;in The gradient direction is represented by radians. The gradients are in the x and y directions; Gradient direction radian value Convert to gradient direction angle value ; ,like ,but

[0031] Based on the gradient direction angle value The indicated target object extension trend is used to perform linear interpolation using the depth values ​​of adjacent valid pixels to fill in the depth values ​​of missing pixels; after completion, the gradient continuity between the completed area and the original target object area is verified to ensure that the gradient fluctuation is ≤ 1 pixel.

[0032] If the gradient direction angle value (Horizontally extending to the right), missing pixel depth value: ; It is the depth value of the valid pixel to the left of the missing pixel (or as defined by the orientation); It is the depth gradient in the x-direction of the same location (or effective pixels in the neighborhood), multiplied by 1 because the pixel spacing Δx = 1.

[0033] Similarly, if the gradient direction angle value If ∈ (45°, 135°], then interpolate along the vertical direction; if the gradient direction angle value If ∈ (135°, 225°], then interpolate in the horizontal direction; if the gradient direction angle value If ∈ (225°, 315°), then interpolate in the reverse direction along the vertical.

[0034] Other gradient directions are based on the variation law of partial derivatives in the continuous domain to ensure that the gradient of the completed pixel is continuous and the depth is consistent with that of the original object pixel; Post-completion optimization: The gradient magnitude is recalculated on the completed depth map to ensure that the gradient fluctuation between the completed region and the original object region is ≤1, meeting the smoothness requirement of gradients in the continuous domain, thus obtaining a complete depth map of the small-area object. .

[0035] In particular, for target objects whose surfaces do not have significant texture or intensity characteristics (such as miniature metal pads and ceramic capacitors), the edge extension direction is prone to become blurred in low signal-to-noise ratio environments.

[0036] As a further optimized and enhanced embodiment, a gradient field correction step based on second-order geometric properties can be introduced before or in parallel with the "gradient direction inference and interpolation completion" step. Specifically, this includes: Based on the effective depth map, the rate of change of the normal or the direction of the principal curvature of the local surface is calculated to construct a smooth vector field reflecting the macroscopic geometric trend of the object. This serves as a means of smoothing and optimizing the depth gradient direction and providing auxiliary correction. The smoothed vector field is weighted and fused with the original calculated first-order depth gradient, or its direction is constrained, to eliminate gradient direction noise caused by discrete noise residue. Specifically, this includes: When in a low signal-to-noise ratio environment, when the consistency of the depth gradient direction is lower than a preset threshold, a smooth vector field S(x,y) based on second-order geometric properties is introduced. The smooth vector field S(x,y) is constructed from the direction of the surface normal rate of change or the direction of the principal curvature, and is defined as follows: in, To smooth the direction angle, The smoothing amplitude is the magnitude of the smoothing vector field.

[0037] The directional constraint can be implemented through a regularized optimization model: The fused orientation angle is calculated by weighting the smoothed orientation angle and the original gradient vector angle of the image. ; The fused amplitude θ is calculated by weighting the smoothed amplitude with the original gradient amplitude of the image. fused ; Therefore, the fused gradient vector is: .

[0038] Still using pixel-level depth gradient as the core execution unit, the missing edge pixels are filled by single-pixel interpolation based on the gradient direction after being corrected by the normal change rate / curvature direction.

[0039] By utilizing the gradient direction inference and interpolation completion described above, the positioning basis was switched from "the color pattern of the object surface" to "the geometric abrupt change of the object outline". The first-order gradient was used to ensure execution accuracy, and the second-order curvature was used to ensure inference robustness, which perfectly solved the problem of accurate positioning of low-texture micro parts in industrial environments with strong interference.

[0040] S4. Based on the completed depth map, perform gradient feature-based edge extraction on the target object to obtain the precise single-pixel edge of the target object; extract the target object.

[0041] The gradient magnitude is calculated using the first-order central difference method in the discrete domain for the completed depth map. And by linear normalization, the gradient magnitude is mapped to the range of [0,255] to obtain the normalized gradient map; The complete formula and adaptation instructions are as follows: , ; This is the set of gradient magnitudes for effective pixels of small-area objects, excluding NaN values; adaptation adjustment: because small-area objects have few pixels, , To avoid the influence of a single pixel on the normalization effect, the extreme values ​​within a 5-neighborhood are selected. After normalization, the values ​​are converted to uint8 format. Enhance the contrast of low gradient edges (the edge gradient magnitude of small objects is low, and normalization can improve their recognizability). Set an edge gradient threshold to initially filter candidate edge pixels from the normalized gradient map; set the edge gradient threshold. , The empirical value is 8~12, which adapts to the normalized low gradient edges and preserves... The pixels are used to initially filter out the object edges in order to remove residual low-gradient noise.

[0042] Non-maximum suppression is applied to candidate edge pixels to refine wide edges into single-pixel edges, thus obtaining the precise single-pixel edges of the object. The non-maximum suppression includes: reducing the gradient direction angle value. It is simplified to four main directions (0°, 45°, 90°, 135°). Based on the main direction, two neighboring pixels of the current pixel in the gradient direction are selected. If the gradient magnitude of the current pixel is the maximum value in the neighborhood, it is retained as an edge pixel; otherwise, it is suppressed to 0.

[0043] gradient direction angle value The formula is simplified to four main directions to reduce computational complexity. The complete formula is as follows: Selecting neighboring pixels based on the principal gradient direction and Ensure that all neighboring pixels are valid pixels of the object: if , , ;like , , Other directions are selected according to the corresponding rules; Non-maximum value determination: q and r: are two neighboring pixels selected based on the principal direction of the gradient.

[0044] If the gradient magnitude of the current pixel is the maximum value in its neighborhood, it is retained (determined to be an edge pixel); otherwise, it is suppressed to 0. The final result is... This refers to precise single-pixel edges of small objects, solving the problems of blurred and broken edges.

[0045] It should be noted that the following method is used to distinguish small objects with few features: Analyze the distribution characteristics of the normalized gradient magnitude, the gradient direction characteristics, and the edge morphology after nonmaximum suppression; Gradient magnitude: There are only two symmetrical gradient magnitude peaks (e.g., corresponding to the upper and lower edges of the thin slice). After normalization, the peak value is ≥160, and the gradient magnitude in the middle region is ≈0 (the thin slice is extremely thin, and the depth in the middle region does not change). Gradient direction: The gradient directions of the two peak regions are opposite (e.g., one is 0° and the other is 180°), and there are no gradients in other directions; Key criterion: The edge of NMS consists of two parallel single-pixel straight lines with opposite gradient directions, and the gradient magnitude in the middle region approaches 0.

[0046] S5. Convert the extracted target object into a three-dimensional point cloud, and perform noise reduction and correction on the point cloud; Regarding the target object, this application can distinguish multiple different target object types based on gradient features, specifically: Analyze the distribution characteristics of the normalized depth gradient magnitude, gradient direction characteristics, and edge morphology after non-maximum suppression; For sheet-like objects, the criteria for judgment include: the depth gradient magnitude has two symmetrical peak regions, the gradient directions of the two peak regions are opposite, the gradient magnitude of the middle region is close to 0, and the edges after non-maximum suppression are two parallel single-pixel straight lines. For cube-shaped objects, the criteria for judgment include: the depth gradient magnitude has 4 obvious gradient magnitude peaks, the peak value after normalization is ≥180, and the gradient magnitude in other areas is ≤80; the gradient direction has only 4 fixed directions, corresponding to the 4 edges of the cube, and the gradient direction of adjacent edge pixels changes abruptly. For conical objects, the criteria for determination include: the original depth gradient magnitude increases gradually from the bottom of the cone to the top, and the normalized depth gradient magnitude gradually increases, showing an asymmetrical distribution; the gradient direction is radially distributed around the center of the cone, but the coverage of the direction angle converges with the top.

[0047] Based on the pre-calibrated camera intrinsic parameter matrix K and depth value Z, the 3D coordinates corresponding to each pixel are calculated using the inverse projection formula: in, Focal length Principal point coordinates; For each pixel in the depth map.

[0048] During pre-calibration, Zhang Zhengyou's calibration method was used. By photographing the checkerboard calibration board in different poses, the camera intrinsic parameter matrix K and distortion coefficient D were calculated. in, Focal length Principal coordinates; for each pixel in the top depth map Combined with the corresponding depth values ​​in the depth map Calculate the three-dimensional coordinates using the inverse projection formula It should be noted that the 3D coordinates at this time are in the camera coordinate system; Remove the point cloud corresponding to noisy pixels at the edge of the depth map (usually within a 3-pixel range of the edge); Statistical filtering (with a standard deviation threshold of 3) is used to remove outliers from the point cloud, retaining only the valid object point cloud set. .

[0049] S6. Based on the corrected point cloud computing, the optimal grasping target point of the target object is used as the robot's grasping target point.

[0050] Traditional geometric center methods obtain the 3D center through arithmetic averaging of point clouds, considering only geometric position without taking into account the actual stability of industrial gripping. This can easily lead to slippage, off-center loading, and excessive clamping force damaging the workpiece when gripping small, smooth objects with no surface features. To address this, this paper proposes an optimal gripping stability optimization algorithm. This algorithm transforms the positioning problem into a multi-objective optimization problem, finding the optimal gripping target point within the effective gripping area that balances contact area, center of gravity constraints, surface flatness, and clamping force matching.

[0051] First, based on the single-pixel level precise edge obtained by S4 and the object geometry type determined by S7, the effective gripping area Ω is defined, and easily damaged areas of the object edge are eliminated, while a safe and stable gripping range is retained.

[0052] Subsequently, a crawling stability evaluation function integrating four key indicators was constructed: In the formula: The clamping contact area factor represents the proportion of the effective point cloud in the neighborhood of the grasping point; the larger the value, the more complete the contact. With the calculated point cloud center Centered on the effective crawling domain Define the k-neighborhood (thin sheet class) Pixels correspond to the neighborhood of the point cloud, cube / cone classes are taken (pixel corresponds to the neighborhood of the point cloud). This represents the total number of point clouds within the neighborhood. The effective number of point clouds of the target object in the neighborhood (judgment criteria: the deviation between the point cloud depth value and the average depth value of the target object is ≤ ±0.1mm). The higher the percentage of effective point cloud within the neighborhood, the larger the value; when there is no effective point cloud... When there is no contact cavity .

[0053] The optimal clamping force factor characterizes the degree of fit between the required clamping force at the current point and the ideal clamping force of the object material. ; The optimal clamping force factor is used to characterize the center of the calculated point cloud. The degree of matching between the required clamping force and the ideal clamping force of the target object material is important to avoid slippage due to insufficient clamping force or damage to precision parts due to excessive clamping force.

[0054] Among them, the clamping force required for calculation theory ,in, The surface density of the target object ( (Calculated from point cloud volume and material density) Let gravitational acceleration be (take) ), The coefficient of friction between the gripper and the object surface (0.3-0.5 for rubber grippers, 0.1-0.2 for metal grippers). The ideal clamping force (N) for the target object material is selected, and the median value of the safe clamping force range is chosen (0.5-1.0N for metal parts, 0.2-0.5N for ceramic parts, and 0.1-0.3N for plastic parts). like ,but .therefore, The closer the theoretical clamping force is to the ideal clamping force, the larger the value; when perfectly matched... ;

[0055] It is the center of gravity projection stability factor, used to suppress center of gravity off-center loading and improve clamping posture stability; ; Calculate the effective crawling domain Center of gravity of the inner target object , , in, To effectively crawl domains Total point cloud count of internal target objects For a single point cloud coordinate; center of gravity Projected onto the clamping plane containing the center of the calculated point cloud (horizontal clamping, camera coordinate system) (surface), to obtain the centroid projection point , For the calculation of the point cloud center The depth value; Computational point cloud center arrive Planar Euclidean distance: ; for The maximum diagonal length of the plane within the object (determined by the point cloud of the object's single-pixel edge).

[0056] Value characteristics: When the calculated point cloud center coincides with the centroid projection point , The calculated point cloud center is located at At the edge , ; The depth gradient smoothing factor reflects the surface smoothness of the grasping area, preventing positioning offset caused by abrupt changes in local depth; the depth gradient smoothing factor is used to characterize the center of the calculated point cloud. The surface smoothness within the neighborhood reflects the degree of depth gradient fluctuation in the gripping area. The smaller the gradient fluctuation, the smoother the surface, and the more uniform the force on the gripper. The specific calculation formula is as follows: The neighborhood delineation is completely consistent with the clamping contact area factor, that is, based on the calculated point cloud center. Define a k-neighborhood around the center; calculate the mean depth gradient magnitude of all point clouds within the neighborhood. The depth gradient magnitude is calculated using the first-order central difference formula in the discrete domain according to this method; the variance of the depth gradient magnitude within the neighborhood is calculated. This reflects the degree of gradient dispersion.

[0057] When the depth gradient magnitude in the neighborhood is 0 (absolutely flat), , , The greater the gradient fluctuation, the smaller the value.

[0058] For the weighting coefficients, satisfying Adjustments are made based on sheet-like, cube-like, and cone-like objects. Specific adaptation values ​​are: Thin slices: α=0.3, β=0.3, γ=0.2, δ=0.2; Cube type: α=0.25, β=0.25, γ=0.25, δ=0.25; Cone type: α=0.2, β=0.3, γ=0.3, δ=0.2.

[0059] In the effective crawling domain The evaluation function is optimized by finding the extreme value, and the three-dimensional coordinates corresponding to the maximum value of the function are taken as the optimal target point for capture.

[0060] Finally, when outputting the positioning information for optimal target grasping, the three-dimensional center coordinates in the camera coordinate system are converted to the world coordinate system, and then to the grasping coordinates in the robot arm base coordinate system. Generate gripping instructions that include gripping position, posture, and force, and output them to the robotic arm actuator.

[0061] like Figure 2 The present invention also provides a positioning device, implemented based on the positioning method described above, comprising: The data acquisition module is used to collect the original depth map of the target object and its surrounding scene; the target object is an object whose surface does not have significant features. The data preprocessing module preprocesses the original depth map, performs depth gradient continuity analysis, removes discrete noise points from the original depth map, and obtains an effective depth map. The depth map completion module, based on the effective depth map, restores the complete outline of the target object through gradient direction inference and interpolation to obtain the completed depth map; The target extraction module extracts the target object by performing gradient feature-based edge extraction based on the completed depth map and the target object. This yields a single-pixel-level accurate edge of the target object. The point cloud correction module converts the extracted target object into a three-dimensional point cloud and performs noise reduction and correction on the point cloud. The target localization module calculates the optimal grasping target point of the target object based on the calibrated point cloud computing, and uses this as the robot's grasping target point.

[0062] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A positioning method, characterized in that, Includes the following steps: S1. Acquire the original depth map of the target object and its surrounding scene; the target object is an object whose surface does not have significant features; S2. Preprocess the original depth map by using depth gradient continuity analysis to remove discrete noise points from the original depth map and obtain an effective depth map. S3. Based on the effective depth map, the complete outline of the target object is restored by gradient direction inference and interpolation to obtain the completed depth map; S4. Based on the completed depth map and the target object type, perform gradient feature-based edge extraction to obtain the single-pixel level precise edge of the target object and extract the target object. S5. Convert the extracted target object into a three-dimensional point cloud, and perform noise reduction and correction on the point cloud; S6. Based on the corrected point cloud computing, the optimal grasping target point of the target object is used as the robot's grasping target point, including: Based on the single-pixel level precise edge obtained by S4 and the determined target object type, the effective grasping domain Ω is defined, the easily damaged area of ​​the object edge is eliminated, and the safe clamping range is retained. Construct a multi-target grasping stability evaluation function that includes clamping contact area, optimal clamping force, centroid projection stability, and depth gradient smoothness: ； in, For the weighting coefficients, satisfying ; The clamping contact area factor represents the percentage of the effective point cloud in the neighborhood of the grab point; The optimal clamping force factor characterizes the degree of fit between the required clamping force at the current point and the ideal clamping force of the object material. This is the center of gravity projection stability factor, used to suppress center of gravity off-center loading; The depth gradient smoothing factor is used to characterize the center of the calculated point cloud. Surface smoothness within the neighborhood; In the effective crawling domain The evaluation function is optimized by finding the extreme value, and the three-dimensional coordinates corresponding to the maximum value of the function are taken as the optimal target point for capture. ； These are the X, Y, and Z coordinates of the optimal target capture point; It also includes converting the three-dimensional coordinates of the optimal grasping target point in the camera coordinate system to the world coordinate system, and then to the grasping coordinates in the robot arm base coordinate system; Generate gripping instructions that include gripping position, posture, and force, and output them to the robotic arm actuator.

2. The positioning method according to claim 1, characterized in that, In S2, the preprocessing of the original depth map includes: invalid value masking and median filtering, specifically including the following steps: Construct a mask matrix with the same size as the original depth map and mark invalid pixels; For the regions in the original depth map that are not marked as invalid pixels, a median filter is used to filter them to obtain a preliminary denoised depth map. The filtered depth map and the mask matrix are retained as input for depth gradient continuity analysis.

3. The positioning method according to claim 1, characterized in that, In S2, the depth gradient continuity analysis includes the following steps: The depth gradient magnitude G(x,y) of each pixel in the original depth map is calculated based on the first-order central difference of the discrete domain. ； ； Based on the area constraint of the target object ; simplified to ； ； Depth gradient magnitude ; in, This indicates the preprocessed depth map at the pixel level ( The depth value at ( ); () represents the pixel coordinates in the depth map; and y represents the pixel spacing in the x and y directions, respectively; Set a low gradient threshold when three or more consecutive adjacent pixels satisfy the depth gradient magnitude. If the gradient value is greater than or equal to the low gradient threshold and the gradient amplitude fluctuation is less than or equal to 2 pixels, it is determined to be a valid pixel of the target object; otherwise, it is marked as noise. The mask matrix is ​​updated and the noise is removed. The updated mask matrix contains only valid pixels or valid pixel markings.

4. The positioning method according to claim 3, characterized in that, In S3, based on the effective depth map, gradient direction inference and interpolation completion are performed, including: Locate the missing pixels at the edge of the target object in the effective depth map. The missing pixels are pixels marked as invalid and adjacent to the effective pixels in the updated mask matrix. The gradient direction θ(x,y) of the valid pixels in the neighborhood of the missing pixel is calculated based on discrete domain numerical differentiation: ;in The gradient direction is represented by radians. and pixels Depth gradients in the x and y directions; Gradient direction radian value Convert to gradient direction angle value ; Based on the gradient direction angle value The indicated target object extension trend is used to perform linear interpolation using the depth values ​​of adjacent valid pixels to fill in the depth values ​​of missing pixels; after completion, the gradient continuity between the completed area and the original target object area is verified to ensure that the gradient amplitude fluctuation is ≤ 1 pixel.

5. The positioning method according to claim 4, characterized in that, The gradient direction angle value The indicated object extension trend is obtained by linear interpolation using the depth values ​​of adjacent valid pixels, including: If the gradient direction angle value If the value is ∈[0°, 45°], then interpolate along the horizontal direction; if the gradient direction angle value If ∈ (45°, 135°], then interpolate along the vertical direction; if the gradient direction angle value If ∈ (135°, 225°], then interpolate in the horizontal direction; if the gradient direction angle value If ∈ (225°, 315°), then interpolate in the reverse direction along the vertical.

6. The positioning method according to claim 1, characterized in that, In S4, the step of performing gradient feature-based edge extraction on the target object based on the completed depth map to obtain the single-pixel-level precise edge of the target object includes: calculating the depth gradient magnitude of the completed depth map using the first-order central difference in the discrete domain. And by linear normalization, the depth gradient magnitude is mapped to the range of [0,255] to obtain the normalized gradient map; Set an edge gradient threshold to initially filter candidate edge pixels from the normalized gradient map; Non-maximum suppression is applied to candidate edge pixels to refine wide edges into single-pixel edges, thus obtaining the precise single-pixel edges of the object. The non-maximum suppression includes: reducing the gradient direction angle value. θ The gradient direction is simplified to four main directions: 0°, 45°, 90°, and 135°. Based on the main directions, two neighboring pixels of the current pixel in the gradient direction are selected, and the gradient magnitude of the current pixel is calculated. If the gradient magnitude of the current pixel is the maximum value in the neighborhood, it is retained as an edge pixel; otherwise, it is suppressed to 0.

7. The positioning method according to claim 6, characterized in that, It also includes distinguishing multiple different target object types based on gradient features, specifically: Analyze the distribution characteristics of the normalized depth gradient magnitude, gradient direction characteristics, and edge morphology after non-maximum suppression; For sheet-like objects, the criteria for judgment include: the depth gradient magnitude has two symmetrical peak regions, the gradient directions of the two peak regions are opposite, the gradient magnitude of the middle region is close to 0, and the edges after non-maximum suppression are two parallel single-pixel straight lines. For cube-shaped objects, the criteria for judgment include: the depth gradient magnitude has 4 obvious gradient magnitude peaks, the peak value after normalization is ≥180, and the gradient magnitude in other areas is ≤80; the gradient direction has only 4 fixed directions, corresponding to the 4 edges of the cube, and the gradient direction of adjacent edge pixels changes abruptly. For conical objects, the criteria for determination include: the original depth gradient magnitude increases gradually from the bottom of the cone to the top, and the normalized depth gradient magnitude gradually increases, showing an asymmetrical distribution; the gradient direction is radially distributed around the center of the cone, but the coverage of the direction angle converges with the top.

8. The positioning method according to claim 1, characterized in that, In S5, converting the extracted target object into a 3D point cloud includes: Based on the pre-calibrated camera intrinsic parameter matrix K and depth value Z, the 3D coordinates corresponding to each pixel are calculated using the inverse projection formula: ； in, Focal length Principal point coordinates; For each pixel in the depth map; The three-dimensional coordinates of each pixel are aggregated to obtain the final three-dimensional point cloud.

9. A positioning device, implemented based on the positioning method according to any one of claims 1-8, characterized in that, include: The data acquisition module is used to collect the original depth map of the target object and its surrounding scene; The target object is an object whose surface does not have significant features; The data preprocessing module preprocesses the original depth map, performs depth gradient continuity analysis, removes discrete noise points from the original depth map, and obtains an effective depth map. The depth map completion module, based on the effective depth map, restores the complete outline of the target object through gradient direction inference and interpolation to obtain the completed depth map; The target extraction module extracts the target object by performing gradient feature-based edge extraction based on the completed depth map and the target object. This yields a single-pixel-level accurate edge of the target object. The point cloud correction module converts the extracted target object into a three-dimensional point cloud and performs noise reduction and correction on the point cloud. The target localization module calculates the optimal grasping target point of the target object based on the calibrated point cloud computing, and uses this as the robot's grasping target point.

Images