UHPC guardrail hole non-penetration three-dimensional positioning and grabbing method and system

Abstract

This invention belongs to the field of industrial robot visual perception and intelligent control technology, specifically involving a method and system for non-penetrating 3D positioning and grasping of UHPC guardrail holes. It limits all sampling points to the guardrail surface outside the fitted ellipse, fundamentally eliminating penetration errors and eliminating the need for a learning-based depth completion module. The perception model layer uses a lightweight YOLOv11n-PMS-seg instance segmentation module, integrating a PPLCNet backbone network, a multi-scale attention module, and a Shape-IoU loss function, improving the segmentation accuracy of slender elliptical targets while reducing computational load. The control layer adopts a distance-adaptive CAN bus transmission scheme, combined with Kalman filtering and a weighted moving average coordinate smoothing mechanism, to achieve stable, low-latency robot grasping control. The system has been verified through outdoor multi-condition testing, demonstrating stability over a wide temperature range and under complex lighting conditions, possessing industrial-grade reliability and year-round outdoor deployment capability.

Description

Technical Field

[0001] This invention belongs to the field of industrial robot visual perception and intelligent control technology, specifically relating to a method and system for three-dimensional positioning and grasping of UHPC guardrail holes without penetration. Background Technology

[0002] Highway guardrail installation is a typical process in road construction and maintenance that is labor-intensive and highly dangerous. With the improvement of industrial automation, robot-assisted guardrail installation has become an important development direction. Modern robot solutions must be able to accurately grasp structural components in unstructured outdoor environments—including harsh conditions such as drastic changes in lighting and low temperatures. The core prerequisite for this is to accurately obtain the three-dimensional pose of the guardrail's central hole.

[0003] However, the central hole of the UHPC guardrail is a hollow elliptical structure of 300 mm × 100 mm. Its inner wall typically lacks an effective reflective surface, causing conventional RGB-D sensors to mistake the background distance within the hole for the target surface depth. This results in a systematic penetration error on the order of several meters, rendering traditional depth sampling methods completely ineffective. This problem is fundamentally different in cause from the depth loss problem in transparent / reflective scenes such as glass and mirrors. Existing depth completion methods are not designed for the inherent background penetration error of such hollow structures and are therefore difficult to apply directly.

[0004] While deep learning-based object detection technologies (such as the YOLO series) have made significant progress in object perception, existing methods have the following shortcomings: First, standard instance segmentation models are not accurate enough for segmenting slender elliptical targets with an aspect ratio of approximately 3:1, and traditional bounding box loss functions (such as GIoU and CIoU) are difficult to effectively constrain elliptical shape deviations; Second, general-purpose backbone networks introduce unnecessary computational overhead on edge industrial control hardware, making it difficult to meet real-time deployment requirements; Third, existing perception solutions generally lack a complete system architecture that effectively integrates robust visual perception with industrial-grade real-time control.

[0005] Therefore, there is an urgent need for a perception-control integrated system specifically designed for the three-dimensional positioning of holes in hollow UHPC guardrails. This system should be able to fundamentally eliminate penetration errors and achieve industrial-grade precision hole positioning and robot grasping control without relying on a learning-based depth completion module. Summary of the Invention

[0006] To address the technical problems existing in the current robot-assisted guardrail installation operations, this invention provides a UHPC guardrail hole non-penetrating 3D positioning and grasping method and system. It constructs a three-layer hierarchical architecture covering the perception layer, coordinate layer, and control layer, aiming to solve the depth perception failure problem of hollow elliptical industrial targets and achieve high-precision real-time guardrail hole grasping.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A method for non-penetrating three-dimensional positioning and grasping of UHPC guardrail holes includes the following steps: S1. Acquire synchronized color and depth images of the target area using an RGB-D depth camera; S2. Use an instance segmentation model to perform pixel-level segmentation on the color image to obtain the accurate mask of the target entity region; S3. Perform morphological processing and mask inversion on the segmentation mask, and extract the center pixel coordinates of the holes using the least squares elliptic fitting algorithm. and the major axis direction angle θ; S4. Based on the fitted ellipse parameters, generate at least two concentric sampling rings on its outer side, with sampling points evenly distributed in each layer. All sampling points are located on the surface of the target entity to eliminate the penetration error caused by the hollow structure. S5. Perform validity filtering, statistical outlier removal, and mean trimming on the depth values ​​of each sampling point, and apply thickness compensation to obtain the accurate hole center depth value. S6. Combine the camera intrinsic parameter matrix to backproject the center pixel coordinates of the hole and the precise depth value into three-dimensional camera coordinates; apply Kalman filtering and multi-frame weighted moving average smoothing to the obtained coordinates in sequence, and perform amplitude limiting processing on the inter-frame coordinate changes to eliminate the impulse anomalies caused by abrupt changes in depth measurement, and output a stable three-dimensional coordinate sequence.

[0008] S7. The smoothed three-dimensional coordinates and orientation angles are encoded into standard CAN communication frames. A distance adaptive threshold strategy is used to control the transmission frequency. Frame transmission is triggered only when the change in any dimension exceeds the corresponding interval threshold. The position information frame and control command frame are transmitted to the robot controller via the CAN bus to complete precise grasping.

[0009] The YOLOv11n-PMS-seg model described in step S2 consists of the following three parts: PPLCNet backbone network: It adopts depthwise separable convolution, SE attention module, H-Swish activation function and large kernel convolution to replace the default backbone network of YOLOv11n, which reduces the amount of computation and suppresses the false activation of background texture on the surface of metal railing. Multi-scale attention module (MSAM): Inserts the feature pyramid P4 layer to adaptively aggregate multi-scale contextual information, solving the scale variation problem of large pixel regions for nearby holes and small pixel regions for distant holes; Shape-IoU loss function: Independently penalizes the normalized width and height deviations, providing a stronger shape constraint signal for slender elliptical targets with an aspect ratio of approximately 3:1.

[0010] The scaling factor of the sampling loop in step S4 is: in: The radial expansion distance ranges from 5 to 30 pixels, where w and h are the minor and major axes of the fitted ellipse, respectively. The adaptive scale ensures that the guardrail targets at different distances are expanded proportionally. The number of sampling rings L ranges from 2 to 5 layers, and the number of sampling points N per layer ranges from 16 to 32.

[0011] The thickness compensation formula in step S5 is: in: To trim the mean depth, The thickness of the guardrail is measured in ranges from 20 to 150 mm, and the compensation amount is... This eliminates the systematic depth underestimation error caused by the sampling point being located on the outer surface of the guardrail.

[0012] The state vector of the Kalman filter in step S6 is Process noise and The measurement noise R is configured according to the target motion characteristics and the sensor accuracy. After Kalman filtering, a multi-frame weighted moving average is superimposed with a smoothing window size of 3 to 16 frames. The inter-frame coordinate changes are limited to eliminate shock anomalies caused by abrupt changes in depth estimation.

[0013] In step S7, the robot controller communication adopts a distance adaptive threshold strategy, which divides the workspace into several intervals according to the target distance, and independently configures the horizontal coordinate change threshold, depth coordinate change threshold, and orientation angle change threshold for each interval; a communication frame is sent only when the change in any one dimension exceeds the threshold of the corresponding interval, so as to reduce the bus load; when a new target ID is detected, an initial frame is sent unconditionally to register the target.

[0014] The workspace is divided into three zones, with the following thresholds for each zone: proximal zone 0-800 mm; lateral zone threshold. =5 mm, depth threshold = 8 mm, angle threshold θ = 1.0°; Mid-range 800-1500 mm: Lateral threshold = 8 mm, depth threshold = 10 mm, angle threshold θ = 2.0°; distal end 1500-2000 mm: lateral threshold = 12 mm, depth threshold = 15 mm, angle threshold θ = 3.0°.

[0015] The method for extracting the center pixel coordinates of the hole and the major axis direction angle θ of the ellipse using the least squares ellipse fitting algorithm in step S3 is as follows: S3.1. Perform 7×7 morphological elliptic kernel noise reduction on the original mask, perform mask inversion to isolate hollow areas, and perform least-squares elliptic fitting on the internal contour to obtain the pixel coordinates of the hole center. and the direction angle θ of the major axis of the ellipse; S3.2. Use geometric prior constraints to filter out interfering candidate regions: the aspect ratio AR is limited to [2.5, 4.5], and the circularity is limited to [0.45, 0.82]; when extreme occlusion causes ellipse fitting to fail, a centroid backoff mechanism is triggered to maintain tracking stability. S3.3, Using the optimally fitted ellipse center As a pixel-level reference point, combined with the precise depth estimation described in step S5 3D back projection is performed using the camera intrinsic parameter matrix K: For Intel RealSense D415, px, , , Resolution, accuracy within 800 mm. - )mm, and even further distances are - mm.

[0016] A UHPC guardrail hole non-penetrating three-dimensional positioning and grasping system includes a perception layer, a coordinate layer and a control layer; The perception layer includes an RGB-D depth camera for acquiring synchronized color and depth images; it deploys an instance segmentation model to perform pixel-level segmentation of the color images to obtain accurate masks of the guardrail entity regions; and it maintains cross-frame target IDs through a multi-target tracking algorithm. Coordinate Layer: Morphological processing and mask inversion are performed on the segmentation mask. The center pixel coordinates of the holes are extracted using an ellipse fitting algorithm, and the major axis direction angle θ of the ellipse is extracted for attitude orientation. Multi-layer concentric sampling rings are generated outside the fitted ellipse, with all sampling points located on the surface of the guardrail. The depth values ​​of the sampling points are filtered for validity, outliers are removed, and the mean is trimmed. Thickness compensation is applied to obtain accurate hole depth. The pixel coordinates and depth values ​​are back-projected into 3D camera coordinates using the camera intrinsic parameter matrix, and coordinate smoothing is performed using Kalman filtering and multi-frame weighted moving average. Control layer: The smoothed 3D coordinates and orientation angles are encoded into standard communication frames and transmitted to the robot controller via an industrial bus; an adaptive coordinate change threshold strategy based on distance intervals is used to control the transmission frequency, reducing bus load; and a human-machine interface monitoring panel is provided to realize real-time visualization of the system's operating status.

[0017] The perception layer also includes a tracking ID recovery mechanism, which can automatically recover the target ID during the process of the target being briefly occluded or the robotic arm repeatedly moving forward and backward, ensuring that the control layer receives a continuous and stable coordinate data stream. The control layer defines two types of CAN frames: Position Information Frame ID: 0x200, which encodes the X, Y, Z coordinates and orientation angles using big-endian signed 16-bit integers; Control Command Frame ID: 0x210, which encodes grab, place, emergency stop trigger, and emergency stop release operations. When an emergency stop signal is detected, coordinate transmission is immediately suspended and only resumed after receiving an explicit release command, ensuring fail-safe behavior consistent with industrial safety requirements.

[0018] Compared with the prior art, the beneficial effects of this invention are: 1. The edge-outside multi-layer depth sampling strategy proposed in this invention achieves a frame-level depth sampling success rate of 99.4% with a redundancy mechanism of 72 sampling points, without the need for a complex depth completion network, and has high computational efficiency; 2. The YOLOv11n-PMS-seg model of this invention achieves 73.2% mAP@0.5:0.95 with an extremely low computational cost of 8.4 GFLOPs, which is superior to the baseline models of the same series in both accuracy and efficiency; 3. The complete system of this invention has a comprehensive positioning error of 17.8±4.2 mm at a standard operating distance of 1250 mm. After 200 outdoor tests, the success rate of capture is 97.0%. All failures are due to hardware abnormalities rather than algorithm defects. 4. The system of the present invention exhibits stability in a temperature range of -5°C to 20°C and under different lighting conditions, and is capable of year-round outdoor industrial deployment. Attached Figure Description

[0019] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0020] The structures, proportions, sizes, etc. illustrated in this specification are only for the purpose of assisting those skilled in the art in understanding and reading the content disclosed herein, and are not intended to limit the conditions under which the present invention can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size, without affecting the effects and objectives that the present invention can produce, should still fall within the scope of the technical content disclosed in the present invention.

[0021] Figure 1 This is a schematic diagram of the three-layer hierarchical architecture of the intelligent grasping system of the present invention.

[0022] Figure 2 This is the real-time statistical display interface of the HMI monitoring panel of the present invention.

[0023] Figure 3 A schematic diagram of the improved network structure for YOLOv11n-PMS-seg.

[0024] Figure 4 Representative detection results of hole center extraction and three-dimensional coordinate mapping at different distances are presented.

[0025] Figure 5 The diagram shows a multi-layer depth sampling strategy outside the edge, where (a) is a comparison of penetration error, (b) is a spatial distribution diagram of 72 sampling points, and (c) is an example of actual detection results.

[0026] Figure 6 The convergence curve of YOLOv11n-PMS-seg over 200 training epochs.

[0027] Figure 7 The image shows a comparison of instance segmentation results for the four models in representative scenarios.

[0028] Figure 8 This is a schematic diagram showing the installation location of the depth camera on the forklift's gripping mechanism.

[0029] Figure 9 The cumulative capture success rate curve is shown in 200 trials. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. These descriptions are only for further illustrating the features and advantages of the present invention, and not for limiting the claims of the present invention. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0031] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0032] I. System Overall Architecture The intelligent grasping system proposed in this invention adopts a three-layer hierarchical architecture, such as... Figure 1 As shown, these are the perception layer, coordinate layer, and control layer, respectively, ensuring high-precision performance in unstructured outdoor environments.

[0033] The perception layer uses an Intel RealSense D415 camera to capture synchronous RGB-D data streams, such as... Figure 8 As shown, the YOLOv11n-PMS-seg model is deployed for pixel-level instance segmentation, and the tracking ID recovery mechanism based on OC-SORT and ByteTrack is used to achieve cross-frame target maintenance.

[0034] The coordinate layer converts two-dimensional pixel spatial data into accurate three-dimensional camera coordinates: first, ellipse fitting is performed on the region after the segmentation mask is inverted to locate the center of the hole, then reliable depth estimation is obtained through multi-layer sampling outside the edge, and finally, measurement jitter is eliminated by Kalman filtering and eight-frame weighted moving average.

[0035] The control layer converts the smoothed 3D coordinates and orientation angles into motion commands, which are transmitted to the robot controller via the CAN 2.0B bus at 250kbps. A distance-adaptive threshold strategy manages the transmission frequency, reducing bus load while maintaining positioning responsiveness. Figure 2 As shown, the HMI panel developed based on PyQt5 provides real-time monitoring.

[0036] II. Hardware Configuration The main hardware configuration of the system of this invention is shown in Table 1.

[0037] Table 1 System Hardware Configuration Table III. YOLOv11n-PMS-seg model Standard segmentation models are insufficient for handling slender elliptical targets with an aspect ratio of approximately 3:1, and general-purpose backbone networks introduce unnecessary computational overhead when deployed at edges. To address this, YOLOv11n-PMS-seg employs a feature optimization-attention enhancement-shape constraint design chain, such as... Figure 3 As shown.

[0038] (1) The PPLCNet backbone network replaces the default YOLOv11n backbone. Through depthwise separable convolution, SE attention module, H-Swish activation function and large kernel convolution, it maximizes inference efficiency on industrial onboard hardware and effectively suppresses the activation of background texture on the surface of metal guardrail.

[0039] (2) The Multi-Scale Attention Module (MSAM) is inserted into the P4 layer of the feature pyramid to adaptively aggregate multi-scale context information and directly solve the problem of scale variation where the central hole of the guardrail occupies a large pixel area at close range and a small pixel proportion at far range.

[0040] (3) The Shape-IoU loss function independently penalizes the normalized width and height deviations, providing a stronger shape constraint signal for the detection of slender elliptical targets and making up for the deficiency of CIoU loss in shape sensitivity on high aspect ratio targets.

[0041] Compared to the YOLOv11n baseline, this module reduces the number of parameters by 7.8% and GFLOPs by 10.7%, while improving mAP@0.5:0.95 by 3.3 percentage points, making it suitable for real-time edge deployment. Figure 6 The figure shows the convergence curve of YOLOv11n-PMS-seg over 200 training epochs. Figure 7 The image shows a comparison of instance segmentation results for four models in representative scenarios. IV. Hole Center Extraction and Coordinate Mapping Using the segmentation mask generated by YOLOv11n-PMS-seg, the system extracts accurate 3D coordinates according to the following process: Step 1: Perform 7×7 morphological elliptic kernel noise reduction on the original mask, perform mask inversion to isolate hollow areas, and perform least-squares elliptic fitting on the internal contour to obtain the pixel coordinates of the hole center. and the direction angle θ of the major axis of the ellipse.

[0042] Step 2: Use geometric prior constraints to filter out interfering candidate regions: the aspect ratio AR is limited to [2.5, 4.5], and the circularity is limited to [0.45, 0.82]; when extreme occlusion causes ellipse fitting to fail, the centroid backoff mechanism is triggered to maintain tracking stability.

[0043] Step 3: Find the optimal fit for the ellipse center As a pixel-level reference point, combined with the precise depth estimation described in step S5 3D back projection is performed using the camera intrinsic parameter matrix K: For Intel RealSense D415 ( px, , , Resolution), with an accuracy within 800 mm. - )mm, and even further distances are - mm. Representative test results are as follows: Figure 4 As shown.

[0044] V. Multi-layer depth sampling strategy outside the edge like Figure 5 As shown, directly sampling depth values ​​within the hole area will capture the background distance due to the penetration effect of the hollow structure, introducing an error on the order of several meters. This invention proposes an edge-external multi-layer depth sampling strategy, which limits all sampling points to the surface of the guardrail entity outside the fitted ellipse.

[0045] Sampling point generation: For the fitted ellipse Generate L=3 concentric sampling loops, with the scale factor of the k-th layer being: in = 15 pixels is the radial expansion distance. N=24 sampling points are uniformly distributed in each layer, and the parameterization equation is: in , There are a total of 72 sampling points across the three layers. The representative ellipse has three scale factors of approximately 1.10, 1.20, and 1.30.

[0046] Depth processing: Perform validity filtering on the depth value of each sampling point sequentially (retaining depths within the range of 0.2-5.0m). Statistical outlier removal and trimming mean calculation. Then, thickness compensation is applied as follows: for mm guardrail, compensation amount mm. Experimental verification shows that the measured depth is systematically underestimated by 20-30 mm without compensation.

[0047] Coordinate smoothing: Kalman filter state vector is The parameters are , , An eight-frame weighted moving average was superimposed, and the inter-frame coordinate change was limited to 50 mm to suppress shock anomalies.

[0048] Sampling configuration selection: By comparing the ablation experiments of five schemes, namely 2L×24N, 3L×16N, 3L×24N, 4L×16N and 4L×24N, the 3-layer × 24-point configuration (72 points) was determined to be the optimal trade-off point, with a depth confidence rate of 99.4% and a single-frame inference time of 16.8 ms, as shown in Table 2.

[0049] Table 2 Comparison of Depth Sampling Configurations VI. CAN Bus Intelligent Transmission Strategy To reduce the communication load on the CAN bus, an intelligent transmission strategy based on change detection is designed. The minimum inter-frame interval is 50ms, and the workspace is divided into three intervals based on distance, each configured with an independent coordinate change threshold, as shown in Table 3. A CAN frame is sent only when the change in at least one dimension exceeds the corresponding interval threshold. When a new target ID appears for the first time, an initial frame is unconditionally sent to register the target.

[0050] The system communicates with the transport robot controller via the EMUC-B202 USB-CAN adapter using the 250 kbps CAN 2.0B protocol. Two types of frames are defined: position information frames (ID: 0x200) encode the X, Y, and Z coordinates (mm) and orientation angle (0.1° / LSB) using big-endian signed 16-bit integers; control command frames (ID: 0x210) encode grasping, placing, emergency stop triggering, and emergency stop release operations. A dedicated receive thread listens for input frames at 20 Hz to match the fixed control cycle of the industrial controller and reduce IPC interrupt overhead.

[0051] Table 3 Distance Adaptive Threshold Configuration Table Example 2: Explanation of Method Universality and Parameter Adaptation The core principle of the edge-outside multi-layer depth sampling method proposed in this invention lies in limiting all sampling points to the target entity surface outside the fitted ellipse, fundamentally eliminating the penetration error caused by the hollow structure. This principle does not depend on the specific appearance or size of the UHPC guardrail. Any industrial target with a hollow structure that causes penetration error in the RGB-D sensor can be adapted to the method of this invention by adjusting the corresponding parameters. Taking UHPC guardrail holes of different specifications as examples: when the guardrail hole size is 200 mm × 80 mm (height-to-width ratio 2.5:1), and the guardrail thickness is... When the aspect ratio is 80 mm, it is only necessary to adjust the aspect ratio range in the geometric prior constraints from [2.5, 4.5] to [2.0, 4.0], the thickness compensation from 25 mm to 40 mm, and the radial extension distance. The adaptation can be completed by adjusting the pixel size from 15 pixels to 12 pixels while keeping the rest of the model structure and control layer configuration unchanged.

[0052] The method of this invention can also be extended to other types of hollow industrial targets. For example, precast concrete pipe pile end holes (circular hollow, approximately 300–500 mm in diameter) and industrial pipe flange holes (circular or elliptical hollow) all suffer from the same penetration error problem as UHPC guardrail holes. For such targets, the parameters that need to be adjusted include: the aspect ratio range and roundness range in the geometric prior constraints (to adapt to the target shape), and the thickness compensation amount for ellipse fitting. (Adapted to target wall thickness), radial expansion distance (Adapting to target pixel size). The YOLOv11n-PMS-seg model can be fine-tuned by re-acquiring data for new targets, or replaced with other instance segmentation models, without requiring changes to the coordinate layer and control layer architecture. This embodiment illustrates that the method of the present invention has good versatility and can be adapted to hollow industrial targets of different shapes, sizes, and wall thicknesses through parameter adjustment, possessing broad application and promotion value.

[0053] VII. Experimental Verification A custom dataset was constructed by collecting 1561 images in an industrial logistics environment, covering diverse working conditions from 400-3000 mm distance, ±45° viewing angle, and strong direct sunlight to artificial lighting at night. The dataset was trained for 200 epochs on the AutoDL cloud platform (NVIDIA RTX 4090) using PyTorch 2.4.1, achieving a validation set mAP@0.5:0.95 of 73.2%.

[0054] The system is deployed on an industrial vehicle-mounted computer platform, with an Intel RealSense D415 installed in the forklift's gripping mechanism. The frame rate is stable at 29.5-30 fps, with a single-frame processing latency of approximately 33 ms and GPU utilization of approximately 17%. In outdoor winter conditions ranging from -5°C to 5°C, 200 gripping tests were performed at a standard operating distance of 1250 mm, achieving a comprehensive gripping success rate of 97.0% (194 successful grips out of 200). The X-axis error was 9.2±2.8 mm, the Y-axis error was 8.5±2.6 mm, the Z-axis error was 12.4±3.2 mm, and the angle error was ±0.5°. Further verification under strong sunlight conditions at approximately 20°C in summer showed a success rate of no less than 97.0%, demonstrating the system's year-round outdoor deployment capability. Figure 9 The figure shows the cumulative capture success rate curve across 200 trials.

[0055] All six failures were due to hardware anomalies (two CAN bus communication timeouts and four instances of thermal throttling after prolonged operation of the industrial control computer). No failures were attributed to defects in the vision algorithm, thus verifying the industrial-grade reliability of the proposed method.

[0056] The above description only illustrates the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention, and all such changes should be included within the protection scope of the present invention.

Claims

1. A UHPC guardrail hole non-penetration three-dimensional positioning and grabbing method, characterized in that: Includes the following steps: S1. Acquire synchronized color and depth images of the target area using an RGB-D depth camera; S2. Use an instance segmentation model to perform pixel-level segmentation on the color image to obtain the accurate mask of the target entity region; S3, morphological processing is carried out on the segmentation mask and the mask is inverted, and a least square ellipse fitting algorithm is used to extract the hole center pixel coordinates and the long axis direction angle θ; S4. Based on the fitted ellipse parameters, generate at least two concentric sampling rings on its outer side, with sampling points evenly distributed in each layer. All sampling points are located on the surface of the target entity to eliminate the penetration error caused by the hollow structure. S5. Perform validity filtering, statistical outlier removal, and mean trimming on the depth values ​​of each sampling point, and apply thickness compensation to obtain the accurate hole center depth value. S6. Combine the camera intrinsic parameter matrix to back-project the center pixel coordinates of the hole and the precise depth value into three-dimensional camera coordinates; apply Kalman filtering and multi-frame weighted moving average smoothing to the obtained coordinates in sequence, and perform amplitude limiting processing on the inter-frame coordinate changes to eliminate the impulse anomaly values ​​caused by the sudden change in depth measurement, and output a stable three-dimensional coordinate sequence. S7. The smoothed three-dimensional coordinates and orientation angles are encoded into standard CAN communication frames. A distance adaptive threshold strategy is used to control the transmission frequency. Frame transmission is triggered only when the change in any dimension exceeds the corresponding interval threshold. The position information frame and control command frame are transmitted to the robot controller via the CAN bus to complete precise grasping.

2. The method of claim 1, wherein the method is a method of non-penetrating three-dimensional positioning and gripping of a UHPC guardrail hole. The YOLOv11n-PMS-seg model described in step S2 consists of the following three parts: PPLCNet backbone network: It adopts depthwise separable convolution, SE attention module, H-Swish activation function and large kernel convolution to replace the default backbone network of YOLOv11n, which reduces the amount of computation and suppresses the false activation of background texture on the surface of metal railing. Multi-scale attention module (MSAM): Inserts the feature pyramid P4 layer to adaptively aggregate multi-scale contextual information, solving the scale variation problem of large pixel regions for nearby holes and small pixel regions for distant holes; Shape-IoU loss function: Independently penalizes the normalized width and height deviations, providing a stronger shape constraint signal for slender elliptical targets with an aspect ratio of approximately 3:

1.

3. The method of claim 1, wherein the method is a method of non-penetrating three-dimensional positioning and gripping of a UHPC guardrail hole. The scaling factor of the sampling loop in step S4 is: wherein: is the radial expansion distance, the value range is 5-30 pixels, w and h are the short axis and long axis of the fitted ellipse respectively, and the adaptive scale ensures the equal proportion expansion of guardrail targets at different distances; the value range of the sampling ring number L is 2-5 layers, and the value range of the sampling point number N in each layer is 16-32.

4. The method for non-penetrating three-dimensional positioning and grasping of UHPC guardrail holes according to claim 1, characterized in that: The thickness compensation formula in step S5 is: in: To trim the mean depth, The thickness of the guardrail is measured in ranges from 20 to 150 mm, and the compensation amount is... This eliminates the systematic depth underestimation error caused by the sampling point being located on the outer surface of the guardrail.

5. The method for non-penetrating three-dimensional positioning and grasping of UHPC guardrail holes according to claim 1, characterized in that: The state vector of the Kalman filter in step S6 is Process noise and The measurement noise R is configured according to the target motion characteristics and the sensor accuracy. After Kalman filtering, a multi-frame weighted moving average is superimposed with a smoothing window size of 3 to 16 frames. The inter-frame coordinate changes are limited to eliminate shock anomalies caused by abrupt changes in depth estimation.

6. The method for non-penetrating three-dimensional positioning and grasping of UHPC guardrail holes according to claim 1, characterized in that: In step S7, the robot controller communication adopts a distance adaptive threshold strategy, which divides the workspace into several intervals according to the target distance, and independently configures the horizontal coordinate change threshold, depth coordinate change threshold, and orientation angle change threshold for each interval; a communication frame is sent only when the change in any one dimension exceeds the threshold of the corresponding interval, so as to reduce the bus load; when a new target ID is detected, an initial frame is sent unconditionally to register the target.

7. The method for non-penetrating three-dimensional positioning and grasping of UHPC guardrail holes according to claim 6, characterized in that: The workspace is divided into three zones, with the following thresholds for each zone: proximal zone 0-800 mm; lateral zone threshold. = 5 mm, depth threshold = 8 mm, angle threshold θ = 1.0°; Mid-range 800-1500 mm: Lateral threshold = 8 mm, depth threshold = 10 mm, angle threshold θ = 2.0°; distal end 1500-2000 mm: lateral threshold = 12 mm, depth threshold = 15 mm, angle threshold θ = 3.0°.

8. The method for non-penetrating three-dimensional positioning and grasping of UHPC guardrail holes according to claim 1, characterized in that: The method for extracting the center pixel coordinates and major axis direction angle θ of the hole using the least squares elliptic fitting algorithm in step S3 is as follows: S3.

1. Perform 7×7 morphological elliptic kernel noise reduction on the original mask, perform mask inversion to isolate hollow areas, and perform least-squares elliptic fitting on the internal contour to obtain the pixel coordinates of the hole center. and the direction angle θ of the major axis of the ellipse; S3.

2. Use geometric prior constraints to filter out interfering candidate regions: the aspect ratio AR is limited to [2.5, 4.5], and the circularity is limited to [0.45, 0.82]; when extreme occlusion causes ellipse fitting to fail, a centroid backoff mechanism is triggered to maintain tracking stability. S3.3, Using the optimally fitted ellipse center As a pixel-level reference point, combined with the precise depth estimation described in step S5 3D back projection is performed using the camera intrinsic parameter matrix K: For Intel RealSense D415, px, , , Resolution, accuracy within 800 mm. - )mm, and even further distances are - mm.

9. A UHPC guardrail hole non-penetrating three-dimensional positioning and grasping system, characterized in that: It includes a perception layer, a coordinate layer, and a control layer; Perception layer: includes an RGB-D depth camera, used to acquire synchronized color and depth images; Deploy an instance segmentation model to perform pixel-level segmentation on color images to obtain accurate masks of guardrail entity regions; and implement cross-frame target ID maintenance through a multi-target tracking algorithm; Coordinate Layer: Morphological processing and mask inversion are performed on the segmentation mask. The center pixel coordinates of the holes are extracted using an ellipse fitting algorithm, and the major axis direction angle θ of the ellipse is extracted for attitude orientation. Multi-layer concentric sampling rings are generated outside the fitted ellipse, with all sampling points located on the surface of the guardrail. The depth values ​​of the sampling points are filtered for validity, outliers are removed, and the mean is trimmed. Thickness compensation is applied to obtain accurate hole depth. The pixel coordinates and depth values ​​are back-projected into 3D camera coordinates using the camera intrinsic parameter matrix, and coordinate smoothing is performed using Kalman filtering and multi-frame weighted moving average. Control layer: The smoothed 3D coordinates and orientation angles are encoded into standard communication frames and transmitted to the robot controller via an industrial bus; an adaptive coordinate change threshold strategy based on distance intervals is used to control the transmission frequency, reducing bus load; and a human-machine interface monitoring panel is provided to realize real-time visualization of the system's operating status.

10. A UHPC guardrail hole non-penetrating three-dimensional positioning and grasping system according to claim 9, characterized in that: The perception layer also includes a tracking ID recovery mechanism, which can automatically recover the target ID during the process of the target being briefly occluded or the robotic arm repeatedly moving forward and backward, ensuring that the control layer receives a continuous and stable coordinate data stream. The control layer defines two types of CAN frames: Position Information Frame ID: 0x200, which encodes the X, Y, Z coordinates and orientation angles using big-endian signed 16-bit integers; Control Command Frame ID: 0x210, which encodes grab, place, emergency stop trigger, and emergency stop release operations. When an emergency stop signal is detected, coordinate transmission is immediately suspended and only resumed after receiving an explicit release command, ensuring fail-safe behavior consistent with industrial safety requirements.

Images