Attitude sensing and self-adaptive grabbing method and system for centrifugal tube connected with cover

Abstract

The invention discloses a cover-connected centrifugal tube posture sensing and self-adaptive grabbing method and system. The method comprises the steps that a transparent cover-connected centrifugal tube is imaged, 2.5 D texture images and 3D point cloud data are processed in a fusion mode, combined characteristics of a tube body and a bottle cover are extracted and decoupled, and accurate posture information of the centrifugal tube is calculated; and according to the calculated pose information, the grabbing point position of the mechanical arm and the opening and closing degree of the gripper are planned, the hybrid-driven self-adaptive gripper is controlled to execute the grabbing action, and multi-dimensional stability detection and adjustment are conducted in the grabbing process and after grabbing. Anti-interference 2.5 D-3D fusion high-precision attitude sensing is performed on the transparent cover-connected centrifugal tube, intelligent planning and self-adaptive grabbing control are performed on the basis of a sensing result, stable and accurate automatic operation on a complex optical characteristic target is realized, the grabbing success rate and efficiency are remarkably improved, the reliability is high, and the method is suitable for large-scale popularization and application. And the method has the potential of being popularized to capture scenes from other transparent objects.

Description

Technical Field

[0001] This invention relates to the field of automated grasping technology, and more specifically, to a method and system for attitude sensing and adaptive grasping of centrifuge tubes with caps. Background Technology

[0002] Centrifuge tubes are commonly used laboratory equipment in biomedical and chemical experiments, especially capped centrifuge tubes, which are widely used because they effectively prevent sample contamination and leakage. With the development of experimental automation, the demand for automated centrifuge tube handling is increasing. However, the characteristics of transparent capped centrifuge tubes present several challenges for automated handling.

[0003] Traditional gripping methods often employ mechanical positioning or 2D vision positioning. Mechanical positioning has low accuracy and struggles to adapt to subtle changes in centrifuge tube posture. 2D vision positioning, on the other hand, cannot acquire depth information of the centrifuge tube and is easily affected by ambient light, especially for transparent centrifuge tubes, leading to inaccurate posture recognition. Furthermore, different sizes of transparent centrifuge tubes with caps vary in shape and size. Traditional gripping devices have fixed gripping mechanisms that lack adaptability, making it difficult to stably grip centrifuge tubes of different sizes.

[0004] Currently, existing 3D visual perception methods for transparent objects also have some shortcomings. For example, some methods use structured light 3D imaging technology, but for transparent materials, structured light is prone to refraction and reflection, resulting in missing point cloud data or high noise. Other methods are based on deep learning for pose estimation, but require a large amount of labeled data, and their robustness in complex backgrounds needs to be improved.

[0005] There is currently no effective solution to the above problems. Summary of the Invention

[0006] In response to the aforementioned technical problems in related technologies, this invention proposes a method and system for attitude perception and adaptive grasping of centrifuge tubes with lids. This method overcomes the shortcomings of existing technologies, such as inaccurate attitude perception of transparent centrifuge tubes with lids and lack of adaptive capability in the grasping mechanism. It achieves accurate attitude recognition and stable grasping of transparent centrifuge tubes with lids, improves the efficiency and reliability of automated experimental operations, and overcomes the aforementioned deficiencies of existing technologies.

[0007] To achieve the above-mentioned technical objectives, the technical solution of the present invention is implemented as follows: A method for attitude sensing and adaptive grasping of centrifuge tubes with lids includes the following steps: S1: 3D vision-based pose perception: Imaging of transparent centrifuge tubes with caps, and by fusing 2.5D texture images and 3D point cloud data, extracting and decoupling the joint features of the tube body and cap, and calculating the precise pose information of the centrifuge tubes. S2: Adaptive grasping based on pose information: Based on the calculated pose information, the grasping point and gripper opening degree of the robotic arm are planned, the adaptive gripper driven by the hybrid drive is controlled to perform the grasping action, and multi-dimensional stability detection and adjustment are performed during and after the grasping process.

[0008] Further, step S1 specifically includes: S11: By projecting a structured light pattern modulated by frequency and phase, and combining it with polarization filtering technology, anti-interference multimodal image data is acquired. The multimodal image data includes 3D point clouds acquired by a binocular stereo camera and 2.5D texture images acquired by a high-resolution 2D camera. S12: Based on the material and reflective properties of the centrifuge tube, the light intensity ratio of the multi-zone light source is adaptively adjusted, and pulsed lighting is applied to the tube-cap connection to enhance edge contrast. S13: Perform cross-modal registration and fusion of the 2.5D texture image and the 3D point cloud, and fill in the missing point cloud areas caused by the transparent tube in layers; S14: Extract the edge gradient features of the 2.5D image and the normal vector features of the 3D point cloud through the multi-scale fusion module to generate joint features, and use the feature decoupling attention head to separate the joint features into tube feature channels and bottle cap feature channels. S15: The separated features are processed using a model that combines a baseline feature library with an improved iterative nearest point algorithm to quickly calculate the pose of the centrifuge tube.

[0009] Furthermore, in step S12, the light source pattern mapping function trained by machine learning adaptively outputs the optimal intensity ratio of dome light and strip light according to the ambient light intensity and target material parameters; the duty cycle of the pulsed lighting is 50%, and the pulse frequency is 100Hz.

[0010] Furthermore, in step S14, an improved Soft-NMS algorithm is used to filter the extracted feature points. The improved Soft-NMS algorithm uses a Gaussian decay function to suppress low-confidence feature points and removes occluded feature points determined by the point cloud depth difference.

[0011] Furthermore, in step S15, the improved iterative nearest point algorithm adopts a two-stage strategy of key feature point pre-matching and local fine iteration; the fast solution is accelerated by a heterogeneous computing architecture composed of CPU, GPU and FPGA.

[0012] Further, step S2 specifically includes: S21: Based on the calculated pose information and point cloud data, the optimal grasping point is planned using the particle swarm optimization algorithm, and the target joint angle of the robotic arm and the initial opening degree of the gripper are calculated. S22: Controls the robotic arm to move to the target position and uses an adaptive gripper with a hybrid mode of pneumatic flexible drive and servo motor drive to perform grasping. The gripping force is adjusted in real time and the posture deviation is compensated through pressure closed-loop control. S23: Integrates 3D vision, pressure, and acceleration sensor data to assess grasping stability, and triggers pressure fine-tuning, pose correction, or secondary grasping based on the assessment results.

[0013] Further, in step S21, the initial opening degree Dgrip of the gripper is calculated according to the formula "opening degree = tube diameter D + 2 × safety margin S + pressure compensation amount Δd", where D is the tube diameter fitted by point cloud, S is the safety margin dynamically adjusted according to the tube wall thickness, and Δd is the pressure compensation amount calculated from the target gripping force and the gripper flexibility coefficient.

[0014] A centrifuge tube attitude sensing and adaptive grasping system for implementing the method, characterized in that it comprises: The visual acquisition module is used to acquire multimodal data including 2.5D texture images and 3D point clouds. It includes a binocular stereo camera, a structured light source, a polarization filter component, a high-resolution 2D camera, and a multi-zone light source system. The data processing module is used to fuse, extract features and calculate poses of the multimodal data, and it adopts a heterogeneous architecture of CPU, GPU and FPGA. The grasping execution module, used to perform grasping actions, includes a multi-degree-of-freedom robotic arm and an adaptive gripper mounted at its end, the gripper integrating a pressure sensor; The control module is used to receive the pose calculation results, plan the grasping trajectory and parameters, and control the grasping execution module to complete adaptive grasping and stability adjustment.

[0015] Furthermore, the visual acquisition module also includes an FPGA time synchronization trigger module, used to synchronously control the acquisition actions of the structured light source, the binocular stereo camera, and the high-resolution 2D camera; the multi-zone light source system includes a dome light source and a strip light source.

[0016] Furthermore, the adaptive gripper has a three-finger symmetrical structure and adopts a hybrid drive method combining pneumatic drive and servo motor drive; the control module includes a main controller for decision planning and a motion controller for real-time motion control and pressure closed-loop control.

[0017] The beneficial effects of this invention are as follows: This invention implements anti-interference 2.5D-3D fusion high-precision attitude perception on transparent centrifuge tubes with caps, and performs intelligent planning and adaptive grasping control based on the perception results, realizing stable and accurate automated operation of targets with complex optical characteristics; through dynamic structured light coding, polarization-phase cooperative anti-interference, and material adaptive illumination technology, it effectively overcomes the problems of point cloud loss and noise caused by reflection and refraction of transparent materials; by utilizing particle swarm optimization algorithm, DH parameter correction, and force-position hybrid control of hybrid drive gripper, it achieves adaptive matching of grasping points, opening and closing degrees, and grasping forces for centrifuge tubes of different sizes (10-100mL); the entire system realizes closed-loop automation from perception, decision-making to execution, significantly improving the grasping success rate and efficiency, with high reliability, effectively reducing human intervention, and has the potential to be promoted to other transparent object grasping scenarios. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of the attitude perception method of the attitude perception and adaptive grasping method for centrifuge tubes with caps according to an embodiment of the present invention; Figure 2 This is a flowchart of the adaptive grasping method of the centrifuge tube attitude sensing and adaptive grasping method according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of the centrifuge tube attitude sensing and adaptive grasping system according to an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the scope of protection of the present invention.

[0021] like Figure 1-2 As shown in the embodiment of the present invention, the method for attitude perception and adaptive grasping of centrifuge tubes with caps includes a high-precision attitude perception method based on 2.5D-3D fusion and an adaptive grasping method.

[0022] 1. A high-precision attitude perception method based on 2.5D-3D fusion This method addresses the core problem of inaccurate perception of 2.5D texture and 3D depth information caused by the special material of transparent centrifuge tubes, which is prone to refraction and reflection interference. Through collaborative innovation in five technical aspects, it integrates 2.5D texture contours and 3D depth information to overcome the perception challenge of transparent materials, achieving high-precision and robust attitude perception. The innovative details of each aspect are as follows: (1) Innovation in anti-interference multimodal data acquisition: Dynamic structured light coding technology: To address the issue of phase aliasing caused by multiple refractions in a transparent tube, a structured light source scheme of "frequency-phase joint modulation" is adopted. A dynamic frequency-modulated sub-pattern that adaptively switches from 100 to 800 Hz is superimposed on a 500 Hz basic sinusoidal grating. Different frequency components are separated by Fourier transform. Ultimately, phase aliasing is eliminated, ensuring the basic effectiveness of multimodal data acquisition.

[0023] Polarization-phase coordinated anti-interference: To address the issue of low effective point cloud rate in highly reflective scenarios (reflectivity > 60%), a dual-axis adjustable polarizer (adjustment accuracy 0.1°) is used in conjunction with 2.5D phase deflection technology. By analyzing the phase change at different polarization angles, a reflective interference suppression model is constructed. Ultimately, it can still maintain an effective point cloud rate of over 90% in highly reflective scenarios, improving adaptability to reflective environments.

[0024] Enhanced Spatiotemporal Synchronization: To address the issues of data mismatch and high synchronization response delay when ambient light changes abruptly (change rate > 20% / ms), a dual-mode "event trigger + periodic sampling" approach is introduced into the FPGA time synchronization trigger module. When the light intensity sensor detects a sudden change in ambient light, it automatically switches to event trigger mode. Ultimately, the synchronization response delay is reduced from 10μs to 5μs, avoiding data mismatch under dynamic light interference.

[0025] (2) Innovation of multi-source collaborative strategy: Material-Light Source Mapping Model: To address the issue of mismatched lighting schemes under different tube / cap materials and ambient light intensities, which leads to contour extraction errors, a database of reflective characteristics of tubes and caps is established. The light source pattern mapping function is trained through machine learning. After inputting ambient light intensity and target material parameters, the optimal dome light-strip light intensity ratio is output (adjustment accuracy 5%). Ultimately, material-adaptive lighting is achieved, improving the adaptability of lighting under different materials.

[0026] Zoned light intensity closed-loop control: To address the problem of contour extraction error caused by light intensity drift in traditional open-loop light sources, a DALI digital dimming module is independently configured for each light source zone, and a PID closed-loop control is constructed in combination with the feedback from a light intensity sensor with a sampling frequency of 100Hz; the final light intensity stability is controlled within ±2%, thus solving the light intensity drift problem.

[0027] Dynamic boundary enhancement: To address the issues of weak edge contrast and poor edge detection signal-to-noise ratio at the tube-cap connection, a "pulse-type strip light" illumination mode (50% duty cycle, 100Hz pulse frequency) is designed to enhance edge contrast through an inter-frame difference algorithm; ultimately improving the edge detection signal-to-noise ratio at the tube-cap connection and ensuring the accuracy of edge extraction.

[0028] (3) Innovation in data fusion and preprocessing: Cross-modal feature association mechanism: To address the issues of unclear correspondence between key feature points in 2.5D contours and 3D point clouds and large registration errors, an attention mechanism is introduced on the basis of spatial registration to construct a 2.5D-3D feature association network, which automatically identifies the cross-modal correspondence of key feature points such as the center of the pipe opening and the corner of the bottle cap; the final registration error is reduced from 0.1mm to 0.05mm, improving the accuracy of cross-modal data registration.

[0029] Layered point cloud filling strategy: To address the issue of missing point clouds in transparent tubes affecting the integrity of attitude perception, a layered processing strategy of "coarse filling + fine optimization" is adopted. First, coarse filling is completed through cubic spline interpolation, and then fine optimization is performed based on the geometric symmetry of the tube (cylindricity error < 0.02 mm). After filling, the point cloud density is increased by 40%, and the geometric error is < 0.03 mm, ensuring the integrity of the point cloud data.

[0030] Adaptive noise suppression: To address the issue that traditional filtering methods cannot simultaneously preserve the shape of smooth areas of the tube and the details of textured areas of the bottle cap, a dynamic filtering algorithm based on point cloud curvature is proposed. A large radius filter of 0.5mm is used for smooth areas of the tube to preserve the overall shape, while a small radius filter of 0.2mm is used for textured areas of the bottle cap to preserve detailed features. The final filtered point cloud signal-to-noise ratio is improved by 20%, achieving precise adaptation to the filtering requirements of different areas.

[0031] (4) Innovation in feature extraction: Dual-modal feature fusion extraction: To address the low target discrimination of single-modal features, a multi-scale fusion module of "3 convolutional layers + 2 pooling layers + 1 feature concatenation layer" is designed. The 2.5D image first extracts preliminary features using a 3×3 convolutional kernel (stride 1, padding=1), then calculates edge gradient features using a Laplacian operator (convolution kernel [[-1,-1,-1],[-1,8,-1],[-1,-1,-1]]), outputting a 64-dimensional feature map. The 3D point cloud calculates the neighborhood point set through K-nearest neighbor (K=20) search, and uses PCA to decompose the covariance matrix to obtain normal vector features (3-dimensional), which are then upscaled to 64 dimensions using a 1×1×1 3D convolution. The two are concatenated at the feature concatenation layer to form a 128-dimensional "gradient-normal vector" joint feature. The nonlinear expression is enhanced using BatchNorm and ReLU activation functions, improving target discrimination by 35% compared to single-modal features.

[0032] Tube-Bottle Cap Feature Decoupling: To address the coupling interference between the tube body (cylindrical symmetry features) and the bottle cap (irregular polyhedral features), a dual-channel feature decoupling attention head is introduced. First, a 2×2 mask matrix is ​​constructed, with rows corresponding to the "tube body / bottle cap" target category and columns corresponding to the "shape / texture" feature dimensions. The correlation weights between feature channels are calculated using a self-attention mechanism, and the attention coefficients are normalized using Softmax. The tube body channel is weighted more for shape features such as cylindricity and axial direction, while the bottle cap channel is weighted more for detail features such as edges and texture gradients. After decoupling, the feature matching IoU improves from 0.72 to 0.92, and the matching accuracy improves by 28%.

[0033] Dynamic Feature Filtering and Optimization: To address the issues of numerous pseudo-feature points, high computational cost, and impact on registration efficiency, an improved Soft-NMS algorithm is employed for feature point purification. A Gaussian decay function (σ=0.5) is set to suppress the decay of feature points with a confidence level below 0.6. Feature points in occluded areas (determined by a point cloud depth difference > 2mm) are marked as pseudo-feature points and removed. Simultaneously, a "feature point density constraint" is introduced, retaining feature points with a density ≥ 5 / mm² in critical areas such as pipe openings and bottle cap edges, while reducing the density to 2 / mm² in non-critical areas. This ensures registration accuracy while reducing computational cost. After filtering, the total number of feature points is reduced by 40%, the pseudo-feature point removal rate is ≥ 95%, the number of subsequent ICP registration iterations is further reduced by 15%, and the solution efficiency is improved by 20%.

[0034] (5) Innovation in fast attitude calculation: Dual-branch collaborative solution: To address the issues of insufficient cross-modal feature enhancement and large matching errors between instance masks and target point clouds, a mutual attention layer of "3D semantic features - 2D mask features" is constructed in the Voxel-Net model. The feature extraction branch uses 3 layers of 3D convolutions (3×3×3 kernels, stride 2) + 2 layers of deconvolutions (output resolution 128×128×128) to extract 3D semantic features; the instance segmentation branch generates 256×256 resolution pipe / bottle cap instance masks based on a lightweight Mask R-CNN (backbone using MobileNetV3). The mutual attention layer unifies the feature dimensions of both to 256 dimensions through 1×1 convolutions, calculates the similarity matrix (size 256×256) between feature maps, and distributes attention weights through Softmax normalization to achieve cross-modal feature enhancement. By introducing a “mask-point cloud matching loss” (MSE loss + IoU loss weighted, with a weight ratio of 3:7), the matching error between the instance mask and the target point cloud is reduced from 0.12mm to 0.10mm, a reduction of 15%.

[0035] Dynamic Expansion of the Benchmark Library: To address the issues of poor generalization and long adaptation cycles for new centrifuge tube specifications in traditional fixed benchmark libraries, a coupled benchmark feature library architecture of "base model + incremental fine-tuning" is constructed. The base model uses ResNet50 pre-trained point cloud templates and geometric parameters (tube opening radius, tube length, etc.) for 10 common centrifuge tube specifications (10-100mL). When adding a new specification, only 50 sets of multi-pose point cloud data need to be collected, and fine-tuning is performed through knowledge distillation (the teacher model is the base model, and the student model is the model for the new specification), with a fine-tuning cycle of ≤30min. A "feature similarity threshold" (0.92) is introduced to screen the differences between the new data and the base model, avoiding model overfitting. After expansion, the pose calculation accuracy still remains above 99%, and the generalization performance is improved by 35% compared to the traditional fixed benchmark library.

[0036] Lightweight Registration Acceleration: Addressing the slow registration speed and limited accuracy of traditional ICP algorithms, an improved ICP algorithm employs a two-stage strategy of "key feature pre-matching + local fine-tuning iteration." The first stage extracts 5-8 key feature points, including the center of the pipe opening (using RANSAC to fit a circle, with a center positioning error ≤0.05mm) and the cap vertices (using Harris corner detection, with corner response values ​​>200). Coarse matching is then performed using SIFT feature descriptors (128 dimensions), reducing the initial registration error from 10mm to within 5mm. The second stage uses the coarse matching result as the initial value and employs an improved ICP iteration (20 iterations, convergence threshold 0.001mm), introducing a "distance weight factor" (weights are assigned based on point cloud density, with higher density resulting in greater weight) to optimize the objective function. This improves the registration speed from 15fps to 30fps (a 2x increase) while achieving a registration accuracy of 0.08mm.

[0037] Hardware-Algorithm Co-deployment: Addressing the issues of insufficient real-time performance in attitude calculation and high power consumption in traditional GPU solutions, a heterogeneous computing architecture of "CPU+GPU+FPGA" is adopted to achieve end-to-end acceleration. The FPGA (Xilinx Zynq UltraScale+) handles data preprocessing (point cloud filtering, feature point extraction), compressing processing latency to <0.5ms through hardware pipelined design. The GPU (NVIDIA Jetson AGX Orin) deploys the attitude calculation model, using TensorRT for INT8 quantization (quantization calibration set of 1000 sets of point cloud data), reducing the model size from 256MB to 64MB (a 75% reduction) and achieving an inference speed of 35fps. The CPU (Intel i7-12700K) handles system scheduling and multi-module collaboration, enabling inter-hardware data transfer via the PCIe 4.0 interface (rate 8GB / s). The overall processing speed reaches 30fps, meeting real-time requirements, while power consumption is reduced from 60W in traditional GPU solutions to 36W, a 40% reduction.

[0038] 2. Adaptive crawling method This adaptive gripping method addresses the issues of poor adaptability, easy damage, or slippage when gripping transparent centrifuge tubes with caps of different sizes and orientations. Based on high-precision attitude perception results, it integrates mechanical analysis and intelligent planning strategies to achieve stable gripping. The detailed steps are as follows: (1) Intelligent planning based on captured parameters: Dynamic optimization of gripping points: Addressing the poor adaptability of gripping points to centrifuge tubes of different specifications and postures, which can easily lead to gripping offset or instability, the optimal gripping point is planned using a particle swarm optimization (PSO) algorithm based on the tube axis equation (Ax+By+Cz+D=0), cap center coordinates (X0,Y0,Z0), and point cloud stiffness distribution (the tube wall thickness is inverted from the point cloud density; a density > 20 points / mm² indicates a high-stiffness region). The algorithm is set with 50 particles, 30 iterations, and the fitness function is maximizing "gripping lever arm length × stiffness coefficient." Constraints include an angle ≤ 15° between the gripping point line and the tube axis, ensuring the gripping force line passes through the centrifuge tube's centroid (calculated from the point cloud centroid algorithm: G = ∑(mᵢPᵢ) / ∑mᵢ, where mᵢ is the point cloud weight), and a gripping point offset ≤ 0.2mm. Establish a "specification-grab point distance" mapping table: 20mm from the tube opening for 10mL / 15mL tubes, 30mm from the tube opening for 20mL / 30mL tubes, 40mm from the tube opening for 50mL / 60mL tubes, and 50mm from the tube opening for 80mL / 100mL tubes, to achieve adaptive matching of specifications.

[0039] Pose Parameter Calculation and Compensation: Addressing the issues of large pose calculation errors and low gripper-tube contact when the tube is tilted, a kinematic forward and inverse kinematic model is established based on the robot arm's DH parameter table (link lengths a1-a6, joint angles θ1-θ6, offsets d1-d6, and torsional angles α1-α6). The centrifuge tube's attitude parameters (positions X / Y / Z, tilt angles α / β / γ, and cap orientation θ) are converted into robot arm joint angle parameters using a homogeneous transformation matrix. Link error correction terms (Δaᵢ=±0.01mm, Δαᵢ=±0.05°) are introduced, and the inverse kinematic model is optimized using the least squares method, resulting in a pose calculation error ≤0.05mm. When the tube tilt angle >5°, servo compensation is triggered: the robot arm's end effector rotates around the X / Y axis to the corresponding tilt angle, with the compensation formula Δθ=α×k (k=1.02, with a safety factor of 0.02), ensuring a gripper-tube contact >95%.

[0040] Adaptive calculation of opening degree: To address the issue of mismatch between the gripper's opening degree and the tube's specifications, which can easily lead to tube damage or weak gripping, the RANSAC algorithm is used to fit the tube's cross-sectional circle (fitting error ≤ 0.1mm) to obtain the tube diameter D. Combined with the gripper's flexibility coefficient k = 0.8mm / N (measured by materials mechanics experiments), the initial opening degree is calculated using the formula "Opening degree = D + 2 × Safety margin + Pressure compensation". The safety margin is dynamically adjusted according to the tube wall thickness: 0.5mm for wall thickness < 1mm, and 0.8mm for wall thickness 1-2mm. The pressure compensation is calculated from the target gripping force F (F = 5-8N for transparent tubes, F = 10-12N for bottle caps), i.e., Δd = F × k, achieving a linkage and adaptation between gripping force and opening degree.

[0041] (2) Gripping Mechanism Control: To address the limitations of traditional rigid gripping mechanisms, which cannot adapt to centrifuge tubes of different sizes and are prone to damage or instability, a multi-degree-of-freedom robotic arm is used as the gripping actuator. An adaptive gripper is installed at the end of the robotic arm. Based on the calculated target pose, the motion controller controls the robotic arm to move to the designated position. The adaptive gripper uses a flexible drive and incorporates a pressure sensor to detect the gripping force in real time during the gripping process. When the gripping force reaches a preset threshold, the gripper stops closing, achieving adaptive gripping of centrifuge tubes of different sizes and preventing damage due to excessive gripping force.

[0042] (3) Stability detection and adjustment of the gripping: To address the issue that the centrifuge tube may shift or slip after gripping, and the stability cannot be guaranteed, the posture information of the centrifuge tube is acquired again through a 3D vision acquisition device after gripping to detect whether the centrifuge tube has shifted or slipped. If the posture shift is detected to exceed the allowable range, the position of the robotic arm is adjusted according to the shift amount to ensure stable gripping of the centrifuge tube.

[0043] like Figure 3 As shown in the embodiment of the present invention, the adaptive grasping system for transparent centrifuge tubes with caps based on 3D vision adopts a fully closed-loop architecture of "perception-decision-control-execution". The 3D vision acquisition module, data processing module, grasping execution module, and control system module are interconnected through standardized hardware interfaces and real-time communication protocols to form a complete signal and data stream transmission link. The connection relationship and transmission path between the modules are as follows: 1. 3D Vision Acquisition Module: Composed of a binocular stereo camera (baseline 120mm, resolution 2048×1536), a structured light source (frequency 500Hz, projecting sinusoidal grating), a polarization filter component, a high-resolution 2D camera (4096×3072), a multi-zone light source system (5000lux dome light + 30° strip light), a light intensity sensor (accuracy ±5%), an FPGA time synchronization trigger module (synchronization error ≤10μs), and a PCI-E image acquisition card (transmission rate 1GB / s).

[0044] Connection relationships: ① Directly connected to the FPGA unit of the data processing module via PCI-E 4.0 interface; ② Connected to the main controller of the control system module via RS485 interface; ③ The FPGA time synchronization trigger module achieves timing synchronization with the data processing module and the control system module via 1PPS synchronization signal interface.

[0045] Data Stream / Signal: ① Transmit multimodal raw data such as 2.5D texture images, 3D depth point clouds, and polarization phase data to the data processing module; ② Feed back the ambient light intensity signal detected by the light intensity sensor to the control system module; ③ Receive control commands from the control system module for starting / stopping the light source and adjusting parameters (such as light intensity and frequency).

[0046] 2. Data Processing Module: Adopts a heterogeneous computing architecture of "CPU+GPU+FPGA", containing five functional units: ① Anti-interference data preprocessing unit (implementing polarization-phase collaborative filtering and dynamic light source control); ② Cross-modal fusion unit (completing 2.5D-3D feature association and point cloud filling); ③ Dual-modal feature extraction unit (generating gradient-normal vector joint features); ④ Incremental benchmark library unit (supporting dynamic expansion specifications, including a knowledge distillation fine-tuning module); ⑤ Fast pose calculation unit (deploying a dual-branch Voxel-Net mutual attention model and improving the ICP algorithm to achieve heterogeneous acceleration).

[0047] Connection relationships: ① The PCI-E 4.0 interface is used as the core data interface to receive raw data from the 3D vision acquisition module; ② It communicates bidirectionally with the main controller of the control system module via Ethernet (TCP / IP protocol); ③ The FPGA unit is linked with the synchronization trigger module of the 3D vision acquisition module through the high-speed GPIO interface.

[0048] Data Stream / Signal: ① Receive multimodal raw data from the 3D vision acquisition module; ② Output high-precision attitude perception results (including core parameters such as tube position X / Y / Z, tilt angle α / β / γ, bottle cap orientation θ, and tube diameter) to the control system module; ③ Receive attitude calculation start / stop commands and reference library expansion trigger commands issued by the control system module.

[0049] 3. Grasping Execution Module: Composed of a 6-DOF robotic arm (repeatability ±0.02mm, maximum load 5kg) and a three-finger hybrid drive adaptive gripper. The gripper integrates a 0.1N precision pressure sensor and a 0.01mm resolution displacement sensor, supporting switching between pneumatic flexible drive and precise servo motor drive; the robotic arm joints have built-in DH parameter correction modules.

[0050] Connection relationships: ① The robotic arm is connected to the motion controller of the control system module via the EtherCAT real-time industrial Ethernet bus; ② The pressure sensor and displacement sensor of the gripper are connected to the motion controller via the analog input interface (4-20mA); ③ The joint encoder of the robotic arm feeds back position signals to the motion controller via the RS422 interface.

[0051] Data stream / signals: ① Receive joint angle control commands, gripper opening and closing commands, and drive mode switching commands from the control system module; ② Feed back real-time gripping force signals, gripper displacement signals, and actual posture signals of the robotic arm to the control system module.

[0052] 4. Control System Module: This module includes a motion controller (DSP+FPGA architecture, 1ms control cycle) and a main controller (industrial-grade PLC, 1μs / step processing speed), which is the core of the system's "decision-control" mechanism. The main controller integrates intelligent planning algorithms for grasping parameters (including PSO grasping point optimization and DH parameter correction model); the motion controller implements pressure PID closed-loop control and dynamic attitude compensation.

[0053] Connection relationships: ① The main controller communicates bidirectionally with the data processing module via Ethernet and is linked with the 3D vision acquisition module via an RS485 interface; ② The motion controller is connected to the grasping execution module via an EtherCAT bus and interacts with the main controller via an internal PCI bus; ③ An Ethernet interface is reserved for system status monitoring and parameter configuration.

[0054] Data Flow / Signals: ① Receives attitude perception results from the data processing module and ambient light intensity signals from the 3D vision acquisition module; ② Sends control commands to the data processing module and light source adjustment commands to the 3D vision acquisition module; ③ The main controller outputs planning commands such as gripper opening and closing degree and target joint angle of the robotic arm to the motion controller; ④ The motion controller outputs real-time control commands based on the planning commands and feedback signals from the grasping execution module, and simultaneously feeds back the execution status to the main controller, forming a closed-loop control.

[0055] To facilitate understanding of the above technical solutions of the present invention, the following detailed description of the above technical solutions of the present invention will be provided through specific usage methods.

[0056] 1. System Hardware Setup The 3D vision acquisition module uses a binocular stereo camera with a resolution of 2048×1536 (baseline 120mm), a structured light source with a frequency of 500Hz, a polarization filter component, a 2D camera with a resolution of 4096×3072, a multi-zone light source system with 5000lux dome light + 30° strip light, a light intensity sensor with an accuracy of ±5%, an FPGA time synchronization trigger module (synchronization error ≤10μs), and a PCI-E interface image acquisition card (transmission rate 1GB / s). The gripping execution module uses a 6-DOF robotic arm (repeatability ±0.02mm, maximum load 5kg). The adaptive gripper adopts a three-finger symmetrical structure (hybrid pneumatic flexibility + servo drive, with built-in 0.1N pressure sensor, 0.01mm displacement sensor, and 0.01mm displacement sensor). The control system module's main controller is an industrial-grade PLC (1μs / step), and the motion controller adopts a DSP+FPGA architecture (1ms control cycle).

[0057] 2. Software Algorithm Implementation The software algorithm is designed based on the entire process of "perception-decision-control" and adopts a modular development architecture. The functions and implementation details of each module are as follows: (1) Anti-interference multimodal data acquisition and control module: Based on C++ combined with OpenCV and PCL library, it realizes frequency-phase joint modulation of the nine-vertical intelligent structured light source (500Hz basic grating + 100-800Hz dynamic frequency modulation), and configures "event triggering + periodic sampling" dual-mode synchronization (synchronization error ≤5μs) through FPGA driver module; the polarization filter component is controlled by calling the hardware interface with Python to realize 0.1° precision dual-axis polarization angle adjustment, and with the 2.5D phase deflection algorithm (based on Fourier transform phase unwrapping), it completes anti-interference imaging in strong reflective scene.

[0058] (2) Multi-source collaborative control module: PLC programming combined with LabVIEW host computer development is used to establish a database of tube-cap reflectivity characteristics (store the reflectivity curves of centrifuge tubes and caps under different light intensities), and a lightweight material-light source mapping model is deployed through TensorFlowLite (input ambient light intensity and material type, output light intensity ratio); each light source partition is configured with a DALI digital dimming protocol interface, combined with 100Hz sampling feedback from the light intensity sensor, to construct PID closed-loop control (proportional coefficient Kp=2.5, integral coefficient Ki=0.8, derivative coefficient Kd=0.3), and the light intensity stability is controlled within ±2%; the weak edge enhancement module realizes 100Hz pulsed strip light control and inter-frame difference algorithm.

[0059] (3) Data fusion and preprocessing module: 2.5D image preprocessing is completed based on Python + OpenCV, including 5×5 Gaussian filtering (σ=1.2) and adaptive Canny edge detection (high threshold 80, low threshold 40); 3D point cloud preprocessing uses PCL library to implement statistical filtering (K=50 neighboring points, standard deviation multiple 1.5) and radius filtering (radius 0.5mm, minimum number of points 10); cross-modal registration is carried out by building an attention mechanism feature association network (including 2 fully connected layers + Softmax attention weight allocation) through PyTorch, and the registration error is ≤0.05mm; the point cloud filling module implements cubic spline interpolation coarse filling and geometric symmetry fine optimization (cylindricity error <0.02mm).

[0060] (4) Feature extraction and pose calculation module: The feature extraction part uses PyTorch to build a multi-scale fusion network (3 layers of convolution + 2 layers of pooling), the Laplacian operator extracts 2.5D edge gradient features, PCA calculates 3D point cloud normal vector features and concatenates them into 128-dimensional joint features; the feature decoupling attention head achieves tube-bottle cap feature separation through a 2×2 mask matrix (IoU≥0.92); the pose calculation module is based on TensorFlow to build a dual-branch improved Voxel-Net (backbone is 3DResNet), introduces a mutual attention interaction layer (feature dimension 256), combines an incremental learning benchmark library (supports 50 sets of sample fine-tuning) and an improved ICP algorithm (SIFT pre-matching + 20 local iterations); TensorRT is used for INT8 quantization acceleration (calibration set is 1000 sets of point cloud data), and the inference speed reaches 35fps.

[0061] (5) Adaptive grasping control module: The grasping parameter planning algorithm is developed based on C++. The grasping point is optimized by the particle swarm optimization algorithm (50 particles, 30 iterations, fitness function maximizes grasping arm × stiffness coefficient). Combined with the inverse kinematics solution of the robotic arm with DH parameter correction (error ≤ 0.05mm); The grasping mechanism control adopts the linkage of PLC and motion control card to realize segmented pressure PID control (speed 50mm / s in the rapid approach stage, 5mm / s in the flexible contact stage, and pressure holding stage fluctuation ≤ ±0.02MPa); The stability detection module integrates 3D vision (attitude offset), pressure (distribution uniformity), and acceleration (vibration amplitude ≤ 0.5g) data, and writes graded adjustment logic (fine-tuning / correction / secondary grasping) in Python.

[0062] 3. Experimental Verification Twelve sizes of transparent centrifuge tubes with caps (10mL, 15mL, 20mL, 30mL, 50mL, 60mL, 80mL, 100mL, etc.) were selected, and 1000 grasping experiments were conducted under ambient light conditions of 500-5000 lux, reflectivity of 30%-80%, and tube tilt angle of 0°-15°. The results showed that the posture perception position error was ≤0.08mm, the angle error was ≤0.3°, the dual-target recognition accuracy was 99.2%, and the processing speed was 30fps. The grasping parameter planning time was <0.2s, the grasping point offset was ≤0.2mm, the fit was >95%, the pressure control accuracy was ±0.02MPa, the grasping instability rate was <0.5%, the grasping success rate was over 99.5%, and the average grasping time was 2.0s. Compared with traditional methods, the overall performance is significantly enhanced.

[0063] In summary, by utilizing the technical solutions described above, this invention achieves stable and precise automated operation of targets with complex optical characteristics through anti-interference 2.5D-3D fusion high-precision attitude perception of transparent centrifuge tubes with caps, and intelligent planning and adaptive grasping control based on the perception results. Through dynamic structured light coding, polarization-phase cooperative anti-interference, and material-adaptive illumination technology, it effectively overcomes the problems of point cloud loss and noise caused by reflection and refraction of transparent materials. By employing particle swarm optimization algorithms, DH parameter correction, and force-position hybrid control of the hybrid-driven gripper, it achieves adaptive matching of grasping points, opening and closing degrees, and grasping forces for centrifuge tubes of different sizes (10-100mL). The entire system achieves closed-loop automation from perception and decision-making to execution, significantly improving the grasping success rate and efficiency, exhibiting high reliability, effectively reducing manual intervention, and possessing the potential to be extended to other transparent object grasping scenarios.

[0064] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for attitude sensing and adaptive grasping of a covered centrifuge tube, characterized in that, Includes the following steps: S1: 3D vision-based pose perception: Imaging of transparent centrifuge tubes with caps, and by fusing 2.5D texture images and 3D point cloud data, extracting and decoupling the joint features of the tube body and cap, and calculating the precise pose information of the centrifuge tubes. S2: Adaptive grasping based on pose information: Based on the calculated pose information, the grasping point and gripper opening degree of the robotic arm are planned, the adaptive gripper driven by the hybrid drive is controlled to perform the grasping action, and multi-dimensional stability detection and adjustment are performed during and after the grasping process.

2. The method for attitude sensing and adaptive grasping of centrifuge tubes with lids according to claim 1, characterized in that, Step S1 specifically includes: S11: By projecting a structured light pattern modulated by frequency and phase, and combining it with polarization filtering technology, anti-interference multimodal image data is acquired. The multimodal image data includes 3D point clouds acquired by a binocular stereo camera and 2.5D texture images acquired by a high-resolution 2D camera. S12: Based on the material and reflective properties of the centrifuge tube, the light intensity ratio of the multi-zone light source is adaptively adjusted, and pulsed lighting is applied to the tube-cap connection to enhance edge contrast. S13: Perform cross-modal registration and fusion of the 2.5D texture image and the 3D point cloud, and fill in the missing point cloud areas caused by the transparent tube in layers; S14: Extract the edge gradient features of the 2.5D image and the normal vector features of the 3D point cloud through the multi-scale fusion module to generate joint features, and use the feature decoupling attention head to separate the joint features into tube feature channels and bottle cap feature channels. S15: The separated features are processed using a model that combines a baseline feature library with an improved iterative nearest point algorithm to quickly calculate the pose of the centrifuge tube.

3. The method for attitude sensing and adaptive grasping of centrifuge tubes with lids according to claim 1, characterized in that, In step S12, the light source pattern mapping function trained by machine learning adaptively outputs the optimal intensity ratio of dome light and strip light according to the ambient light intensity and target material parameters; the duty cycle of the pulsed lighting is 50% and the pulse frequency is 100Hz.

4. The method for attitude sensing and adaptive grasping of centrifuge tubes with lids according to claim 1, characterized in that, In step S14, the extracted feature points are filtered using an improved Soft-NMS algorithm. The improved Soft-NMS algorithm uses a Gaussian decay function to suppress low-confidence feature points and removes occluded feature points determined by the point cloud depth difference.

5. The method for attitude sensing and adaptive grasping of centrifuge tubes with caps according to claim 1, characterized in that, In step S15, the improved iterative nearest point algorithm adopts a two-stage strategy of key feature point pre-matching and local fine iteration; the fast solution is accelerated by a heterogeneous computing architecture composed of CPU, GPU and FPGA.

6. The method for attitude sensing and adaptive grasping of centrifuge tubes with lids according to claim 1, characterized in that, Step S2 specifically includes: S21: Based on the calculated pose information and point cloud data, the optimal grasping point is planned using the particle swarm optimization algorithm, and the target joint angle of the robotic arm and the initial opening degree of the gripper are calculated. S22: Controls the robotic arm to move to the target position and uses an adaptive gripper with a hybrid mode of pneumatic flexible drive and servo motor drive to perform grasping. The gripping force is adjusted in real time and the posture deviation is compensated through pressure closed-loop control. S23: Integrates 3D vision, pressure, and acceleration sensor data to assess grasping stability, and triggers pressure fine-tuning, pose correction, or secondary grasping based on the assessment results.

7. The method for attitude sensing and adaptive grasping of centrifuge tubes with caps according to claim 1, characterized in that, In step S21, the initial opening degree Dgrip of the gripper is calculated according to the formula "opening degree = tube diameter D + 2 × safety margin S + pressure compensation amount Δd", where D is the tube diameter fitted by point cloud, S is the safety margin dynamically adjusted according to the tube wall thickness, and Δd is the pressure compensation amount calculated from the target gripping force and the gripper flexibility coefficient.

8. A system for attitude sensing and adaptive grasping of a covered centrifuge tube for implementing the method of any one of claims 1-7, characterized in that, include: The visual acquisition module is used to acquire multimodal data including 2.5D texture images and 3D point clouds. It includes a binocular stereo camera, a structured light source, a polarization filter component, a high-resolution 2D camera, and a multi-zone light source system. The data processing module is used to fuse, extract features and calculate poses of the multimodal data, and it adopts a heterogeneous architecture of CPU, GPU and FPGA. The grasping execution module, used to perform grasping actions, includes a multi-degree-of-freedom robotic arm and an adaptive gripper mounted at its end, the gripper integrating a pressure sensor; The control module is used to receive the pose calculation results, plan the grasping trajectory and parameters, and control the grasping execution module to complete adaptive grasping and stability adjustment.

9. The centrifuge tube attitude sensing and adaptive grasping system according to claim 8, characterized in that, The visual acquisition module also includes an FPGA time synchronization trigger module, which is used to synchronously control the acquisition actions of the structured light source, the binocular stereo camera, and the high-resolution 2D camera; the multi-zone light source system includes a dome light source and a strip light source.

10. The centrifuge tube attitude sensing and adaptive grasping system according to claim 8, characterized in that, The adaptive gripper has a three-finger symmetrical structure and adopts a hybrid drive method that combines pneumatic drive and servo motor drive; the control module includes a main controller for decision planning and a motion controller for real-time motion control and pressure closed-loop control.

Images