A high-precision pressure estimation method for a mechanical hand based on visual perception

Abstract

The application discloses a high-precision pressure estimation method for a manipulator based on visual perception, and belongs to the field of visual perception. The method comprises the following steps: acquiring an RGB image, and pre-processing the RGB image to obtain a standardized image; constructing a deep learning network based on a double-channel feature enhancement module, inputting the standardized image into the deep learning network to extract pressure features; and mapping the pressure features into a pressure distribution map to obtain a pressure estimation value. The application realizes high-precision pressure estimation for a manipulator based on visual perception, and provides key support for precise operation and safe interaction of the manipulator.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of visual perception, and particularly relates to a high-precision pressure estimation method for a mechanical hand based on visual perception. BACKGROUND

[0002] The international academic community has shown multi-dimensional innovative exploration in the field of mechanical hand contact surface pressure perception, mainly focusing on three major directions: high-performance tactile sensing skin, pressure distribution intelligent analysis algorithm, and bionic tactile system. In the field of high-performance tactile sensing skin, Ntagios and other research teams pioneered the combination of 3D printing technology and soft material sensors, successfully developing a mechanical hand system with inherent tactile perception capabilities. This research has broken through the limitations of traditional tactile sensors in terms of structural complexity and integration efficiency, providing a paradigm shift in the development of new-generation intelligent mechanical structures. The breakthrough in the field of pressure distribution analysis algorithm research lies in the innovative classification and segmentation method proposed by Albini and Cannata. A special pressure mapping algorithm is designed for contact force patterns in human-machine interaction scenarios. This method converts complex contact pressure distribution into interpretable interaction patterns through advanced computational models, significantly enhancing the depth and accuracy of robot systems in understanding human-machine contact states. This progress not only improves the accuracy of human-machine interaction, but also lays the algorithm foundation for the development of safe collaborative robots. In the research of bionic tactile system, Roberts and others have constructed a theoretical framework for robot soft tactile perception skin through systematic summary and analysis. Their research in-depthly explores various technical paths and material selection strategies for simulating biological tactile systems, and compares and analyzes the performance characteristics and application limitations of different bionic methods, providing comprehensive theoretical guidance and design ideas for the development of new mechanical hand tactile systems with high sensitivity and multi-functionality.

[0003] The existing technology in the field of mechanical hand contact surface pressure sensing also presents a diversified development trend, mainly focusing on the development of new sensing materials, intelligent algorithm assisted sensing and multi-modal system integration. In the research of flexible sensors, Zhang et al. developed a multifunctional sensing unit integrating pressure measurement and ultrasonic detection functions. By adopting innovative composite material structure, the sensor realizes high-precision contact force detection, providing a new sensing dimension for robot grasping operation. This kind of new flexible sensor not only improves the sensing accuracy of the mechanical hand, but also enhances its adaptability on irregular surfaces. Significant progress has been made in the field of artificial intelligence assisted sensing. Yang et al. combined machine learning algorithm with pressure sensing mechanism to develop an intelligent fingertip tactile system for humanoid robots. The system can accurately identify the contact pressure distribution of complex surfaces and classify the contact type and object material according to the pressure characteristics. Through deep neural network feature extraction and pattern recognition, the research team effectively solves the nonlinear problem in contact force analysis, improving the intelligent level of mechanical hand and environment interaction. In the integration of multifunctional sensing system, Chen et al. designed a flexible dual-mode sensor, which combines precise contact pressure measurement and non-contact distance detection functions. The sensor adopts multi-layer composite structure and micro-nano manufacturing process to realize the sensing conversion in contact and non-contact states. This multi-modal fusion sensing method provides more comprehensive sensing information for the fine operation of robots in complex environments, expanding the adaptability of mechanical hands in uncertain environments.

[0004] Although the existing technology has made significant progress, the current solutions for mechanical hand pressure sensing research still have obvious limitations. Traditional contact sensors face many engineering challenges: flexible sensors are difficult to perfectly fit complex curved surfaces, often causing sensing blind spots; rigid sensors will interfere with fine operations. More importantly, these systems lack reliability in actual application environment, and friction and wear, pollution and environmental changes can cause significant performance degradation. Complex sensor wiring and signal processing not only increases the system volume and weight, but also raises the manufacturing and maintenance costs.

[0005] Vision-based pressure estimation faces reliability challenges in practical applications, with significant differences in appearance of soft grippers under the same pressure due to factors such as posture, lighting, occlusion and surface texture. The performance of data-driven methods depends heavily on the coverage of training data. Models trained in controlled environments (fixed viewing angle, smooth surface) are difficult to generalize to complex scenarios (textured surface, multi-angle, occlusion, etc.). Direct installation of sensors can affect surface characteristics and contact mechanics characteristics. Although physical simulation provides a way to solve this problem, it is still challenging to accurately simulate the deformation of the gripper and the visual phenomenon. To address the above challenges, the present invention breaks through the limitations of traditional sensing methods and provides a high-precision pressure estimation method for mechanical hands based on visual sensing. SUMMARY

[0006] To solve the above technical problems, the application provides a high-precision pressure estimation method for a manipulator based on visual perception to solve the problems existing in the prior art.

[0007] To achieve the above object, the application provides a high-precision pressure estimation method for a manipulator based on visual perception, comprising:

[0008] obtaining an RGB image and preprocessing the RGB image to obtain a standardized image;

[0009] constructing a deep learning network based on a dual-channel feature enhancement module, inputting the standardized image into the deep learning network to extract pressure features;

[0010] mapping the pressure features to a pressure distribution map to obtain a pressure estimation value.

[0011] Optionally, the deep learning network constructed based on the dual-channel feature enhancement module comprises an encoder, a feature pyramid network and a decoder.

[0012] The process of extracting pressure features based on the deep learning network comprises:

[0013] inputting the standardized image into the encoder, and processing the standardized image based on a plurality of cascaded dual-channel feature enhancement modules to obtain multi-scale features by the encoder;

[0014] performing cross-scale feature fusion on the multi-scale features by the feature pyramid network to obtain fused features;

[0015] obtaining pressure features based on the fused features;

[0016] performing up-sampling and convolution operations on the pressure features by the decoder to obtain predicted pressure categories.

[0017] Optionally, the dual-channel feature enhancement module comprises a spatial feature modulation module and a VP-Mamba module.

[0018] The calculation expression for processing the standardized image based on the dual-channel feature enhancement module to obtain multi-scale features is:

[0019] F out =α·F SFM +(1-α)·F Mamba

[0020] In the formula, F SFM is the output of the SFM module, F Mamba is the output of the VP-Mamba module, and a is an adaptive fusion weight.

[0021] Optionally, the process of calculating the SFM module output comprises:

[0022] obtaining input features based on the input standardized image;

[0023] performing feature grouping on the input features to obtain a plurality of feature groups;

[0024] applying a multi-scale feature generation unit to each feature group to obtain multi-scale features;

[0025] performing feature aggregation and attention generation on the multi-scale features to obtain a multi-scale feature map;

[0026] performing feature modulation on the multi-scale feature map and the input features to obtain the SFM module output.

[0027] Optionally, the process of calculating the VP-Mamba module output comprises:

[0028] performing block segmentation and serialization on the input features to obtain a plurality of feature blocks;

[0029] performing state space processing on the plurality of feature blocks using an SSM to obtain state outputs;

[0030] performing scanning on the state outputs to obtain scanning outputs;

[0031] performing feature reorganization on the scanning outputs to obtain the VP-Mamba module output.

[0032] Optionally, the calculation expression of the multi-scale features after cross-scale feature fusion by the feature pyramid network is:

[0033] M n =Conv 1×1 (F n )

[0034] M i =Conv 1×1 (F i )+Upsample(M i+1 ),i∈n-1,n-2,...,1

[0035] wherein Conv 1×1 represents a 1×1 convolution operation, Upsample represents an up-sampling operation, M i is the fused feature of the i-th layer, and F i represents the feature representation of the i-th layer.

[0036] Optionally, the expression of the pressure estimation value is:

[0037]

[0038] In the formula, N=9 represents the number of pressure intervals, P pred (x,y) is the predicted pressure value of pixel (x,y), O(x,y,i) represents the probability that the pixel belongs to the i-th pressure interval, P i is the representative pressure value of the i-th interval.

[0039] Optionally, after constructing the deep learning network, a network training process is further included, wherein a composite loss function is used to guide the deep learning network training.

[0040] The expression of the composite loss function is: L total = L CE + λ1·L SSIM + λ2·L smooth

[0041] In the formula, L total represents the composite loss value, L CE represents the classification cross-entropy loss, L SSIM represents the structural similarity loss, λ1 represents the structural similarity loss weight, L smooth represents the smooth loss, and λ2 represents the smooth loss weight.

[0042] Compared with the prior art, the present application has the following advantages and technical effects:

[0043] The present application realizes high-precision pressure estimation of a mechanical hand based on visual perception, provides key support for precise operation and safe interaction of the mechanical hand, and promotes the development of mechanical hand pressure perception technology in a more practical, reliable and intelligent direction. In particular, in application scenarios such as medical surgery, precision assembly and food processing, which have extremely high requirements for hygiene and safety, the contactless pressure perception method provided by the present application has significant advantages. DETAILED DESCRIPTION

[0044] The accompanying drawings, which form a part of the present application, are intended to provide further understanding of the present application, and the illustrative embodiments of the present application and their description serve the purpose of explaining the present application. The accompanying drawings do not constitute an inappropriate limitation on the present application. In the drawings:

[0045] Figure 1 FIG. 1 is a schematic diagram of a high-precision pressure estimation model based on visual perception according to an embodiment of the present application;

[0046] Figure 2 FIG. 2 is a DCFE module diagram according to an embodiment of the present application. DETAILED DESCRIPTION

[0047] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other in the case of no conflict. The present application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0048] It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that herein.

[0049] Embodiment one

[0050] The present application proposes a high-precision visual pressure estimation method, which realizes high-precision estimation of contact pressure through a single RGB image. The method is based on the contact image segmentation technology of the manipulator, which avoids the pixel-by-pixel pressure estimation process. By introducing a dual-channel feature enhancement (DCFE) module, the effective extraction of manipulator detail features and regional correlation is realized, so that the contact pressure of the manipulator in a complex environment can be effectively estimated.

[0051] As shown in Figure 1 , it is a deep learning framework for robot gripper pressure estimation. The accurate estimation of contact pressure is realized through a single RGB image. The network is designed based on an encoder-decoder architecture, and the spatial information is preserved through a skip connection. The encoder integrates multiple DCFE modules pre-trained on ImageNet, which has strong feature extraction capability. The decoder uses a feature pyramid network (FPN) to realize multi-scale feature fusion, which enhances the recognition ability of pressure distribution at different scales. The model output is a pressure map with the same dimension as the input image, realizing pixel-level contact location and pressure estimation.

[0052] The model proposed in the present application converts the pressure estimation into a classification problem, and uses 8 logarithmic equally spaced intervals and 1 zero pressure interval for pressure value discretization. The model classifies each pixel of the input image into a predefined pressure interval.

[0053] The model uses synchronously collected RGB images and pressure sensing data for training and verification. Through homographic transformation, high-resolution pressure sensor data is mapped to the RGB image space to realize accurate alignment of pressure data and images, facilitating quantitative comparison of model prediction results and measured values.

[0054] The model includes: an encoder: using a DCFE module to extract multi-scale features; a feature pyramid network (FPN): realizing cross-scale feature fusion; a decoder: generating a pressure map with the same dimension as the input image.

[0055] Among them

[0056] The present invention provides a high-precision pressure estimation method for a manipulator based on visual perception, which can achieve high-precision estimation of contact pressure through a single RGB image, avoiding various challenges faced by traditional contact sensors in applications.

[0057] This embodiment provides a high-precision pressure estimation method for a manipulator based on visual perception, comprising the following steps:

[0058] S1. Data acquisition and preprocessing: The images of the manipulator and the contact surface are acquired through an external RGB camera and the images are standardized.

[0059] As a specific implementation of this embodiment, the process of data acquisition and preprocessing includes:

[0060] S1.1 Image Acquisition: This paper uses a standard RGB camera to capture images of the interaction between the manipulator and the contact surface from an external perspective, with an image resolution of 1920 × 1080 pixels. The images captured by the camera contain information about the manipulator's deformation, shadow changes, and characteristics of the contact area.

[0061] S1.2 Training data construction:

[0062] (1) Synchronously collect RGB images of manipulator operation and high-precision pressure sensor data;

[0063] (2) Map the pressure sensor data to the RGB image space through homography transformation:

[0064] P image =H·P sensor

[0065] Among them, P image is the image space pressure value, P sensor is the sensor measurement value, H is the homography transformation matrix;

[0066] (3) Pressure value discretization design: Divide the continuous pressure value into 8 logarithmically equally spaced intervals and 1 zero pressure interval to form a classification problem. The specific interval division formula is:

[0067]

[0068] Among them, P min and P max Represent the minimum and maximum pressure values ​​respectively, N is the total number of pressure intervals (N=9 in the present invention), P threshold is the threshold for determining contact.

[0069] S2. Pressure Feature Extraction: A designed deep learning network architecture based on the DCFE module extracts deformation and contact features from RGB images. The deep learning network-based pressure feature extraction process includes: inputting a standardized image into the encoder, which processes the standardized image using several cascaded dual-channel feature enhancement modules to generate multi-scale features; cross-scale feature fusion of the multi-scale features using a feature pyramid network to generate fused features; pressure features are derived from the fused features; and the decoder performs upsampling and convolution operations on the pressure features to obtain the predicted pressure category.

[0070] The network architecture designed in this embodiment is as follows Figure 1 As shown in the figure, the proposed method, based on an encoder-decoder architecture, preserves spatial information through skip connections. The encoder integrates multiple ImageNet-pretrained DCFE modules, providing powerful feature extraction capabilities. The decoder uses a feature pyramid network (FPN) to fuse multi-scale features, enhancing the ability to recognize pressure distributions at different scales. The model outputs a pressure map with the same dimensions as the input image, enabling pixel-level contact position and pressure estimation.

[0071] The model includes: encoder: uses DCFE module to extract multi-scale features; feature pyramid network (FPN): realizes cross-scale feature fusion; decoder: generates a pressure map with the same dimension as the input image.

[0072] S2.1 encoder

[0073] The encoder is composed of multiple cascaded DCFE modules, and the input is RGB image The output is a multi-scale feature representation F = F1, F2, ..., F n ,in H i 、W i is the spatial dimension of the feature map of layer i, C i is the number of channels.

[0074] The encoding process can be expressed as:

[0075] F i =DCFE i (F i-1 ),i∈1,2,...,n

[0076] Among them, F0=I is the input image, DCFE i is the i-th dual-channel feature enhancement module.

[0077] like Figure 2 As shown in the figure, the dual-channel feature enhancement (DCFE) module includes two sub-modules: spatial feature modulation (SFM) and VP-Mamba, which are used to enhance the model's ability to perceive details and model long-range dependencies.

[0078] The SFM module extracts multi-scale features through independent computation and dynamic aggregation, enhancing the model's detail perception. It dynamically adjusts feature processing strategies based on image content, improving the model's adaptability to diverse low-resolution images. The VP-Mamba module enhances global feature modeling by establishing long-range dependencies, which is crucial for improving performance in tasks such as image classification, segmentation, and object detection.

[0079] The overall calculation process of the dual-channel feature enhancement (DCFE) module is as follows:

[0080] F out =α·F SFM +(1-α)·F Mamba

[0081] Among them, F SFM is the output of the SFM submodule, F Mmba is the output of the VP-Mamba submodule, and α is the adaptive fusion weight.

[0082] S2.1.1 Spatial Feature Modulation (SFM) Module

[0083] The SFM module, specifically designed for image segmentation tasks, extracts multi-scale features through independent computation and dynamic aggregation. SFM first divides the input features into four groups, which are processed by the Multi-Scale Feature Generation Unit (MFGU). The MFGU uses 3×3 depthwise convolutions and self-adjusted max pooling to extract local and global features. The extracted features are concatenated along the channel dimension and aggregated using 1×1 convolutions before generating an attention map using the GELU activation function. Finally, SFM adaptively modulates the input features based on the attention map. The SFM module possesses three core features: multi-scale feature representation, adaptive feature modulation, and non-local feature interaction. Through adaptive pooling and the GELU activation function, the module processes multi-scale information, simultaneously capturing local details and global semantics and dynamically adapting to different image features. Compared to traditional attention mechanisms, SFM uses 1×1 convolutions to aggregate multi-scale features, significantly improving feature representation capabilities.

[0084] The specific calculation steps of the SFM module include: obtaining input features based on the input standardized image; grouping the input features to obtain several feature groups; applying a multi-scale feature generation unit to each feature group to obtain multi-scale features; performing feature aggregation and attention generation on the multi-scale features to obtain a multi-scale feature map; and feature modulating the multi-scale feature map and input features to obtain the SFM module output.

[0085] (1) Feature grouping: Input features Divided into four groups:

[0086] F1, F2, F3, F4 = Split(F in )

[0087] where B is batch size, C is channel number, and H and W are spatial dimensions.

[0088] (2) Multi-scale feature generation: apply a multi-scale feature generation unit (MFGU) to each feature group:

[0089] F i ' = MFGU(F i ), i e 1, 2, 3, 4

[0090] The specific operations of MFGU include:

[0091] 3x3 depthwise convolution to extract local features: F local = DWConv 3×3 (F i )

[0092] Adaptive max-pooling to extract global features: F global = AdaptiveMaxPool(F i )

[0093] Feature combination: F i ' = Concat(F local , F global )

[0094] (3) Feature aggregation and attention generation to obtain multi-scale feature maps:

[0095] F cat = Concat(F1', F2', F3', F4')

[0096] F agg = Conv 1×1 (F cat )

[0097] A map = GELU(F agg )

[0098] where F cat is a multi-scale feature formed by connecting the four feature groups processed by MFGU in the channel dimension, F agg is an aggregated feature obtained by applying a 1x1 convolution operation to F cat , and A map is an attention map generated by applying a GELU activation function to F agg . GELU is a Gaussian Error Linear Unit activation function:

[0099]

[0100] (4) Characteristic modulation:

[0101] F SFM =F in ⊙A map

[0102] Among them, ⊙ represents element-by-element multiplication, which realizes adaptive modulation of features.

[0103] S2.1.2VP-Mamba Module

[0104] VP-Mamba adopts a state-space model (SSM) architecture design, which effectively captures the long-range dependency features of images through SSM. The module consists of an input layer and an SSM layer: the input layer is responsible for image segmentation, and the SSM layer processes the segmented data and establishes dependencies between regions. The module also integrates an adaptive selection mechanism to achieve efficient feature extraction and classification through dynamic filtering and output layer processing. The main advantage of VP-Mamba lies in its long-range dependency modeling capabilities, which excels in image classification and understanding tasks. Based on the dynamic characteristics and selection mechanism of SSM, VP-Mamba can adaptively process different image content and effectively extract classification-related features. This design achieves excellent performance while maintaining a low parameter count and maintains the ability to perceive details in high-resolution image processing, meeting the requirements of high-precision application scenarios.

[0105] VP-Mamba introduces a selection mechanism to dynamically filter information and generate classification results. This selection mechanism dynamically processes and filters spatial information, allowing the model to focus on key features while reducing computational complexity. It extends the one-dimensional sequence model to two-dimensional image processing, achieving efficient processing and stable performance for inputs of varying complexity. The specific steps are as follows:

[0106] First, the scan expansion operation divides the input image into multiple blocks, and then these blocks are scanned sequentially so that the model can gradually obtain the global information of the image.

[0107] Secondly, each tag sequence is selectively scanned to obtain a context tag sequence, and SSM is used to implement long-range dependency modeling to capture the spatial correlation of image features. Finally, the local and global information are fused and the fused feature is output.

[0108] The VP-Mamba module is designed based on the state-space model (SSM) and is used to capture long-range dependencies in images. The specific process includes: the process of calculating the VP-Mamba module output includes: dividing the input features into blocks and serializing them to obtain several feature blocks; using the SSM to perform state-space processing on the several feature blocks to obtain state output; scanning the state output to obtain scanned output; and recombining the features of the scanned output to obtain the VP-Mamba module output.

[0109] The processing flow is as follows:

[0110] (1) Image segmentation and serialization to obtain several feature blocks:

[0111] X tokens =Tokenize(F in )

[0112] Among them, the Tokenize operation transforms the feature map F in Divide into n×n blocks and arrange them in scanning order;

[0113] (2) State space processing to obtain state output: Apply SSM to obtain long-range dependencies:

[0114] X ssm =SSM(X tokens )

[0115] X ssm Indicates status output, X tokens Represents a feature block.

[0116] The core calculation process of SSM is:

[0117] h t =A·h t-1 +B·x t

[0118] y t =C·h t

[0119] in, is the hidden state vector, x t is the input sequence, y t is the output sequence, is the state transition matrix, and are the input mapping and output mapping matrices respectively.

[0120] (3) Selective scanning to obtain scan output:

[0121] X scanned =SelectiveScan(X ssm )

[0122] The selective scanning processes the sequence from four directions, capturing the full azimuthal spatial dependence:

[0123] X out = ScanH(X ssm ) + ScanV(X ssm ) + ScanHR(X ssm ) + ScanVR(X ssm )

[0124] where ScanH, ScanV, ScanHR, ScanVR denote horizontal, vertical, horizontal reverse, and vertical reverse scanning, respectively, and X out denotes the scanned output.

[0125] (4) Feature reshaping:

[0126] F MAmba = Reshape(X out , [B, C, H, W])

[0127] The processed sequence is reshaped to the original feature map shape.

[0128] S2.2 Feature Pyramid Network (FPN)

[0129] The FPN module implements cross-scale feature fusion, and the specific calculation process is:

[0130] M n = Conv 1×1 (F n )

[0131] M i = Conv 1×1 (F i ) + Upsample(M i+1 ), i e n-1, n-2,..., 1

[0132] where Conv 1×1 denotes a 1x1 convolution operation, Upsample denotes an upsample operation, and M i is the fused feature of the i-th layer.

[0133] S2.3 Decoder

[0134] The decoder restores the feature map to the original image size through upsampling and convolution operations, and outputs the predicted pressure category:

[0135] O = Softmax(Conv 3×3 (P1))

[0136] where, P(x,y,i) represents the probability distribution of each pixel belonging to N pressure intervals, N=9 represents the number of pressure intervals. The final pressure value can be obtained by weighted summation:

[0137]

[0138] where P pred (x,y) is the predicted pressure value of pixel (x,y), O(x,y,i) represents the probability of the pixel belonging to the i-th pressure interval, P i is the representative pressure value of the i-th interval.

[0139] S2.4 Model training and optimization

[0140] S2.4.1 Loss function design

[0141] The present application adopts a composite loss function to guide network training:

[0142] L total = L CE +λ1·L SSIM +λ2·L smooth

[0143] Classification cross-entropy loss:

[0144] where y x,y,c is the true label of pixel (x,y) in class c, p x,y,c is the predicted probability;

[0145] Structural similarity loss: L SSIM =1-SSIM(P pred ,P gt )

[0146] where SSIM represents the structural similarity index, which is used to ensure the spatial coherence of the predicted pressure map;

[0147] Smoothness loss:

[0148]

[0149] where and represent the gradient operators in the horizontal and vertical directions, and this loss encourages the pressure map to remain sharp at the image edges and smooth in the smooth regions.

[0150] S2.4.2 Training strategy

[0151] The AdamW optimizer is used, and the initial learning rate is set to 1×10 -4 ;

[0152] Cosine annealing learning rate scheduling, with a minimum learning rate of 1×10 -6 , the calculation formula is:

[0153]

[0154] Among them, η t is the learning rate of the tth round, and T is the total number of training rounds;

[0155] The batch size is set to 8 and the number of training rounds is 100;

[0156] Data augmentation techniques are applied during training, including random cropping, rotation, horizontal flipping, and brightness changes.

[0157] S3, pressure value estimation: Map the extracted features into a pressure distribution map to achieve pixel-level pressure estimation;

[0158] S4. Result output and application: Generate a pressure map with the same dimensions as the input image for subsequent precise control of the manipulator.

[0159] As a specific implementation of this embodiment, the online prediction process includes:

[0160] Get a single RGB image I from the camera; preprocess the image, including resizing and pixel value normalization; input the processed image into the trained model to obtain the pressure distribution map P pred ; Generate a visual pressure distribution map based on the prediction results and superimpose it on the original image.

[0161] The present invention can be applied to the precise operation control of a manipulator, and the specific implementation method is as follows:

[0162] (1) Based on the principle of visual servoing, a closed-loop system of pressure estimation and manipulator position control is established;

[0163] (2) The system control formula is:

[0164]

[0165] Among them, u t For control commands, P target is the target pressure distribution, P pred is the current estimated pressure, K p and K d are the proportional and differential control coefficients respectively;

[0166] (3) Based on the above control strategy, the manipulator can achieve: precise control of the size and distribution of contact pressure; maintain stable pressure while sliding along the surface; and adapt to objects of different stiffness and shapes.

[0167] The present invention has the following technical advantages:

[0168] Non-contact sensing: Avoids the physical limitations and reliability problems of traditional contact sensors in applications;

[0169] High precision estimation: Through the dual-channel design of the DCFE module, fine-grained pressure estimation is achieved, and the average absolute error is lower than that of traditional methods;

[0170] Robustness: The model maintains stable performance under different lighting, viewing angles and object material conditions;

[0171] Real-time: The optimized network structure can achieve real-time inference speed of 30fps on a standard GPU;

[0172] Easy to integrate: Only an external RGB camera is needed, without the need to install additional sensors on the robot, greatly reducing system complexity and cost.

[0173] Through the above technical solutions, the application realizes high-precision pressure estimation of the robot based on visual sensing, provides key support for precise operation and safe interaction of the robot, and promotes the development of robot pressure sensing technology in a more practical, reliable and intelligent direction. Especially in medical surgery, precision assembly and food processing and other application scenarios with extremely high requirements for hygiene and safety, the non-contact pressure sensing method provided by the application has significant advantages.

[0174] The above is only the preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A high-precision pressure estimation method for a manipulator based on visual perception, characterized in that: The following steps are involved: Acquire an RGB image, and preprocess the RGB image to obtain a standardized image; Building a deep learning network based on a dual-channel feature enhancement module, and inputting the standardized image into the deep learning network to extract pressure features; The deep learning network includes: an encoder, a feature pyramid network and a decoder; The process of extracting pressure features based on the deep learning network includes: inputting the standardized image into an encoder, the encoder processing the standardized image based on a plurality of cascaded dual-channel feature enhancement modules to obtain multi-scale features; performing cross-scale feature fusion on the multi-scale features through the feature pyramid network to obtain fused features; obtaining pressure features based on the fused features; and obtaining a predicted pressure category through upsampling and convolution operations on the pressure features through the decoder. The dual-channel feature enhancement module includes a spatial feature modulation module and a VP-Mamba module; The calculation expression for obtaining the multi-scale features by processing the standardized image based on the dual-channel feature enhancement module is: F out =α·F SFM +(1-α)·F Mamba Where, F SFM is the output of the SFM module, F Mamba is the output of the VP-Mamba module, α is the adaptive fusion weight; The process of the SFM module output includes: obtaining input features based on the input standardized image; performing feature grouping on the input features to obtain a plurality of feature groups; applying a multi-scale feature generation unit to each feature group to obtain multi-scale features; performing feature aggregation and attention generation on the multi-scale features to obtain a multi-scale feature map; and performing feature modulation on the multi-scale feature map and the input features to obtain the SFM module output. The process of calculating the VP-Mamba module output includes: dividing the input features into blocks and serializing them to obtain a plurality of feature blocks; performing state space processing on the plurality of feature blocks using SSM to obtain a state output; scanning the state output to obtain a scan output; and performing feature recombination on the scan output to obtain a VP-Mamba module output. The calculation expression of the fused features obtained by cross-scale feature fusion of the multi-scale features through the feature pyramid network is: M n =Conv 1×1 (F n ) M i =Conv 1×1 (F i )+Upsample(M i+1 ),i∈n-1,n-2,...,1 Where, Conv 1×1 Represents a 1×1 convolution operation, Upsample represents an upsampling operation, and M i is the feature after fusion of the i-th layer, F i represents the feature representation of the i-th layer; Mapping the pressure characteristics into a pressure distribution map to obtain a pressure estimation value; The expression of the pressure estimate is: Where, N=9 represents the number of pressure intervals, P pred (x,y) is the predicted pressure value of pixel (x,y), O(x,y,i) represents the probability that the pixel belongs to the i-th pressure interval, P i is the representative pressure value of the i-th interval.

2. The high-precision pressure estimation method for a manipulator based on visual perception according to claim 1 is characterized in that: After constructing the deep learning network, a network training process is also included, wherein a composite loss function is used to guide the deep learning network training; The expression of the composite loss function is: L total =L CE +λ1·L SSIM +λ2·L smooth Where, L total Represents the composite loss value, L CE represents the classification cross entropy loss, L SSIM represents the structural similarity loss, λ1 represents the structural similarity loss weight, L smooth represents the smooth loss, and λ2 represents the smooth loss weight.

Images