Six-degree-of-freedom optical force sensing method based on deformation coding light field

Abstract

The invention discloses a six-degree-of-freedom optical force sensing method based on a deformation coding light field, and relates to the technical field of biomedical engineering, optical sensing, micro-robots and artificial intelligence crossing, and the method comprises the following steps: S100, sensor construction and model initialization; s200, carrying out image acquisition and standardization processing; s300, training a submerged space diffusion model; s400, generating a synthetic data set; s500, training a differential feature extraction network model; s600, performing real-time reasoning; and S700, feeding back a reasoning result. High-precision six-degree-of-freedom force / torque sensing is achieved, valuable tactile feedback can be provided for minimally invasive surgery, and the six-degree-of-freedom force / torque sensing device has important clinical application prospects.

Description

Technical Field

[0001] This application relates to the interdisciplinary fields of biomedical engineering, optical sensing, microrobotics and artificial intelligence, and in particular to a six-degree-of-freedom optical force sensing method based on deformable coded light field. Background Technology

[0002] Six-degrees-of-freedom (6-DoF) force and torque sensing technology has become a key capability for next-generation robotic systems, enabling precise physical interactions across multiple scales, from industrial assembly and micromanipulation to biomedical interventions and soft robotic exploration. Accurate measurement of six degrees of freedom and torque is crucial not only for stable contact and dexterous manipulation but also for inferring material properties and monitoring interaction states. In minimally invasive medicine, for example, the lack of tactile feedback during catheterization or endoscopy prevents clinicians from sensing tissue stiffness or detecting unstable torques; in precision manufacturing and microassembly, the lack of six-DoF feedback hinders controlled manipulation of fragile components and material interfaces. Despite decades of research, existing six-DoF sensors remain limited by size, manufacturing complexity, and cost, restricting their integration into compact or disposable systems. Traditional strain gauge devices require a rigid crossbeam structure and multiple measurement axes, resulting in a bulky size (typically exceeding 10 mm in diameter), unsuitable for conduits or continuous robots. While fiber Bragg grating (FBG) arrays offer some flexibility and electrical isolation, they still require complex spectral demodulation systems, and their size is typically difficult to reduce to below 3 mm due to the need for 9-12 gratings precisely distributed across multiple fibers or cores to decouple different degrees of freedom (L. Xiong, Y. Guo, G. Jiang, et al., “Six-dimensional force / torque sensor based on fiber bragg gratings with lowcoupling,” IEEE Trans. on Ind. Electron. 68, 4079–4089 (2020).). Capacitive sensors require multi-layered electrodes and shielding structures, similarly limiting miniaturization. Among existing six-degree-of-freedom sensors, the smallest capacitive sensor (Coin-FT) has a diameter of approximately 5.8 mm (H. Choi, J. Low, T. Huh, et al., “Coinft: A coin-sized, capacitive 6-axis force torque sensor for robotic applications,” arXiv preprint arXiv:2503.19225 (2025)), while FBG sensors are typically larger than 3 mm, and strain gauges and piezoelectric sensors are generally larger than 10 mm. Alternative optical fiber strategies leverage variations in backscattering intensity or speckle patterns within multimode or fiber bundles to reduce size. These methods enable sub-millimeter profiles and simplify signal processing without requiring spectral demodulation. However, these optical responses exhibit highly nonlinear and cross-axis coupling characteristics and are extremely sensitive to bending, temperature drift, and light source coherence. While recent studies have used machine learning to achieve 3D force classification or regression from single-frame speckle patterns, extending these results to continuous and interpretable six-DOF load tracking remains challenging. Furthermore, comprehensive calibration mapping typically requires substantial experimental data, which is difficult to maintain and ensure consistency under clinical deployment conditions. Of particular note is that calibrating such optical systems remains a major obstacle: the optical response space scales nonlinearly with the six mechanical degrees of freedom, making extensive data collection across the complete force / torque space impractical. Traditional calibration methods require intensive sampling in a multidimensional force / torque space, typically taking one to several days. This is not only time-consuming and labor-intensive but also difficult to recalibrate in practical applications to cope with temperature variations, mechanical deformation, or long-term drift. For example, optical force sensors based on speckle pattern decoding are highly sensitive to temperature / bending changes; even minor perturbations can cause nonlinear changes in the pattern, leading to significant performance degradation and requiring recalibration. Therefore, while miniature optical force sensors hold immense theoretical potential, the lack of a comprehensive solution that achieves extremely small size (<2 mm), addresses the challenges of nonlinear calibration, and maintains high accuracy and environmental robustness limits their practical application in minimally invasive surgery and microrobotic systems.

[0003] The invention patent "A six-axis force sensor and its preparation method", publication number CN117109796A, discloses a monolithic MEMS six-axis force sensor and its preparation method. However, this invention relies on electronic components, cannot achieve passive detection, and limits its application in electromagnetic environments and biomedical surgery. Furthermore, it is highly sensitive to the environment and is affected by liquid environments. The rigid MEMS structure cannot be bent, resulting in high cost and limiting its single-use.

[0004] The utility model "A Novel Strain-Type Six-Axis Force Sensor", publication number CN210426854U, discloses a novel strain-type six-axis force sensor. However, this utility model is too large, which cannot meet the needs of minimally invasive medical procedures. Furthermore, it relies on electronic components, making passive detection impossible and limiting its application in electromagnetic environments and biomedical surgeries. It also exhibits high environmental sensitivity and is affected by liquid environments. The assembly process is complex and costly.

[0005] Therefore, those skilled in the art are dedicated to developing a six-degree-of-freedom optical force sensing method based on deformable coded light fields. Summary of the Invention

[0006] In view of the above-mentioned deficiencies of the prior art, the technical problem to be solved by this application is the scarcity of calibration data and the difficulty of modeling caused by nonlinear and strongly coupled responses in six-degree-of-freedom optical force / torque sensing.

[0007] The applicant analyzed that existing optical speckle decoding force sensors are extremely sensitive to bending and temperature drift; even minor environmental changes can lead to calibration failure. The applicant, through a fully passive, enclosed optical path design and the phase alignment characteristics of the coherent fiber bundle, ensures that the internal optical-force coupling mechanism of the sensor head is unaffected by the bending of the external fiber segment, and the graphite-PDMS composite material exhibits excellent thermal stability within the 25-50°C range. Experiments have demonstrated that the system maintains high linearity (R²>0.93, most channels>0.98) even under bending radii as low as 80 mm and temperature variations up to 25°C, eliminating the need for frequent recalibration and demonstrating robustness for clinical environments.

[0008] The applicant analyzed that existing technologies either rely on complex spectral demodulation systems or require multi-channel / multi-frame measurements, making it difficult to achieve single-frame, single-channel, six-degree-of-freedom real-time feedback in a sub-2 mm passive probe. Based on a deformable coded optical field transduction mechanism, the applicant encodes six-dimensional mechanical information into a single-frame distal CMOS image and combines it with a lightweight neural network and a generative calibration framework, achieving an inference latency of <2.4 ms with only 11.3M parameters. This meets the stringent requirements of compactness and real-time performance for minimally invasive surgery and micro-robotics.

[0009] The applicant analyzes that existing technologies are susceptible to optical response fluctuations, manufacturing inconsistencies, and background noise interference. Directly regressing six-DOF loads from a single frame of light intensity image is also prone to common-mode interference, leading to decreased accuracy and inter-axis crosstalk. The applicant employs a differential feature learning strategy, jointly encoding an unloaded reference image and a deformed image under load, and calculating their feature difference (…). This effectively amplifies optical changes caused by external forces, suppresses irrelevant disturbances, thereby enhancing sensitivity to minute forces / torques and achieving effective decoupling between degrees of freedom.

[0010] The applicant argues that existing miniature optical force sensors exhibit highly nonlinear optical field responses and severe inter-axis coupling. Traditional calibration methods require dense sampling in a six-dimensional force / torque space, resulting in high costs, low efficiency, and difficulty in implementation under clinical or field conditions. The applicant proposes a generative self-calibration framework based on a Latent Diffusion Model (LDM). This framework requires only a small amount of measured data to synthesize physically consistent deformation images covering all six degrees of freedom, significantly reducing dependence on calibration data while improving model generalization ability and calibration robustness.

[0011] In one embodiment of this application, a six-degree-of-freedom optical force sensing method based on deformable coded light fields is provided, comprising the following steps: S100. Sensor Construction and Model Initialization: Construct a miniature optical force sensor, including a fully optical sensing head, an optical transmission module, an image acquisition module, and a data processing module. The fully optical sensing head includes a mechanical support, an elastomer, and a fiber optic readout system. Integrate the elastomer and the fiber optic readout system to form the fully optical sensing head. Construct a latent space diffusion model (LDM) and a differential feature extraction network model. The latent space diffusion model (LDM) includes a variational autoencoder (VAE) and a conditional diffusion module. The differential feature extraction network model includes a backbone network and a regression head. Set the state of the differential feature extraction network model to the training state. S200, image acquisition and standardization processing: A miniature optical force sensor is fixed on a six-degree-of-freedom loading platform. The miniature optical force sensor captures the original light intensity field image when the all-optical sensing head is not deformed, and uses it as a reference image. External loads are applied through the motion of a six-degree-of-freedom loading platform. Acquire deformed images To form a paired image of the reference image and the deformed image. The paired images are preprocessed to standardize them to obtain standardized paired images. The state of the differential feature extraction network model is determined. When the differential feature extraction network model is in the training state, step S300 is executed. When the differential feature extraction network model is in the inference state, step S600 is executed. S300, train the latent space diffusion model, filter standardized paired images, and construct a dataset. The training set, validation set, and test set are divided according to a specified ratio. The variational autoencoder (VAE) is trained using the training set, with reference image. and deformed images x i Inputting the trained variational autoencoder (VAE) into the inputs yields the reference latent space representation. and low-dimensional latent space representation Using low-dimensional latent space representation and external loads The conditional diffusion module is trained to complete the training of the latent space diffusion model; S400, Generate synthetic datasets for external loads. Perform Gaussian fitting, and then independently sample to obtain a set of external loads containing a specified number of loads. Inverse denoising is performed using a trained latent space diffusion model (LDM) to obtain a synthetic set of deformed latent space representations { Input the trained variational autoencoder (VAE) to obtain the synthesized deformed image. Synthesize deformed images With external load w iGenerate a synthetic dataset of deformed images and external load pairs; S500: Train the differential feature extraction network model. Set the pre-training learning rate, pre-training batch size, and pre-training epochs. Use the Adam optimizer and pre-train the differential feature extraction network model using a synthetic dataset. Use the Adam optimizer to adjust the learning rate to the mixed training learning rate. Set the mixed training batch size and mixed training epochs. Mix the training set and the synthetic data randomly extracted from the synthetic dataset for mixed training. Complete the training of the differential feature extraction network model and set the differential feature extraction network model to the inference state. Return to step S200. S600, real-time inference, inputs standardized paired images into the trained differential feature extraction network model to generate composite feature vectors. After processing by the regression head of the differential feature extraction network model, the mapping relationship between optical features and mechanical quantities is established, and the six-degree-of-freedom force / torque vector is calculated. ; S700, inference result feedback, outputs a six-degree-of-freedom force / torque vector. It completes six-degree-of-freedom optical force sensing.

[0012] Optionally, in the six-degree-of-freedom optical force sensing method based on deformable coded light field in the above embodiments, the miniature optical force sensor includes: The all-optical sensor head, including a mechanical support, an elastomer, and a fiber bundle readout system, deforms in response to an externally applied load, modulating the shape of the internal cavity and the scattered light field. The optical transmission module includes a coherent fiber bundle, in which the illumination fiber is responsible for near-end illumination, and the remaining fibers transmit the scattered light field. The image acquisition module receives the scattered light field and forms a light intensity field image; The data processing module decodes the light intensity field image through a deep learning network and outputs a six-degree-of-freedom force / torque vector. The all-optical sensor head and the optical transmission module are fixedly connected, the optical transmission module and the image acquisition module are fixedly connected, the data processing module runs on a computer, and the image acquisition module is connected to the data processing module and transmits data in the form of an external device; The LED light source provides near-end illumination through the illumination fiber. The applied six-degree-of-freedom force / torque external load causes deformation of the all-optical sensor head, modulating the internal cavity shape and scattered light field of the all-optical sensor head. The scattered light field is transmitted to the image acquisition module through the optical transmission module to form a light intensity field image, which is then decoded into a six-degree-of-freedom force / torque vector by the data processing module.

[0013] Optionally, in the six-degree-of-freedom optical force sensing method based on deformable coded light field in the above embodiments, the image acquisition module uses a CMOS camera.

[0014] Optionally, in the six-degree-of-freedom optical force sensing method based on deformable coded light field in any of the above embodiments, step S100 includes: S110. Construct a mechanical support, including metal needles, stainless steel springs and stainless steel capillaries. The metal needles, stainless steel springs and stainless steel capillaries are strictly concentrically assembled to form a mechanical support with stable mechanical properties. S120. Construct an elastomer and prepare a graphite-PDMS composite material. Immerse the mechanical support into the graphite-PDMS composite material and observe that the tip of the stainless steel spring is completely submerged. After the first dip coating, a hemispherical elastomer dome is formed. Remove the metal needle and perform a second dip coating process to form an elastomer. S130. Construct a fiber optic bundle readout system by inserting a coherent fiber optic bundle endoscope into a stainless steel capillary tube for precise alignment and observing its field of view covering the elastomer cavity; seal the interface between the coherent fiber optic bundle and the stainless steel capillary tube to form a fiber optic bundle readout system. S140. Construct a fully optical sensor head, integrating the elastomer and fiber optic bundle readout system to form a fully optical sensor head, with the elastomer, mechanical support and fiber optic bundle readout system tightly coupled. S150. Model Construction: Construct a Latent Space Diffusion Model (LDM) and a Differential Feature Extraction Network (DFR). The LDM includes a Variational Autoencoder (VAE) and a Conditional Diffusion module. The VAE includes an encoder... and decoder The differential feature extraction network model includes a backbone network and a regression head; S160. Initialize the model state and set the state of the differential feature extraction network model to the training state.

[0015] Furthermore, in the six-degree-of-freedom optical force sensing method based on deformable coded light field in the above embodiments, the diameter of the metal needle is 0.8 mm.

[0016] Furthermore, in the six-degree-of-freedom optical force sensing method based on deformable coded light field in the above embodiments, the stainless steel spring has a wire diameter of 0.15 mm, an outer diameter of 1.2 mm, an uncompressed length of 10.0 mm, and a total of 25 turns.

[0017] Furthermore, in the six-degree-of-freedom optical force sensing method based on deformable coded light field in the above embodiments, the stainless steel capillary has a length of 10.0 mm, an outer diameter of 1.0 mm, and an inner diameter of 0.8 mm.

[0018] Furthermore, in the six-degree-of-freedom optical force sensing method based on deformable coded light field in the above embodiments, the outer diameter of the coherent fiber bundle is 0.78 mm.

[0019] Furthermore, in the six-degree-of-freedom optical force sensing method based on deformable coded light field in the above embodiments, the total size of the all-optical sensing head is 1.7 mm.

[0020] Optionally, in the six-degree-of-freedom optical force sensing method based on deformable coded light field in any of the above embodiments, step S110 includes: S111. Insert a metal needle, which is inserted axially into the stainless steel capillary tube to form a needle-tube composite structure. S112. Insert the stainless steel capillary tube, attach the stainless steel spring to the outside of the stainless steel capillary tube, and keep the stainless steel spring and the port of the stainless steel capillary tube axially aligned. S113. A stable mechanical support is formed by adjusting the relative positions of each component to achieve strict concentric assembly of the metal needle, stainless steel spring and stainless steel capillary tube, thus forming a mechanical support with stable mechanical properties.

[0021] Optionally, in the six-degree-of-freedom optical force sensing method based on deformable coded light field in any of the above embodiments, step S120 includes: S121. Prepare composite material by mixing graphite powder with Sylgard 184 (a two-component polydimethylsiloxane elastomer material produced by Dow Corning) at a ratio of 1:22 to generate graphite-PDMS (graphite-polydimethylsiloxane) composite material. The graphite-PDMS composite material has ideal elastic and optical properties. Sylgard 184 includes PDMS coagulant and PDMS base material, which are mixed at a weight ratio of 1:10. S122. First dip coating: Immerse the mechanical support in the graphite-PDMS composite material until the tip of the stainless steel spring is completely submerged. Then, hang it vertically in a constant temperature environment of 25°C and let it stand for 24 hours to cure, forming a hemispherical elastomer dome that is tightly integrated with the stainless steel spring structure. S123. Second dip coating: Remove the metal needle to leave a precise cavity structure at the tip of the stainless steel spring. Then, perform a second dip coating process to immerse the lower 2mm area of ​​the mechanical support into the graphite-PDMS composite material and cure at 25°C for 24 hours to ensure uniformity of mechanical properties and form an elastomer.

[0022] Optionally, in the six-degree-of-freedom optical force sensing method based on deformable coded light field in any of the above embodiments, step S130 includes: S131. Precise alignment: Insert the coherent fiber bundle endoscope into the stainless steel capillary tube for precise alignment, and observe the field of view to completely cover the cavity area of ​​the elastomer. S132. The interface is sealed and fixed. The near-end illumination of the coherent fiber bundle is transmitted through the illumination fiber. After confirming that the imaging geometry meets the requirements, the coherent fiber bundle and the stainless steel capillary interface are sealed with UV curing adhesive and fixed by UV irradiation to form a fiber bundle readout system.

[0023] Furthermore, in the six-degree-of-freedom optical force sensing method based on deformable coded light field in the above embodiments, the graphite powder is 3500 mesh graphite powder.

[0024] Optionally, in the six-degree-of-freedom optical force sensing method based on deformable coded light field in any of the above embodiments, step S150 includes: S151. Design a latent space diffusion model (LDM), including a variational autoencoder (VAE) and a conditional diffusion module. The variational autoencoder (VAE) adopts a structure with four sets of linear layers and ReLU activation layers alternating. The conditional diffusion module adopts a noise prediction network with a U-Net architecture. S152. Design a differential feature extraction network model, including a backbone network and a regression head. The backbone network adopts ResNet-18, and the regression head adopts a linear fully connected layer with a 6-dimensional output. This ensures sufficient feature representation capabilities while meeting the strict computational resource constraints of embedded deployment.

[0025] Optionally, in the six-degree-of-freedom optical force sensing method based on deformable coded light field in any of the above embodiments, step S200 includes: S210, Miniature optical force sensor fixed, the miniature optical force sensor is fixed on a six-degree-of-freedom loading platform, and the translation and lifting of the miniature optical force sensor are controlled; S220: Acquire the original light intensity field. Use a miniature optical force sensor to capture an image of the original light intensity field when the all-optical sensing head is not deformed, as a reference image. ; S230. Apply external load by moving the six-degree-of-freedom loading platform. The six-degree-of-freedom loading platform moves in uniform steps to ensure full coverage of the six-degree-of-freedom force / torque and avoid uneven data distribution. S240. Acquire paired images, capturing images each time an external load is applied. The deformed image below To form a paired image of the reference image and the deformed image. ; S250. Paired image standardization preprocessing: crop the circular field of view of the coherent fiber bundle of the paired image to eliminate the influence of stray light at the edges; then perform grayscale normalization to scale the pixel values ​​to the 0-1 range; finally downsample to 128×128 resolution to balance spatial detail and computational complexity to obtain the standardized paired image.

[0026] Optionally, in the six-degree-of-freedom optical force sensing method based on deformable coded light field in any of the above embodiments, the six-degree-of-freedom loading platform includes a rotary table, a translation table, and a lifting table.

[0027] Furthermore, in the six-degree-of-freedom optical force sensing method based on deformable coded light field in the above embodiments, the range of the external load is: Fx and Fy Varying within the range of ±0.1N, Fz Apply in a compressed form within the range of 0-0.6 N. Tx and Ty Covering a range of ±1 N·mm, Tz This is limited to within ±0.1 N·mm. Furthermore, in the six-degree-of-freedom optical force sensing method based on deformable coded light field in the above embodiments, the external loads of the six degrees of freedom are recorded by external sensors.

[0028] Furthermore, in the six-degree-of-freedom optical force sensing method based on deformable coded light field in the above embodiments, an external sensor is installed as a calibration sensor (ATI Nano43) six-axis force sensor.

[0029] Furthermore, in the six-degree-of-freedom optical force sensing method based on deformable coded light field in the above embodiments, the process of acquiring images maintains consistent lighting conditions and imaging parameters, thereby ensuring that the optical response is caused only by external mechanical stimuli and avoiding interference caused by ambient light fluctuations.

[0030] Optionally, in the six-degree-of-freedom optical force sensing method based on deformable coded light field in any of the above embodiments, step S300 includes: S310. Construct a dataset by filtering standardized paired images to exclude abnormal samples caused by platform vibration, lighting fluctuations, or mechanical noise. The training set, validation set, and test set are divided according to a specified ratio, and the load configurations of the training set, validation set, and test set do not overlap, maintaining statistical independence. S320, Training the Variational Autoencoder (VAE): Train the variational autoencoder (VAE) using the training set, while simultaneously optimizing the encoder. and decoder By minimizing each deformed image x iWith decoder The reconstruction loss between reconstructed images enables the variational autoencoder (VAE) to learn to utilize the encoder to reconstruct images. Compressed to a low-dimensional latent space, and the decoder can be used from the low-dimensional latent space. Accurate image reconstruction is achieved by using the Adam optimizer, setting the learning rate, batch size, and training epochs, and using mean squared error as the loss function to complete the training of the variational autoencoder (VAE), which has reliable image encoding and decoding capabilities. S330, Obtain the latent space representation of the image, using the reference image. Input the trained variational autoencoder (VAE) to obtain the corresponding reference latent space representation. ; one by one deformed images x i Input the trained variational autoencoder (VAE) to obtain the corresponding low-dimensional latent space representation. ; S340, Training Condition Diffusion Module, converts external loads Embedded through a fully connected network, external payloads Spatial extension to low-dimensional latent space representation Hidden conditional fields with perfectly matched resolution Representation of low-dimensional latent space By progressively adding Gaussian noise, a low-dimensional latent space representation of superimposed noise is obtained. The learning prediction adds noise residuals, and the training is performed by minimizing the mean square error between the predicted noise and the real noise as the loss function. The Adam optimizer is used, and the learning rate, batch size and training epochs are set to complete the training of the conditional diffusion module, thereby completing the training of the latent space diffusion model.

[0031] Furthermore, in the six-degree-of-freedom optical force sensing method based on deformable coded light field in the above embodiments, the reconstruction loss includes pixel-level L1 loss and SSIM sensing loss.

[0032] Furthermore, in the six-degree-of-freedom optical force sensing method based on deformable coded light field in the above embodiments, the specified ratio is training set: validation set: test set = 8:1:1.

[0033] Furthermore, in the six-degree-of-freedom optical force sensing method based on deformable coded light field in the above embodiments, step S340 includes: S341, Embedding implicit condition fields, external loads Embedded through a fully connected network, external payloads Spatial extension to low-dimensional latent space representation Hidden conditional fields with perfectly matched resolution ; S342, Training Condition Diffusion Module, representing to a low-dimensional latent space. Adding Gaussian noise yields a low-dimensional latent space representation of superimposed noise. Then the conditional diffusion module With latent conditional fields and time step As a condition, the noise residuals added to the learning prediction are used. The training is performed by minimizing the mean square error between the predicted noise and the real noise as the loss function. The Adam optimizer is used, and the learning rate, batch size and training epochs are set to complete the training of the conditional diffusion module, thereby completing the training of the latent space diffusion model.

[0034] Optionally, in the six-DOF optical force sensing method based on deformable coded light field in any of the above embodiments, the learning rate is set to The batch size was set to 64, and the training rounds were set to 100.

[0035] Furthermore, in the six-degree-of-freedom optical force sensing method based on deformable coded light field in the above embodiments, step S400 includes: S410, Generate a set of composite deformation latent space representations for external loads. Perform Gaussian fitting, and then independently sample to obtain a set of external loads containing a specified number of loads. Represented by reference latent space and As a condition, from randomly generated Gaussian noise Run the inverse denoising process of the trained latent space diffusion model (LDM), iterating for a specified number of denoising steps to generate a synthesized model with the external payload. Corresponding deformable latent space representation The synthesized deformable latent space representation set { is obtained. }; S420. Synthesize deformed images, and represent the synthesized deformed latent space as a set { The trained variational autoencoder (VAE) is input and transformed back into image space to obtain the complete synthetic deformable image. ; S430. Construct a synthetic dataset, combining synthetic deformed images. and its corresponding reference image With external load w i Generate a synthetic dataset of deformed images and external load pairs. Covering the range of external loads with six degrees of freedom. Fx and Fy Varying within the range of ±0.1N, FzApply in a compressed form within the range of 0-0.6 N. Tx and Ty Covering a range of ±1 N·mm, Tz It is then limited to within ±0.1 N·mm.

[0036] Preferably, in the six-degree-of-freedom optical force sensing method based on deformable coded light field in the above embodiments, the number of specified loads is 100,000.

[0037] Preferably, in the six-degree-of-freedom optical force sensing method based on deformable coded light field in the above embodiments, the number of denoising steps is specified as 50.

[0038] Optionally, in the six-degree-of-freedom optical force sensing method based on deformable coded light field in any of the above embodiments, step S500 includes: S510. Pre-training: Set the pre-training learning rate, batch size, and number of pre-training epochs. Use the Adam optimizer and pre-train the differential feature extraction network model using a synthetic dataset. Extract reference images. Corresponding feature embedding Extract deformed images Corresponding feature embedding Calculate the feature difference To form a composite feature vector , to composite feature vector The force / torque is mapped to a six-degree-of-freedom force / torque regression to establish a preliminary mechanical response mapping relationship; S520. Hybrid training: Using the Adam optimizer, adjust the learning rate to the hybrid training learning rate, set the hybrid training batch size and hybrid training rounds, and perform hybrid training by mixing the training set and synthetic data randomly drawn from the synthetic dataset at the hybrid training ratio. This maintains generalization ability while ensuring physical consistency. The training of the differential feature extraction network model is completed, and the state of the differential feature extraction network model is set to the inference state. Return to step S200 to start the inference process.

[0039] Preferably, in the six-DOF optical force sensing method based on deformable coded light field in the above embodiments, the pre-training learning rate is set to 1×10⁻⁶. -3 The pre-training batch size was set to 64, and the pre-training rounds were set to 100.

[0040] Preferably, in the six-degree-of-freedom optical force sensing method based on deformable coded light field in the above embodiments, the mixed training ratio is 80:20.

[0041] Preferably, in the six-DOF optical force sensing method based on deformable coded light field in the above embodiments, the hybrid training learning rate is set to 1×10⁻⁶. -4The batch size for mixed training is set to 64, and the number of mixed training rounds is set to 300.

[0042] This application utilizes a deformation-encoded optical field transduction mechanism to encode six-degree-of-freedom forces / torques into a single-frame light intensity image through the deformation encoding of a graphite-PDMS elastomer dome, achieving single-channel, single-frame six-degree-of-freedom sensing without requiring multi-channel / multi-frame measurements or complex spectral demodulation systems. It employs a miniature all-optical sensor structure, using a concentric assembly of a coherent fiber bundle, a graphite-PDMS elastomer dome, and a miniature spring support. The elastomer sensor head is formed through a dip-coating process, achieving a six-degree-of-freedom force sensor of the smallest size, overcoming miniaturization bottlenecks. A generative self-calibration framework is used, constructing a two-stage training strategy based on the latent space diffusion model (LDM). Using an unloaded reference image as a condition, it generates physically consistent synthetic deformation images covering the entire six-degree-of-freedom force / torque space, significantly reducing dependence on calibration data and solving the calibration technical problems caused by highly nonlinear optical response and strong coupling. A differential feature learning strategy is used to calculate the feature difference (…) by jointly encoding the unloaded reference image and the deformation image under load. Δf = f i - f This invention effectively amplifies optical changes caused by external forces, suppresses light source fluctuations, manufacturing unevenness, and background noise interference, and achieves effective decoupling between degrees of freedom. Furthermore, based on environmentally robust design, the fully passive, closed optical path design and the phase alignment characteristics of the coherent fiber bundle ensure that the internal optical-force coupling mechanism of the sensor head is unaffected by the bending of external fiber segments. The graphite-PDMS composite material exhibits excellent thermal stability within the 25-50°C range, ensuring stable operation of the system under clinically relevant environmental changes. This application not only achieves high-precision six-degree-of-freedom force / torque sensing but also provides valuable tactile feedback for minimally invasive surgery, demonstrating significant clinical application potential.

[0043] The following will further explain the concept, specific structure and technical effects of this application in conjunction with the accompanying drawings, so as to fully understand the purpose, features and effects of this application. Attached Figure Description

[0044] Figure 1 This is a flowchart of a six-degree-of-freedom optical force sensing method according to an exemplary embodiment; Figure 2 This is a static performance evaluation diagram of an exemplary embodiment; Figure 3 This is a dynamic performance evaluation diagram of an exemplary embodiment; Figure 4 This is an exemplary embodiment of the performance evaluation diagram for bending; Figure 5 This is a performance evaluation graph for temperature in an exemplary embodiment; Figure 6 This is a graph of six-degree-of-freedom force / torque values ​​measured by tangential measurement at the center of a simulated tumor, as shown in the exemplary embodiment. Figure 7 This is a load characteristic diagram of an exemplary embodiment when the probe contacts the left side of the tumor; Figure 8 This is a load characteristic diagram of the probe contacting the right side of the tumor in an exemplary embodiment; Figure 9 This is a load feature diagram of the probe contacting the tumor directly above in an exemplary embodiment; Figure 10 This is a baseline state diagram of an exemplary embodiment where the probe only contacts the surface of a uniform gel. Detailed Implementation

[0045] The following description, with reference to the accompanying drawings, illustrates several preferred embodiments of this application to make its technical content clearer and easier to understand. This application can be embodied in many different forms, and the scope of protection of this application is not limited to the embodiments mentioned herein.

[0046] In the accompanying drawings, components with the same structure are designated by the same numerical designation, and components with similar structures or functions are designated by similar numerical designations. The dimensions and thicknesses of each component shown in the drawings are arbitrary, and this application does not limit the dimensions and thicknesses of each component. To make the illustrations clearer, the thickness of components is schematically exaggerated in some places in the drawings.

[0047] The applicant designed a six-degree-of-freedom optical force sensing method based on deformable coded light fields, such as... Figure 1 As shown, it includes the following steps: S100. Sensor Construction and Model Initialization: Construct a miniature optical force sensor, including a fully optical sensing head, an optical transmission module, an image acquisition module, and a data processing module. The fully optical sensing head includes a mechanical support, an elastomer, and a fiber optic readout system. Integrate the elastomer and the fiber optic readout system to form the fully optical sensing head. Construct a latent space diffusion model (LDM) and a differential feature extraction network model. The latent space diffusion model (LDM) includes a variational autoencoder (VAE) and a conditional diffusion module. The differential feature extraction network model includes a backbone network and a regression head. Set the state of the differential feature extraction network model to the training state. The miniature optical force sensor includes: The all-optical sensor head, including a mechanical support, an elastomer, and a fiber bundle readout system, deforms in response to an applied external load, modulating the shape of the internal cavity and the scattered light field; The optical transmission module includes a coherent fiber bundle, in which the illumination fiber is responsible for near-end illumination, and the remaining fibers transmit the scattered light field. The image acquisition module uses a CMOS camera to receive the scattered light field and form a light intensity field image. The data processing module decodes the light intensity field image through a deep learning network and outputs a six-degree-of-freedom force / torque vector. The all-optical sensor head and the optical transmission module are fixedly connected, the optical transmission module and the image acquisition module are fixedly connected, the data processing module runs on a computer, and the image acquisition module is connected to the data processing module and transmits data in the form of an external device; The LED light source provides near-end illumination through the illumination fiber. The applied external load causes the all-optical sensor head to deform, modulating the internal cavity shape and scattered light field of the all-optical sensor head. The scattered light field is transmitted to the image acquisition module through the optical transmission module to form a light intensity field image, which is then decoded into a six-degree-of-freedom force / torque vector by the data processing module.

[0048] Step S100 specifically includes: S110. Construct a mechanical support, including a metal needle, a stainless steel spring, and a stainless steel capillary tube. The metal needle has a diameter of 0.8 mm, the stainless steel spring has a wire diameter of 0.15 mm, an outer diameter of 1.2 mm, an uncompressed length of 10.0 mm, and 25 turns. The stainless steel capillary tube has a length of 10.0 mm, an outer diameter of 1.0 mm, and an inner diameter of 0.8 mm. The metal needle, stainless steel spring, and stainless steel capillary tube are assembled concentrically to form a mechanical support with stable mechanical properties. Specifically, this includes: S111. Insert a metal needle, which is inserted axially into the stainless steel capillary tube to form a needle-tube composite structure. S112. Insert the stainless steel capillary tube, attach the stainless steel spring to the outside of the stainless steel capillary tube, and keep the stainless steel spring and the port of the stainless steel capillary tube axially aligned. S113. A stable mechanical support is formed by adjusting the relative positions of each component to achieve strict concentric assembly of the metal needle, stainless steel spring and stainless steel capillary tube, thus forming a mechanical support with stable mechanical properties.

[0049] S120. Constructing an elastomer and preparing a graphite-PDMS composite material: Immersing a mechanical support into the graphite-PDMS composite material, observing that the stainless steel spring tip is completely submerged; after the first dip coating, a hemispherical elastomer dome is formed; the metal needle is removed, and a second dip coating process is performed to form the elastomer; specifically including: S121. Prepare composite material by mixing 3500 mesh graphite powder with Sylgard 184 (a two-component polydimethylsiloxane elastomer material produced by Dow Corning) at a ratio of 1:22 to generate graphite-PDMS (graphite-polydimethylsiloxane) composite material. The graphite-PDMS composite material has ideal elastic and optical properties. Sylgard 184 includes PDMS coagulant and PDMS base material, which are mixed at a weight ratio of 1:10. S122. First dip coating: Immerse the mechanical support in the graphite-PDMS composite material until the tip of the stainless steel spring is completely submerged. Then, hang it vertically in a constant temperature environment of 25°C and let it stand for 24 hours to cure, forming a hemispherical elastomer dome that is tightly integrated with the stainless steel spring structure. S123. Second dip coating: Remove the metal needle to leave a precise cavity structure at the tip of the stainless steel spring. Then, perform a second dip coating process to immerse the lower 2mm area of ​​the mechanical support into the graphite-PDMS composite material and cure at 25°C for 24 hours to ensure uniformity of mechanical properties and form an elastomer.

[0050] S130. Construct a fiber optic bundle readout system by inserting a coherent fiber optic bundle endoscope into a stainless steel capillary tube for precise alignment and observing its field of view covering the elastomer cavity; seal the interface between the coherent fiber optic bundle and the stainless steel capillary tube to form the fiber optic bundle readout system. The outer diameter of the coherent fiber optic bundle is 0.78 mm. Specifically, this includes: S131. Precise alignment: Insert the coherent fiber bundle endoscope into the stainless steel capillary tube for precise alignment, and observe the field of view to completely cover the cavity area of ​​the elastomer. S132. The interface is sealed and fixed. The near-end illumination of the coherent fiber bundle is transmitted through the illumination fiber. After confirming that the imaging geometry meets the requirements, the coherent fiber bundle and the stainless steel capillary interface are sealed with UV curing adhesive and fixed by UV irradiation to form a fiber bundle readout system.

[0051] S140. Construct a fully optical sensor head by integrating the elastomer and fiber optic bundle readout system to form a fully optical sensor head. The elastomer, mechanical support, and fiber optic bundle readout system are tightly coupled. The total size of the fully optical sensor head is 1.7 mm.

[0052] S150. Model Construction: Construct a Latent Space Diffusion Model (LDM) and a Differential Feature Extraction Network (DFR). The LDM includes a Variational Autoencoder (VAE) and a Conditional Diffusion module. The VAE includes an encoder... and decoder The differential feature extraction network model includes a backbone network and a regression head; specifically, it includes: S151. Design a latent space diffusion model (LDM), including a variational autoencoder (VAE) and a conditional diffusion module. The variational autoencoder (VAE) adopts a structure with four sets of linear layers and ReLU activation layers alternating. The conditional diffusion module adopts a noise prediction network with a U-Net architecture. S152. Design a differential feature extraction network model, including a backbone network and a regression head. The backbone network adopts ResNet-18, and the regression head adopts a linear fully connected layer with a 6-dimensional output. This ensures sufficient feature representation capabilities while meeting the strict computational resource constraints of embedded deployment.

[0053] S160. Initialize the model state and set the state of the differential feature extraction network model to the training state.

[0054] S200, image acquisition and standardization processing: A miniature optical force sensor is fixed on a six-degree-of-freedom loading platform. The miniature optical force sensor captures the original light intensity field image when the all-optical sensing head is not deformed, and uses it as a reference image. External loads are applied through the motion of a six-degree-of-freedom loading platform. Acquire deformed images To form a paired image of the reference image and the deformed image. The paired images are preprocessed using standardization to obtain standardized paired images; the state of the differential feature extraction network model is determined. If the differential feature extraction network model is in the training state, step S300 is executed; if the differential feature extraction network model is in the inference state, step S600 is executed. Specifically, this includes: S210, Miniature optical force sensor fixed, the miniature optical force sensor is fixed on a six-degree-of-freedom loading platform, and the translation and lifting of the miniature optical force sensor are controlled; S220: Acquire the original light intensity field. Use a miniature optical force sensor to capture an image of the original light intensity field when the all-optical sensing head is not deformed, as a reference image. ; S230. Apply external loads by moving a six-degree-of-freedom loading platform, which includes a rotary table, a translation table, and a lifting table. The six-degree-of-freedom loading platform moves in uniform steps to ensure full coverage of the six-degree-of-freedom forces / torques and avoid uneven data distribution. The external loads of the six degrees of freedom are recorded by external sensors, which are equipped with calibration sensors (ATI Nano43) six-axis force sensors. S240. Acquire paired images, capturing images each time an external load is applied. The deformed image below To form a paired image of the reference image and the deformed image. ; S250. Paired image standardization preprocessing: crop the circular field of view of the coherent fiber bundle of the paired image to eliminate the influence of stray light at the edges; then perform grayscale normalization to scale the pixel values ​​to the 0-1 range; finally downsample to 128×128 resolution to balance spatial detail and computational complexity to obtain the standardized paired image.

[0055] S300, train the latent space diffusion model, filter standardized paired images, and construct a dataset. The training set, validation set, and test set are divided according to a specified ratio. The variational autoencoder (VAE) is trained using the training set, with reference image. and deformed images x i Inputting the trained variational autoencoder (VAE) into the inputs yields the reference latent space representation. and low-dimensional latent space representation Using low-dimensional latent space representation and external loads The conditional diffusion module is trained to complete the training of the latent space diffusion model; specifically, it includes: S310. Construct a dataset by filtering standardized paired images to exclude abnormal samples caused by platform vibration, lighting fluctuations, or mechanical noise. The training set, validation set, and test set are divided into three groups in a ratio of 8:1:1, with no overlap in load configuration between them to maintain statistical independence.

[0056] S320, Training the Variational Autoencoder (VAE): Train the variational autoencoder (VAE) using the training set, while simultaneously optimizing the encoder. and decoder By minimizing each deformed image x i With decoder The reconstruction loss between reconstructed images, including pixel-level L1 loss and SSIM perceptual loss, enables the variational autoencoder (VAE) to learn to utilize the encoder to reconstruct images. Compressed to a low-dimensional latent space, and the decoder can be used from the low-dimensional latent space. Accurate image reconstruction is achieved by using the Adam optimizer, setting the learning rate, batch size, and training epochs, and using mean squared error as the loss function to complete the training of the variational autoencoder (VAE), which has reliable image encoding and decoding capabilities.

[0057] S330, Obtain the latent space representation of the image, using the reference image. Input the trained variational autoencoder (VAE) to obtain the corresponding reference latent space representation. ; one by one deformed images xi Input the trained variational autoencoder (VAE) to obtain the corresponding low-dimensional latent space representation. .

[0058] S340, Training Condition Diffusion Module, converts external loads Embedded through a fully connected network, external payloads Spatial extension to low-dimensional latent space representation Hidden conditional fields with perfectly matched resolution Representation of low-dimensional latent space By progressively adding Gaussian noise, a low-dimensional latent space representation of superimposed noise is obtained. The training process involves learning to predict the added noise residuals, using the mean squared error between the predicted and actual noise as the loss function. The Adam optimizer is employed, and the learning rate, batch size, and training epochs are set to train the conditional diffusion module, thereby completing the training of the latent space diffusion model. Specifically, this includes: S341, Embedding implicit condition fields, external loads Embedded through a fully connected network, external payloads Spatial extension to low-dimensional latent space representation Hidden conditional fields with perfectly matched resolution ; S342, Training Condition Diffusion Module, representing to a low-dimensional latent space. Adding Gaussian noise yields a low-dimensional latent space representation of superimposed noise. Then the conditional diffusion module With latent conditional fields and time step As a condition, the learning predicts the added noise residuals, and is trained by minimizing the mean square error between the predicted noise and the actual noise as the loss function, using the Adam optimizer with a learning rate set to [value missing]. The batch size was set to 64, and the training rounds were set to 100 rounds to complete the training of the conditional diffusion module, thereby completing the training of the latent space diffusion model.

[0059] S400, Generate synthetic datasets for external loads. Perform Gaussian fitting, and then independently sample to obtain a set of external loads containing a specified number of loads. Inverse denoising is performed using a trained latent space diffusion model (LDM) to obtain a synthetic set of deformed latent space representations { Input the trained variational autoencoder (VAE) to obtain the synthesized deformed image. Synthesize deformed images With external load w iGenerate a synthetic dataset of deformed image-external load pairs; specifically including: S410, Generate a set of composite deformation latent space representations for external loads. Perform Gaussian fitting, and then independently sample to obtain a set of external loads containing a specified number of 100,000 loads. Represented by reference latent space and As a condition, from randomly generated Gaussian noise Run the inverse denoising process of the trained latent space diffusion model (LDM), iterating for a specified number of denoising steps (50), to generate a synthesized model with external payloads. Corresponding deformable latent space representation The synthesized deformable latent space representation set { is obtained. }; S420. Synthesize deformed images, and represent the synthesized deformed latent space as a set { The trained variational autoencoder (VAE) is input and transformed back into image space to obtain the complete synthetic deformable image. ; S430. Construct a synthetic dataset, combining synthetic deformed images. and its corresponding reference image With external load w i Generate a synthetic dataset of deformed images and external load pairs. Covering the range of external loads with six degrees of freedom. Fx and Fy Varying within the range of ±0.1N, Fz Apply in a compressed form within the range of 0-0.6 N. Tx and Ty Covering a range of ±1 N·mm, Tz It is then limited to within ±0.1 N·mm.

[0060] S500: Train the differential feature extraction network model. Set the pre-training learning rate, pre-training batch size, and pre-training epochs. Use the Adam optimizer and pre-train the differential feature extraction network model using a synthetic dataset. Using the Adam optimizer, adjust the learning rate to the mixed training learning rate, set the mixed training batch size and mixed training epochs, and perform mixed training by mixing the training set and synthetic data randomly drawn from the synthetic dataset at a mixed training ratio. Complete the training of the differential feature extraction network model, and set the differential feature extraction network model to the inference state. Return to step S200. Specifically, this includes: S510, pre-training, pre-training learning rate set to 1×10 -3The pre-training batch size was set to 64, the pre-training epochs were set to 100, the Adam optimizer was used, and the differential feature extraction network model was pre-trained using a synthetic dataset to extract reference images. Corresponding feature embedding Extract deformed images Corresponding feature embedding Calculate the feature difference To form a composite feature vector , to composite feature vector The force / torque is mapped to a six-degree-of-freedom force / torque regression to establish a preliminary mechanical response mapping relationship; S520, hybrid training, using the Adam optimizer, hybrid training learning rate set to 1×10⁻⁶. -4 The batch size for mixed training is set to 64, the number of mixed training rounds is set to 300, and the mixed training set and the synthetic data randomly extracted from the synthetic dataset are mixed for training at a ratio of 80:20 to maintain generalization ability while ensuring physical consistency. The training of the differential feature extraction network model is completed, and the state of the differential feature extraction network model is set to the inference state. Then, return to step S200 to start the inference process.

[0061] S600, real-time inference, inputs standardized paired images into the trained differential feature extraction network model to generate composite feature vectors. After processing by the regression head of the differential feature extraction network model, the mapping relationship between optical features and mechanical quantities is established, and the six-degree-of-freedom force / torque vector is calculated. .

[0062] S700, inference result feedback, outputs a six-degree-of-freedom force / torque vector. It completes six-degree-of-freedom optical force sensing.

[0063] To verify the performance and practicality of the present invention, comprehensive static, dynamic and environmental robustness tests were conducted, and application verification was performed in simulated clinical scenarios.

[0064] The applicant conducted a static evaluation experiment, such as Figure 2 As shown, the above embodiments exhibit excellent linearity across all six load channels, where Fx, Fy, Fz, Tx, Ty channel R² The values ​​all exceeded 0.99. Tz Due to its limited measurement range and the resolution limitations of the reference sensor, the channel... R² The value is approximately 0.78. Among them... R²Also known as the coefficient of determination, it is a statistical indicator used to quantify the consistency between the predicted output of a mechanics perception model and a standard reference measurement. Its value ranges from 0 to 1, and it is used to objectively evaluate the accuracy and effectiveness of a regression model. Its calculation formula can be found at: https: / / www.originlab.com / doc / en / Origin-Help / Details_of_R_square. Specifically, R² This characterizes the proportion of variance in the mechanical response data explained by the model. When R² The closer the value is to 1, the stronger the linear correlation between the model's predicted values ​​and the actual measured values, the smaller the deviation between the predicted output and the standard reference value, and the higher the model's prediction accuracy and reliability. Conversely, when... R² The closer the value is to 0, the more it indicates that the model cannot effectively explain the variance of the data and has limited predictive ability.

[0065] In the above embodiments, R² Used for each degree of freedom (including Fx , Fy , Fz , Tx , Ty, Tz The predictive performance of the force or moment component is evaluated independently and comprehensively. For example, for a specific force or moment component, its predictive performance is evaluated independently and comprehensively. R² A value exceeding 0.99 indicates a highly linear relationship between the predicted output of this channel and the actual load, verifying the effectiveness of the optical encoding mechanism and decoding algorithm. R² These indicators provide core, quantifiable criteria for evaluating the static sensing accuracy of sensor systems.

[0066] The applicant conducted a dynamic evaluation experiment, such as Figure 3 As shown, under representative loading conditions, the regressed force and torque values ​​in the above embodiments closely follow the actual measured values, with no obvious response delay or baseline drift, proving that the system has real-time and high-fidelity dynamic response capabilities, meeting the stringent requirements of minimally invasive surgery and micro-robots for real-time feedback.

[0067] To verify the stability and robustness of the above embodiments under different environmental conditions, the applicant conducted fiber optic bending and temperature tests. Figure 4 As shown, in fiber optic bending tests, even under extreme conditions with bending radii as low as 80 mm, the system maintained high linearity across all six channels. R² >0.93, most channels >0.98). For example... Figure 5 As shown, in temperature testing, the system exhibited excellent stability within a temperature range of 25°C to 50°C, and the performance of each channel was [data missing]. R²The values ​​change very little. These results confirm that the present invention, through its fully passive, closed optical path design and phase alignment characteristics with coherent fiber bundles, effectively overcomes the performance degradation caused by environmental disturbances (such as temperature changes and fiber bending), and can operate stably in clinically relevant environments without frequent recalibration.

[0068] The applicant further conducted trials in simulated clinical scenarios, applying the above embodiments to tumor palpation during minimally invasive surgery. A 4 mm diameter plastic sphere, made of 5% (w / v) gelatin material, was used to coat the stomach wall as a simulated tumor. Figure 6 As shown, the force and torque curves measured by simulating the tangent at the center of the tumor show that the force and torque amplitudes are smaller in areas far from the tumor (①); and in the tumor boundary region (②), the normal force... Fz Reaching its maximum value, the torque component reflects the lateral deformation towards the tumor center; above the tumor center (③), Fz The peak value is obvious, while the lateral torque decreases.

[0069] In a simulated minimally invasive surgery setting, the applicant verified the tissue identification capability through four independent trials. Figures 7-10 The four scenarios are shown in turn: contact with the left boundary, the right boundary, the center, and contact with only the simulated skin. Figure 7 This demonstrates the load characteristics when the probe contacts the left boundary of the tumor, at which point the normal force... Fz Significant increase, while torque Tx The presence of a reverse peak indicates that the tissue exerted a lateral reaction force to the right on the probe. Figure 8 The load characteristics and normal force when the probe contacts the right boundary of the tumor are shown. Fz The torque also increased significantly, but Tx It manifests as a positive peak value, indicating that the tissue exerted a lateral reaction force to the left on the probe. Figure 9 The load characteristics of the probe when it contacts the center of the tumor were recorded, and the normal force at this point was recorded. Fz When the torque reaches its maximum value Tx Approaching zero, it conforms to the mechanical properties of symmetrical contact. Figure 10 This presents the baseline state where the probe only contacts the uniform gel surface (simulating skin), and the normal force... Fz and torque Tx All values ​​remained at low levels, close to zero. These results demonstrate that the above embodiments successfully distinguished between the pre-contact, during-contact, and post-contact stages, and accurately identified the tumor boundary and center location. The results prove that this invention not only achieves high-precision six-degree-of-freedom force / torque sensing but also provides valuable tactile feedback for minimally invasive surgery, demonstrating significant clinical application prospects.

[0070] The inference process in the above experiments achieved an extremely low latency of <2.4 ms on an NVIDIA RTX 5090 GPU, meeting the stringent real-time requirements of minimally invasive surgery and micro-robots; in practical applications such as tumor palpation, the system can... Fz Peak values ​​precisely locate the tumor, and through Tx / Ty The resulting vortex field accurately delineates the tumor boundary, providing clinicians with intuitive tactile feedback and significantly improving the safety and precision of surgical procedures.

[0071] The preferred embodiments of this application have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of this application without inventive effort. Therefore, any technical solutions that can be obtained by those skilled in the art based on the concept of this application through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A six-degree-of-freedom optical force sensing method based on deformation- encoding light field, characterized in that, The method comprises the following steps: S100, sensor construction and model initialization, constructing a miniature optical force sensor, including a full optical sensor head, an optical transmission module, an image acquisition module and a data processing module, the full optical sensor head includes a mechanical support, an elastomer and a fiber bundle readout system, the elastomer and the fiber bundle readout system are integrated to form the full optical sensor head, a latent space diffusion model and a differential feature extraction network model are constructed, the latent space diffusion model includes a variational autoencoder and a conditional diffusion module, the differential feature extraction network model includes a backbone network and a regression head, the state of the differential feature extraction network model is set to a training state; S200, image acquisition and standardization processing, fixing the micro optical force sensor on a six-degree-of-freedom loading platform, capturing an original light intensity field image of the full optical sensor head when no deformation occurs as a reference image through the micro optical force sensor , applying external load through the movement of the six-degree-of-freedom loading platform , collecting deformation images , forming a pair of images of the reference image and the deformation image , standardizing the pair of images for preprocessing to obtain a standardized pair of images; judging the state of the differential feature extraction network model, when the differential feature extraction network model is in a training state, executing step S300, and when the differential feature extraction network model is in an inference state, executing step S600; S300, training the latent space diffusion model, screening the standardized paired images to construct a data set , dividing the training set, the validation set and the test set according to the specified proportion, training the variational autoencoder using the training set, the reference image and the morphed image x i respectively input into the trained variational autoencoder to obtain the reference latent space representation and the low-dimensional latent space representation , using the low-dimensional latent space representation and the external load training the conditional diffusion module to complete the training of the latent space diffusion model; S400, generating a synthetic data set, for the external load Gaussian fitting is performed, and then independent sampling is performed to obtain an external load set containing a specified number of loads , using the trained latent space diffusion model to perform reverse denoising to obtain a set of synthetic deformation latent space representations }, input the trained variational autoencoder to obtain a synthetic deformation image , input the synthetic deformation image and the external load w i generate a synthetic data set of synthetic deformation image-external load pairs; S500, training the differential feature extraction network model, setting the pre-training learning rate, the pre-training batch size and the pre-training round, using the Adam optimizer, pre-training the differential feature extraction network model using the synthetic data set; using the Adam optimizer, adjusting the learning rate to a hybrid training learning rate, setting the hybrid training batch size and the hybrid training round, mixing the training set and the synthetic data randomly extracted from the synthetic data set in a hybrid training ratio for hybrid training, completing the training of the differential feature extraction network model, setting the state of the differential feature extraction network model to an inference state, and returning to the step S200; S600, inputting the standardized paired images into a trained differential feature extraction network model in real time to generate a composite feature vector , establishing a mapping relationship between optical features and mechanical quantities through the regression head of the differential feature extraction network model, and calculating a six-degree-of-freedom force / torque vector ; S700, the inference result is fed back, and the six-degree-of-freedom force / torque vector is output , and six-degree-of-freedom optical force sensing is completed.

2. The six degree-of-freedom optical force sensing method based on deformation- encoding light field of claim 1, wherein, The miniature optical force sensor comprises: a full optical sensor head, including a mechanical support, an elastomer and a fiber bundle readout system, which deforms in response to an applied external load, modulates the internal cavity shape and scatters the light field; an optical transmission module, including a coherent fiber bundle, wherein an illumination fiber is responsible for proximal illumination, and the remaining fibers transmit the scattered light field; an image acquisition module that receives the scattered light field and forms a light intensity field image; a data processing module that decodes the light intensity field image through the differential feature extraction network model and outputs a six-degree-of-freedom force / torque vector; The full optical sensor head and the optical transmission module are fixedly connected, the optical transmission module and the image acquisition module are fixedly connected, the data processing module runs on a computer, and the image acquisition module is connected and data is transmitted with the data processing module in the form of an external device.

3. The six degree-of-freedom optical force sensing method based on deformation- encoded light field of claim 2, wherein, The step S100 comprises: S110, constructing a mechanical support, including a metal needle, a stainless steel spring and a stainless steel capillary, the metal needle, the stainless steel spring and the stainless steel capillary are strictly concentrically assembled to form the mechanical support with stable mechanical properties; S120, constructing an elastomer, preparing a graphite-PDMS composite material, immersing the mechanical support in the graphite-PDMS composite material, observing that the stainless steel spring tip is completely immersed, forming a hemispherical elastomer dome after the first dip coating, extracting the metal needle, and performing a second dip coating process to form the elastomer; S130, constructing a fiber bundle readout system, inserting a coherent fiber bundle endoscope into the stainless steel capillary, performing accurate alignment, and observing that its field of view covers the elastomer cavity; sealing the interface between the coherent fiber bundle and the stainless steel capillary to form the fiber bundle readout system; S140, constructing a full optical sensing head, integrating the elastomer and the fiber bundle readout system to form the full optical sensing head, the elastomer, the mechanical support and the fiber bundle readout system being tightly coupled; S150, constructing a model, constructing the latent space diffusion model and the differential feature extraction network model, the latent space diffusion model comprising a variational autoencoder and a conditional diffusion module, the variational autoencoder comprising an encoder and a decoder , the differential feature extraction network model comprising a backbone network and a regression head; S160, model state initialization, setting the state of the differential feature extraction network model to a training state.

4. The six degree-of-freedom optical force sensing method based on deformation- encoded light field of claim 3, wherein, The step S110 includes: S111, inserting the metal needle, the metal needle is inserted into the stainless steel capillary along the axial direction to form a needle-tube composite structure; S112, placing the stainless steel spring sleeve outside the stainless steel capillary, and making the stainless steel spring axially aligned with the stainless steel capillary port; S113, forming the mechanical support, adjusting the relative positions of the components to realize the strict concentric assembly of the metal needle, the stainless steel spring and the stainless steel capillary, and forming the mechanical support with stable mechanical properties.

5. The six degree-of-freedom optical force sensing method based on deformation- coded light field of claim 3, wherein, The step S120 includes: S121, preparing a composite material, mixing graphite powder and Sylgard 184 at a ratio of 1:22 to generate the graphite-PDMS composite material, the graphite-PDMS composite material having ideal elasticity and optical properties, the Sylgard 184 including a PDMS curing agent and a PDMS base, the PDMS curing agent and the PDMS base being mixed at a weight ratio of 1:10; S122, first dip coating, immersing the mechanical support in the graphite-PDMS composite material until the tip of the stainless steel spring is completely immersed, and then vertically hanging in a constant temperature environment of 25°C for 24 hours for curing to form a hemispherical elastomer dome tightly combined with the stainless steel spring structure; S123, second dip coating, extracting the metal needle to leave an accurate cavity structure at the tip of the stainless steel spring, and then performing a second dip coating process to immerse the lower 2mm region of the mechanical support in the graphite-PDMS composite material, and curing for 24 hours in a 25°C environment to ensure uniformity of mechanical properties, and forming the elastomer.

6. The six degree-of-freedom optical force sensing method based on anamorphic coded light field of claim 3, wherein, The step S130 includes: S131, precise alignment, inserting the coherent fiber bundle endoscope into the stainless steel capillary for precise alignment, and observing that the field of view completely covers the elastomer cavity region; S132, interface sealing and fixing, the coherent fiber bundle proximal end illumination is transmitted through the illumination fiber, after confirming that the imaging geometry meets the requirements, the interface between the coherent fiber bundle and the stainless steel capillary is sealed using ultraviolet curing glue, and is fixed by ultraviolet irradiation, forming the fiber bundle readout system.

7. The six degree-of-freedom optical force sensing method based on deformation- coded light field of claim 3, wherein, The step S200 includes: S210, micro optical force sensor fixing, fixing the micro optical force sensor on the six-degree-of-freedom loading platform to control the translation and lifting of the micro optical force sensor; S220, collecting an original light intensity field, capturing an original light intensity field image of the full optical sensing head when no deformation occurs as the reference image through the micro optical force sensor ; S230, applying an external load, applying the external load through the motion of the six-degree-of-freedom loading platform, and the six-degree-of-freedom loading platform uniformly steps to ensure full coverage of six-degree-of-freedom force / torque, avoiding uneven data distribution; S240, collecting a pair of images, collecting a deformation image under each application of an external load , forming the pair of images of the reference image and the deformation image ;​ S250, paired image standardization preprocessing, cropping the circular field of view area of the coherent optical fiber bundle of the paired image, eliminating the influence of edge stray light; then performing gray scale normalization, scaling the pixel value to the interval of 0-1; finally, down-sampling to 128*128 resolution to balance spatial details and computational complexity, to obtain the standardized paired image.

8. The six degree-of-freedom optical force sensing method based on anamorphic coded light field of claim 7, wherein, The step S300 comprises: S310, constructing a data set, screening the standardized paired images, excluding abnormal samples caused by platform vibration, light fluctuation or mechanical noise, and constructing the data set , dividing the training set, the validation set and the test set according to the specified proportion, the load configuration between the training set, the validation set and the test set is not overlapped, and the statistical independence is maintained; S320, training a variational autoencoder, training the variational autoencoder using the training set while optimizing the encoder and the decoder by minimizing the reconstruction loss between each transformed image x i and the reconstructed image by the decoder , so that the variational autoencoder learns to compress images into a low-dimensional latent space using the encoder and accurately reconstruct images from the low-dimensional latent space using the decoder. The training of the variational autoencoder is completed using the Adam optimizer, setting the learning rate, batch size, and training rounds, and using the mean square error as the loss function, and the variational autoencoder has reliable image encoding and decoding capabilities. S330, obtaining a latent space representation of the image, the reference image inputting the trained variational autoencoder to obtain a corresponding reference latent space representation ; sequentially inputting the deformed images x i inputting the trained variational autoencoder to obtain a corresponding low-dimensional latent space representation ; S340, the training condition diffusion module, the external load by embedding through a fully connected network, the external load space expansion into a latent condition field with a resolution that completely matches the low-dimensional latent space representation , the low-dimensional latent space representation gradually add Gaussian noise to obtain a low-dimensional latent space representation superimposed with noise , learn to predict the added noise residual, train by minimizing the mean square error between the predicted noise and the real noise as the loss function, use the Adam optimizer, set the learning rate, batch size and training rounds, complete the training of the condition diffusion module, and thus complete the training of the latent space diffusion model.​ 9. The six degree-of-freedom optical force sensing method based on anamorphic coded light field of claim 8, wherein, The step S400 comprises: S410, generating a set of synthetic deformation latent space representations, for the external loadings Performing Gaussian fitting and then independently sampling to obtain a set of external loadings containing a specified number of loadings , with the reference latent space representation and as conditions, from randomly generated Gaussian noise Running the inverse denoising process of the trained latent space diffusion model, iterating a specified number of denoising steps, to generate a set of synthetic deformation latent space representations corresponding to the external loadings , obtaining a set of synthetic deformation latent space representations ​ S420, synthesize a morphed image, from the set of synthesized morphed latent space representations } input the trained variational autoencoder, converting it back to image space, resulting in a complete synthesized morphed image ; S430, constructing a synthetic dataset of synthetic deformation images and their corresponding reference images with the external load w i generating a synthetic dataset of synthetic deformation image-external load pairs covering the range of external loads of six degrees of freedom, Fx and Fy varying in the range of ±0.1 N, Fz applied in a compressed form in the range of 0-0.6 N, Tx and Ty covering the range of ±1 N·mm, Tz then limited in the range of ±0.1 N·mm.

10. The six degree-of-freedom optical force sensing method based on anamorphic coded light field of claim 9, wherein, The step S500 comprises: S510, pre-training, setting pre-training learning rate, pre-training batch size and pre-training round, using Adam optimizer, using the synthetic data set to pre-train the differential feature extraction network model, extracting the reference image corresponding feature embedding , extracting the deformed image corresponding feature embedding , calculating feature difference , forming a composite feature vector , mapping the composite feature vector to six degrees of freedom force / torque regression, establishing a preliminary mechanical response mapping relationship; S520, mixed training, using the Adam optimizer, adjusting the learning rate to the mixed training learning rate, setting the mixed training batch size and the mixed training round, and mixing the training set and the synthetic data randomly extracted from the synthetic data set in the mixed training ratio to complete the training of the differential feature extraction network model, and put the differential feature extraction network model state into the inference state, return to the step S200 to start the inference process.

Images