An Embodied Intelligence Method for Shape Control in Fiber Flexible Body Operation Processes

Abstract

This invention relates to an embodied intelligent method for shape control during the operation of flexible fiber optic bodies, belonging to the field of flexible body shape control. The method includes: obtaining point cloud data of the flexible fiber optic body and pose data of the end effector of a robotic arm; inputting a trained PIGNN model and outputting a shape prediction of the flexible fiber optic body; and using a model predictive control method to calculate the required motion trajectory of the robotic arm, thereby enabling the robotic arm to control the flexible fiber optic body to reach the target shape position. This invention, through the PIGNN model-predicted shape changes strictly following the mechanical laws of the flexible fiber optic material, can significantly improve the consistency and reliability of fiber optic sensors. Model predictive control can dynamically adjust the robotic arm trajectory in real time based on the shape prediction, flexibly adapting to the processing requirements of different fiber optic specifications. The combination of physical constraints and data-driven approaches enables the system to quickly adapt to new scenarios, laying a core technological foundation for the large-scale, high-precision application of fiber optic sensors.

Description

Technical Field

[0001] This invention relates to an embodied intelligence method for shape control during the operation of flexible fiber bodies, belonging to the field of flexible body shape control. Background Technology

[0002] In the modern industrial and technological fields, fiber optic sensors have become an indispensable core component in many key areas due to their significant advantages such as high precision, resistance to electromagnetic interference, and tolerance to harsh environments. They are widely used in energy, medical, aerospace, and intelligent security scenarios, and their market size and technological influence occupy a pivotal position among various types of sensors.

[0003] In the production process of fiber optic sensors, optical path connection is the core link that determines product performance and quality. This process includes many precision procedures such as fiber stripping, cleaning, cutting, splicing and coiling. The accuracy of each step directly affects the signal transmission efficiency and stability of the sensor.

[0004] However, these key processes still primarily rely on manual operation. Manual operation is not only inefficient and ill-suited to large-scale production demands, but it is also susceptible to problems such as fiber stripping damage, cutting angle deviations, and splicing bubbles, which are easily caused by factors like operator skill level, physical condition, and emotional fluctuations. This severely restricts the consistency and reliability of fiber optic sensors. This traditional production model can no longer meet the market's urgent need for high-performance, low-cost fiber optic sensors, becoming a key bottleneck hindering their further marketization and industrial upgrading. Summary of the Invention

[0005] To address the problems of low efficiency and poor quality in current manual operations involving flexible fiber bodies, this invention provides an embodied intelligent method for shape control during the operation of flexible fiber bodies. The technical solution is as follows:

[0006] Step 1: During the random movement of the end effector of the robotic arm holding the end of the flexible fiber, obtain the point cloud data of the flexible fiber and the pose data of the end effector of the robotic arm.

[0007] Step 2: Input the point cloud data of the fiber flexible body and the pose data of the end effector of the robotic arm into the trained physical information graph neural network model PIGNN, and the PIGNN model outputs the shape prediction of the fiber flexible body;

[0008] Step 3: Based on the shape prediction, the model predictive control method is used to calculate the motion trajectory required by the robotic arm, so that the robotic arm controls the flexible fiber to reach the target shape position.

[0009] Optionally, the training process of the PIGNN model includes:

[0010] Step 201: Based on the fiber flexible body point cloud data, calculate the position of key points in the fiber flexible body point cloud data and the pose data of the corresponding robotic arm end effector using Gaussian mixture model and Bayesian inference to construct a training image dataset;

[0011] Step 202: Use the Neo-Hookean model to simulate the deformation of the fiber flexible body and derive the partial differential equation of the deformation dynamics of the fiber flexible body;

[0012] Step 203: Train the PIGNN model using the training image dataset. During training, the images first pass through an encoding module. G Projection to low-dimensional representation , where the vertex V The position sequence of the corresponding fiber optic key points, edge E The interaction between key points is then calculated; the interaction between vertices in the latent graph is then calculated by the processor module to simulate the dynamics of the fiber flexible body; finally, the latent representation obtained by the decoder module is restored to the prediction of the original fiber flexible body motion.

[0013] Step 204: The displacement of the fiber flexible body predicted by the PIGNN model is automatically differentiated and substituted into the partial differential equation of the deformation dynamics of the fiber flexible body to obtain the physical constraint loss. This physical constraint loss is then added to the data fitting loss as the total loss and fed back to the PIGNN model.

[0014] Optionally, the partial differential equation of the deformation dynamics of the fiber flexible body is expressed as:

[0015]

[0016] In the formula, Indicating that the flexible fiber body is in t Time-space coordinates x The displacement vector at that point T The traction force applied by the robot's end effector to the flexible fiber. E For Young's modulus, I Let the moment of inertia of the cross section be... This indicates the mass density of the flexible fiber. This represents the cross-sectional area of ​​the flexible fiber.

[0017] Optionally, the data fitting loss is expressed as:

[0018]

[0019] In the formula, It is the location of a key point on the fiber-reinforced flexible body predicted by the PIGNN model. These are the actual observation data for that point. It is the number of key points on the flexible fiber.

[0020] Optionally, the physical constraint loss is expressed as:

[0021]

[0022] In the formula, It is the position vector of the fiber-flexible body predicted by the PIGNN model. It represents the number of points sampled at different times and locations.

[0023] Optionally, the total loss is expressed as:

[0024]

[0025] in, This represents the data fitting loss. Represents physical constraint loss. and These are weight parameters.

[0026] Optionally, in step 203, the process of propagating graph vertex and edge features based on the current graph state and the control input of the robotic arm is represented as follows:

[0027]

[0028] In the formula, and These represent the graph vertex network and the graph edge network, respectively. , Indicates the current vertex and edge features. , Indicates the first k Potential vertex and edge features after the second message passing Represents the vertex i The sum of the characteristics of connected edges. and They are the edges Connecting two vertices i and j Its characteristics.

[0029] Optionally, in step 201, the perceived point cloud is regarded as a sample from the Gaussian mixture model, and sampling points are extracted from the mixture model according to Bayes' theorem. The probability is expressed as:

[0030]

[0031] In the formula, Indicates the first n The weights of the mixed components, Indicates sampling from the mixed components The corresponding probability.

[0032] Optionally, in step 202, an Abaqus simulation is used to construct a mechanism model of the fiber flexible body.

[0033] Optionally, step 1 uses a 3D camera to acquire point cloud data of the flexible fiber and pose data of the end effector of the robotic arm.

[0034] The beneficial effects of this invention are:

[0035] This invention integrates the physical properties of fiber optic flexible materials with graph neural networks through the PIGNN model. The predicted shape changes strictly follow the mechanical laws of fiber optic flexible materials, which can reduce operational errors and solve the quality instability problems such as fiber stripping damage and splicing bubbles in manual operation, thus greatly improving the consistency and reliability of fiber optic sensors.

[0036] Meanwhile, Model Predictive Control (MPC) can dynamically adjust the robotic arm trajectory in real time based on shape predictions, flexibly adapting to the processing needs of different fiber optic specifications and avoiding the drawbacks of repeated adjustments required by traditional manual methods. This intelligent solution also enables continuous, uninterrupted operation, improving operational efficiency and reducing material loss due to operational errors. Furthermore, the combination of physical constraints and data-driven approaches allows the system to quickly adapt to new scenarios, laying a core technological foundation for the large-scale, high-precision application of fiber optic sensors. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a flowchart of the shape control method for fiber flexible body operations according to the present invention.

[0039] Figure 2 This is a diagram of the PIGNN model structure in the shape control method for fiber flexible body operations of the present invention. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0041] Example 1:

[0042] This embodiment provides a shape control method for flexible fiber assembly operations; see [link to documentation]. Figure 1and Figure 2 The method includes:

[0043] Step 1: The end of the optical fiber is held by the end effector of the robotic arm and moved randomly to obtain point cloud data of the optical fiber image captured by the 3D camera and the pose of the end effector of the robotic arm.

[0044] Step 2: Input the real-time fiber optic image data and the pose data of the robotic arm end effector into the trained Physical Information Graph Neural Network (PIGNN) model, and the PIGNN model outputs a shape prediction of the fiber optic cable.

[0045] The PIGNN model is built on a Physical Information Neural Network (PINN). It constructs a physical model reflecting the physical properties of optical fibers within partial differential equations (PDEs), employs a Graph Neural Network (GNN) model to learn and simulate changes in fiber shape, and introduces physical laws as constraints to enable effective adjustment based on physical principles. Training this PIGNN model includes the following steps:

[0046] Step 201: Using the acquired fiber optic point cloud data, calculate the positions of keypoints through a Gaussian mixture model (GMM) and Bayesian inference. Then, correlate this with the pose data of the robotic arm's end effector to obtain the training image dataset. The perceived point cloud is considered as a sample from the GMM, with its centroid representing the keypoint position. According to Bayes' theorem, sampling points are extracted from the mixture model. The probability can be expressed as:

[0047]

[0048] In the formula, Indicates the first n The weights of the mixed components, Indicates sampling from the mixed components The corresponding probability. Then the problem is transformed into a maximum likelihood estimation (MLE) problem, where this embodiment aims to optimize the hybrid centroid in order to maximize the point cloud. The log-likelihood of the sample L:

[0049]

[0050] Step 202: Use Abaqus to simulate and construct a fiber optic mechanism model.

[0051] Optical fibers consist of a core, cladding, and coating, and are concentrically cylindrical. Commonly used silica-based optical fibers have cores and claddings composed of high-purity silica glass and a small amount of dopants. The dopants are used to make the refractive index of the core slightly higher than that of the cladding. The coating is mainly made of acrylate, silicone resin, and nylon in a certain proportion, used to protect the optical fiber from mechanical damage. This embodiment does not involve the influence of the cladding on the refractive index of the fiber core, but focuses on the material properties of the coating. The material of the coating can be used to construct the optical fiber model using hyperelastic materials, and the elastic deformation of hyperelastic materials has highly nonlinear characteristics. For simplicity, this embodiment uses the Neo-Hookean model to simulate optical fiber deformation, and the deformation dynamics of the flexible optical fiber can be described based on Euler-Bernoulli theory:

[0052]

[0053] In the formula, Indicates that the optical fiber is in t Time-space coordinates x The displacement vector at that point T The traction force applied to the optical fiber by the robot's end effector. E For Young's modulus, I Let the moment of inertia of the cross section be... This represents the restoring force of the optical fiber when it is under tension, which conforms to the basic laws of elasticity. It describes the bending resistance of optical fibers, which can effectively resist changes in the curvature of the optical fiber.

[0054] Step 203: Train the PIGNN model using the training image dataset. During training, the image first passes through the encoding module. G Projection to low-dimensional representation To improve efficiency, where the vertex V The position sequence of the corresponding fiber optic key points, edge E The interaction between key points is considered; then, the processor module calculates the interaction between vertices in the latent graph to simulate the dynamics of the optical fiber; finally, the decoder module restores the obtained latent representation to the original prediction of the optical fiber motion. The process of propagating the vertex and edge features of the graph based on the current graph state and the control input of the robotic arm is expressed as follows:

[0055]

[0056] In the formula, and These represent the graph vertex network and the graph edge network, respectively. , Indicates the current vertex and edge features. , Indicates the first kPotential vertex and edge features after the next message passing. Represents the vertex i The sum of the characteristics of connected edges. and They are the edges Connecting two vertices i and j Its characteristics.

[0057] Step 204: The fiber displacement predicted by the GNN model is substituted into the partial differential equation of deformation dynamics of the fiber optic flexible body through automatic differentiation to obtain the calculated physical constraint loss. This loss is then added to the data fitting loss as the total loss and fed back to the PIGNN model to ensure that the model can not only learn from the data but also follow physical laws. The data fitting loss is used to ensure that the PIGNN can fit the actual observation data. In this embodiment, the data comes from the fiber displacement measured by sensors. The data fitting loss can be defined as:

[0058]

[0059] In the formula, It is the location of a key point on the optical fiber predicted by the model. These are the actual observation data for that point. This represents the number of keypoints on the optical fiber. We handle this loss using a minimization method to approximate the experimental data.

[0060] To incorporate fiber deformation dynamics information into PIGNN, a physical constraint loss is needed to ensure that the graph neural network predictions satisfy the partial differential equations of fiber deformation dynamics. Using these equations as constraints, the physical loss can be defined as:

[0061]

[0062] In the formula, It is the fiber optic position vector predicted by the neural network. It is the number of points sampled at different times and locations, through which the residuals of the partial differential equation system are calculated.

[0063] The total loss function is a weighted sum of the data fitting loss and the physical constraint loss:

[0064]

[0065] In the formula, and It is a weighting parameter used to balance the contributions of data fitting and physical constraints.

[0066] Step 3: Obtain the shape prediction of the optical fiber output by the PIGNN model, and use the model predictive control (MPC) method to calculate the motion trajectory required by the robotic arm, so that the robotic arm can accurately control the flexible optical fiber to reach the target shape and position.

[0067] This invention integrates the physical properties of fiber optic flexible materials with graph neural networks through the PIGNN model. The predicted shape changes strictly follow the mechanical laws of fiber optic flexible materials, which can reduce operational errors and solve the quality instability problems such as fiber stripping damage and splicing bubbles in manual operation, thus greatly improving the consistency and reliability of fiber optic sensors.

[0068] Meanwhile, Model Predictive Control (MPC) can dynamically adjust the robotic arm trajectory in real time based on shape prediction, flexibly adapting to the processing needs of different fiber optic specifications and avoiding the drawbacks of repeated adjustments required by traditional manual methods. This intelligent solution also enables continuous and uninterrupted operation, improving operational efficiency and reducing material loss due to operational errors. Furthermore, the combination of physical constraints and data-driven approaches allows the system to quickly adapt to new scenarios without extensive reprogramming, laying a core technological foundation for the large-scale, high-precision application of fiber optic sensors.

[0069] Some steps in the embodiments of the present invention can be implemented using software, and the corresponding software program can be stored in a readable storage medium, such as an optical disc or a hard disk.

[0070] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A body-aware method for shape control of a fiber flexible body during operation, characterized by, The method comprises: Step 1: obtaining point cloud data of the fiber flexible body and pose data of the end gripper of the mechanical arm in the process of random movement of the end gripper of the mechanical arm clamping the fiber flexible body; Step 2: inputting the point cloud data of the fiber flexible body and the pose data of the end gripper of the mechanical arm into a trained physical information graph neural network model PIGNN, and outputting a shape prediction of the fiber flexible body by the PIGNN model; Step 3: based on the shape prediction, a model predictive control method is used to calculate the required motion trajectory of the mechanical arm, so that the mechanical arm controls the fiber flexible body to reach the target shape position; In the training process of the PIGNN model, the deformation dynamics partial differential equation of the fiber flexible body is represented as: wherein denotes the displacement vector of the fiber flexible body at the spatial coordinate t at the time instant x , T is the traction force applied by the robot end to the fiber flexible body, E is the Young's modulus, I is the cross-sectional moment of inertia, denotes the mass density of the fiber flexible body, denotes the cross-sectional area of the fiber flexible body; In the training process, a physical constraint loss is introduced, which is represented as: In the formula, is the fiber soft body position vector predicted by the PIGNN model, is the number of points sampled at different times and positions.

2. The embodied intelligence method for shape control during operation of fiber flexible body according to claim 1, characterized by, The training process of the PIGNN model comprises: Step 201: based on the fiber flexible body point cloud data, the positions of the key points of the fiber flexible body point cloud data are calculated by a Gaussian mixture model and Bayesian inference, and the pose data of the corresponding end gripper of the mechanical arm are used to construct a training image data set; Step 202: a Neo-Hookean model is used to simulate the deformation of the fiber flexible body, and a deformation dynamics partial differential equation of the fiber flexible body is derived; Step 203: training the PIGNN model with the training image dataset, first passing the image G Projecting to low-dimensional representation where the vertices V correspond to the position sequence of the key points of the fiber, and the edges E correspond to the interaction between the key points; then through the processor module, the interaction between the vertices in the latent graph is calculated to simulate the dynamics of the fiber flexible body; finally, the decoder module is used to restore the obtained latent representation to the prediction of the original fiber flexible body motion; Step 204: the fiber flexible body displacement predicted by the PIGNN model is substituted into the deformation dynamics partial differential equation of the fiber flexible body by automatic differentiation to obtain a physical constraint loss, and then a data fitting loss is added as a total loss, which is fed back to the PIGNN model.

3. The embodied intelligence method for shape control during operation of fiber flexible body according to claim 2, characterized by, The data fitting loss is represented as: wherein, is the position of a certain key point on the fiber flexible body predicted by the PIGNN model, is the actual observation data of the point, is the number of key points on the fiber flexible body.

4. The embodied intelligence method for shape control during operation of fiber flexible body according to claim 2, characterized by, The total loss is represented as: wherein, represents a data fitting loss, represents a physical constraint loss, and is a weight parameter.

5. The embodied intelligence method for shape control during operation of fiber flexible body according to claim 2, characterized by, In the step 203, the process of propagating the graph vertex and edge features based on the current graph state and the control input of the mechanical arm is represented as: wherein, and denote the graph vertex network and the graph edge network, respectively, , denote the current vertex and edge features, , denote the latent vertex and edge features after the k-th message passing, k , denotes the sum of the features of the edges connected to the vertex i , and are the features of the two vertices connected by the edge i and j , respectively.

6. The embodied intelligence method for shape control during operation of fiber flexible body according to claim 2, characterized by, The step 201 regards the perceived point cloud as samples from a Gaussian mixture model, and according to Bayes' theorem, samples are drawn from the mixture model The probability is expressed as: In the formula, Indicates the first n The weights of the mixed components, Indicates sampling from the mixed components The corresponding probability.

7. The embodied intelligence method for shape control during operation of fiber flexible body according to claim 2, characterized by, In the step 202, a fiber flexible body mechanism model is constructed by using abaqus simulation.

8. The embodied intelligence method for shape control during operation of fiber flexible body according to claim 1, characterized by, In the step 1, the point cloud data of the fiber flexible body and the pose data of the end gripper of the mechanical arm are collected by using a three-dimensional camera.

Images