Soft mechanical arm kinematics modeling method based on convolutional neural network

Abstract

The invention relates to the technical field of neural network optical fiber measurement, and discloses a soft mechanical arm kinematics modeling method based on a convolutional neural network, which comprises the following steps: step 1, optical fiber curvature reconstruction: detecting the curvature and the curvature deflection angle of a soft mechanical arm through an optical fiber bragg grating sensor; when the soft mechanical arm is bent, the center wavelength of the fiber bragg grating sensor drifts, and curvature and angle information of an optical fiber is calculated based on the drifting distance. 2, three-dimensional position coordinates of the soft arm are obtained; step 3, training a convolutional neural network; and 4, the kinematics of the soft mechanical arm is predicted. Inverse kinematics modeling of the soft arm can be completed only on the basis of curvature information obtained from a fiber bragg grating (FBG) sensor and position information obtained from a visual sensor by adopting a convolutional neural network mode, and the soft arm inverse kinematics modeling method has very high generalization ability and is suitable for soft mechanical arms of different shapes, sizes and materials.

Description

Technical Field

[0001] The present invention relates to the technical field of neural network optical fiber measurement, and in particular to a kinematic modeling method of a soft robotic arm based on a convolutional neural network. Background Art

[0002] A soft robotic arm is a continuum robot made of flexible materials. Due to its strong flexibility and adaptability, it can move flexibly in a small and complex environment and interact safely with humans. It is widely used in medical, rescue, industrial and other fields. However, the kinematic and dynamic models of soft robotic arms are highly nonlinear and complex, which is due to the flexibility and continuum structure of its materials. Therefore, the accurate modeling and control of soft robotic arms have always been a research problem in this field. Traditional kinematic modeling methods for soft robotic arms usually rely on geometric models and physical properties. This type of method constructs a kinematic model of the soft arm by accurately establishing the geometric structure, material properties and motion equations of the soft arm. The main process of this modeling method includes: discretizing the shape of the soft arm, assuming that each part of the soft arm can be described by corresponding geometric parameters, and establishing a mathematical model based on these parameters. Finally, the motion trajectory of the soft arm is obtained by solving the motion equations.

[0003] Although the method based on geometric model can theoretically describe the motion of soft arms, it has the following significant drawbacks in practical applications:

[0004] High complexity: Due to the flexibility and continuum characteristics of the soft robotic arm, its motion is often nonlinear and highly complex. In order to build an accurate kinematic model, the geometric shape of the soft arm must be complexly modeled and calculated, involving a large number of geometric parameters and physical properties. This greatly increases the difficulty and time cost of modeling.

[0005] Limited accuracy: In practical applications, in order to reduce computational complexity, the geometric model of the soft arm is usually simplified or approximated. However, this simplification will inevitably lead to a decrease in modeling accuracy, especially under complex motion conditions, where existing models are difficult to accurately predict the end position of the soft arm.

[0006] Poor model generalization ability: Due to the differences in design, structure and material, each soft robotic arm often requires a separate kinematic model. The geometric model-based method is highly dependent on the structure of the soft arm, resulting in poor model generalization ability. The model must be readjusted or rebuilt for each specific robotic arm and cannot be applied to soft robotic arms of different structures and materials. Summary of the invention

[0007] In view of the shortcomings of the prior art, the present invention provides a kinematic modeling method for a soft robotic arm based on a convolutional neural network, which solves the problems that traditional kinematic modeling methods for soft robotic arms generally rely on geometric models and physical properties, resulting in high complexity, limited accuracy and poor model generalization ability.

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions: A method for kinematic modeling of a soft robotic arm based on a convolutional neural network, comprising the following steps:

[0009] Step 1: Fiber curvature reconstruction

[0010] The curvature and curvature deflection angle of the soft robotic arm are detected by the fiber Bragg grating sensor. When the soft robotic arm is bent, the central wavelength of the fiber Bragg grating sensor drifts, and the curvature and angle information of the optical fiber are calculated based on the drift amount. The relationship between the curvature and the central wavelength drift amount is:

[0011]

[0012] where λ B Indicates the wavelength of reflected light, Δλ B is the center wavelength shift, P e is the elastic-optic coefficient, ε is the axial strain on the optical fiber, ξ is the thermo-optic coefficient, α T is the coefficient of thermal expansion, and ΔT is the change in ambient temperature.

[0013] Without considering torsion, a section of arc length ds in the optical fiber is selected to construct a pure bending model, and the relationship between the optical fiber strain and curvature is:

[0014]

[0015] Where, ds is the arc length of the optical fiber, dl is the arc length of the neutral plane, k is the curvature, ρ is the radius of curvature, and δ is the distance from the paraxial core to the neutral axis;

[0016] Step 2: Get the 3D position coordinates of the software arm

[0017] Arrange several reflective marking points on the soft robotic arm, and obtain the three-dimensional coordinate information of the soft robotic arm through the visual sensor;

[0018] Step 3: Training of Convolutional Neural Network

[0019] Based on the curvature obtained by the optical fiber sensor and the three-dimensional coordinate information obtained by the visual sensor, a convolutional neural network is used for model training to establish the kinematic model of the soft robotic arm;

[0020] Step 4: Prediction of the kinematics of the soft robotic arm

[0021] Use the trained convolutional neural network model to predict the spatial position of the soft robotic arm by inputting curvature information.

[0022] Preferably, the fiber Bragg grating sensors are arranged at multiple bending positions of the soft robotic arm to detect the curvature changes at different positions and obtain the overall deformation information of the soft robotic arm.

[0023] Preferably, the convolutional neural network includes at least one convolution layer and one pooling layer, wherein the convolution layer is used to extract local features from the input curvature and curvature deflection angle, and the pooling layer is used to perform dimensionality reduction processing on the features to reduce computational complexity and avoid overfitting.

[0024] Preferably, the visual sensor is fixedly mounted outside the working space of the soft robotic arm to capture the three-dimensional spatial coordinate information of the reflective marking points on the soft robotic arm.

[0025] Preferably, the loss function of the convolutional neural network uses mean square error to calculate the difference between the three-dimensional position coordinates output by the network and the actual measured position coordinates, and updates the network parameters through a back propagation algorithm.

[0026] Preferably, the convolution kernel of the convolutional neural network extracts multi-dimensional curvature information features in each convolution operation through a parameter sharing mechanism.

[0027] Preferably, the fiber Bragg grating sensor detects the bending curvature of the soft robot arm and calculates the curvature information through the wavelength drift, and the calculation process includes the formula:

[0028]

[0029]

[0030] Among them, λ B Indicates the wavelength of reflected light, Δλ B is the center wavelength shift, P e is the elastic-optic coefficient, ε is the axial strain on the optical fiber, ξ is the thermo-optic coefficient, α T is the thermal expansion coefficient, ΔT is the change in ambient temperature, ds is the arc length of the optical fiber, dl is the arc length of the neutral plane, k is the curvature, ρ is the radius of curvature, and δ is the distance from the paraxial core to the neutral axis.

[0031] Preferably, when the convolutional neural network is trained, the input curvature data and three-dimensional coordinate data are normalized.

[0032] Preferably, the soft robotic arm kinematic modeling method based on convolutional neural network is suitable for soft robotic arms with different geometric shapes and flexible materials, and can be generalized to different types of soft robotic arm kinematic modeling scenarios.

[0033] The present invention provides a method for kinematic modeling of a soft robotic arm based on a convolutional neural network, which has the following beneficial effects:

[0034] 1. The present invention adopts a convolutional neural network approach, and can complete the inverse kinematic modeling of the soft arm based only on the curvature information obtained from the fiber Bragg grating (FBG) sensor and the position information obtained from the visual sensor. This method does not require prior knowledge of the geometric structure of the soft robotic arm, does not rely on complex physical modeling, has strong generalization ability, and is suitable for soft robotic arms of different shapes, sizes and materials. Therefore, it has wide applicability in a variety of soft robotic arm application scenarios and can significantly reduce the complexity of modeling.

[0035] 2. The characteristics of the convolutional neural network make the model of the present invention relatively lightweight. The network extracts features from the data through the convolution layer, and the parameter sharing mechanism reduces the number of parameters of the model, thereby reducing the computational cost. This design not only speeds up the training speed of the model, but also enables the kinematic model of the soft arm to be predicted in a relatively short time. Therefore, in the application of real-time control and rapid response, it has higher computational efficiency.

[0036] 3. The present invention can effectively process complex nonlinear mapping relationships through the curvature information obtained by the optical fiber sensor and the nonlinear fitting ability of the convolutional neural network, and realize high-precision prediction of the position of the soft robotic arm. Experiments show that this method can maintain a low prediction error on soft robotic arms of different shapes and structures, improving the accuracy of the overall system.

[0037] 4. The fiber Bragg grating sensor used in the present invention can work normally in a variety of complex environments due to its high sensitivity and strong anti-interference ability. Compared with traditional electromagnetic sensors or visual sensors that are easily affected by environmental factors, the fiber optic sensor of the present invention can avoid problems such as visual occlusion and electromagnetic interference, making the acquisition of motion data of the soft robot arm more stable and reliable, and is particularly suitable for application scenarios in outdoor or complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 The fiber Bragg grating structure diagram of the present invention;

[0039] Figure 2 It is the FBG stretching and compression model diagram under bending conditions of the present invention;

[0040] Figure 3 This is a flow chart of the training of the soft arm kinematic model of the present invention. DETAILED DESCRIPTION

[0041] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] Please see attached Figure 1 -Attached Figure 3 The embodiment of the present invention provides a method for kinematic modeling of a soft robotic arm based on a convolutional neural network, comprising the following steps:

[0043] Step 1: Fiber curvature reconstruction

[0044] The curvature and curvature deflection angle of the soft robotic arm are detected by the fiber Bragg grating sensor. When the soft robotic arm is bent, the central wavelength of the fiber Bragg grating sensor drifts, and the curvature and angle information of the optical fiber are calculated based on the drift amount. The relationship between the curvature and the central wavelength drift amount is:

[0045]

[0046] where λ B Indicates the wavelength of reflected light, Δλ B is the center wavelength shift, P e is the elastic-optic coefficient, ε is the axial strain on the optical fiber, ξ is the thermo-optic coefficient, α T is the coefficient of thermal expansion, and ΔT is the change in ambient temperature.

[0047] Without considering torsion, a section of arc length ds in the optical fiber is selected to construct a pure bending model, and the relationship between the optical fiber strain and curvature is:

[0048]

[0049] Where, ds is the arc length of the optical fiber, dl is the arc length of the neutral plane, k is the curvature, ρ is the radius of curvature, and δ is the distance from the paraxial core to the neutral axis;

[0050] Step 2: Get the 3D position coordinates of the software arm

[0051] Arrange several reflective marking points on the soft robotic arm, and obtain the three-dimensional coordinate information of the soft robotic arm through the visual sensor;

[0052] Step 3: Training of Convolutional Neural Network

[0053] Based on the curvature obtained by the optical fiber sensor and the three-dimensional coordinate information obtained by the visual sensor, a convolutional neural network is used for model training to establish the kinematic model of the soft robotic arm;

[0054] Step 4: Prediction of the kinematics of the soft robotic arm

[0055] Use the trained convolutional neural network model to predict the spatial position of the soft robotic arm by inputting curvature information.

[0056] Specifically, fiber curvature reconstruction

[0057] Fiber Bragg Grating (FBG) sensors are installed on the soft robotic arm and arranged at multiple bending parts of the flexible arm. The FBG sensor detects the bending state through the drift of the optical fiber wavelength. When the soft robotic arm is bent, the central wavelength of the optical fiber will change, and the drift is used to calculate the curvature and deflection angle of the optical fiber. The calculation formula is:

[0058]

[0059] where λ B Indicates the wavelength of reflected light, Δλ B is the center wavelength shift, P e is the elastic-optic coefficient, ε is the axial strain on the optical fiber, ξ is the thermo-optic coefficient, α T is the coefficient of thermal expansion, and ΔT is the change in ambient temperature.

[0060] Without considering torsion, a section of arc length ds in the optical fiber is selected to construct a pure bending model, and the relationship between the optical fiber strain and curvature is:

[0061]

[0062] Where, ds is the arc length of the optical fiber, dl is the arc length of the neutral plane, k is the curvature, ρ is the radius of curvature, and δ is the distance from the paraxial core to the neutral axis;

[0063] Acquisition of 3D coordinate data

[0064] Several reflective marking points are arranged on the soft robotic arm, and the 3D coordinate information of the soft robotic arm is obtained in real time through an externally installed high-definition camera (visual sensor). The camera is fixedly installed outside the working space of the soft arm to ensure that the motion trajectory of the reflective marking points can be captured. The collected 3D coordinate information will be used for the output label of the convolutional neural network;

[0065] Training of Convolutional Neural Networks

[0066] The input of the convolutional neural network is the curvature data obtained by the fiber optic sensor, and the output is the three-dimensional coordinates collected by the visual sensor. The structure of the CNN includes a convolution layer and a pooling layer, where the convolution layer is used to extract the local features of the curvature information, and the pooling layer is used to reduce the feature dimension to prevent overfitting. The network uses the mean square error (MSE) as the loss function to minimize the error between the predicted coordinate value and the actual measured value. After training, the network can predict the spatial position of the soft robot arm through curvature;

[0067] Predictions from the kinematic model

[0068] Using a trained convolutional neural network model, the curvature information detected by the fiber optic sensor is input in real time, and the spatial position coordinates of the soft arm are output to form a kinematic model prediction of the soft arm.

[0069] Fiber Bragg grating sensors are arranged at multiple bending parts of the soft robotic arm to detect the curvature changes at different positions and obtain the overall deformation information of the soft robotic arm.

[0070] Specifically, FBG sensors are installed at multiple bending parts of the soft robotic arm to obtain the curvature changes of each bending part. Each sensor is responsible for detecting the curvature of the optical fiber at a specific position, thereby accurately recording the deformation process of the entire soft robotic arm. Through a high-precision demodulator, the center wavelength drift of each sensor can be read to obtain the curvature information of each bending part; the fiber Bragg grating sensor is embedded in the external flexible material of the soft robotic arm through a fixing device, and can bend synchronously with the bending of the robotic arm to maintain the stability of the sensor and the accuracy of the data.

[0071] The convolutional neural network contains at least one convolution layer and one pooling layer, wherein the convolution layer is used to extract local features from the input curvature and curvature deflection angle, and the pooling layer is used to reduce the dimension of the features, reduce the computational complexity and avoid overfitting.

[0072] Specifically, the specific design of the convolutional neural network is as follows:

[0073] Convolutional layer: used to extract local features from input data (fiber curvature and deflection angle). The size of the convolution kernel of each layer is adjusted according to actual needs. For example, a 3×3 convolution kernel size can be selected. The purpose of the convolutional layer is to reduce the reliance on manual feature selection and automatically extract the most effective curvature features.

[0074] Pooling layer: A pooling layer is set after each convolutional layer, using maximum pooling or average pooling operations to reduce the dimension of the feature map, reduce the complexity of data calculation, and reduce the risk of overfitting.

[0075] Fully connected layer: After the convolutional layer and the pooling layer, the data is mapped through the fully connected layer to convert the extracted features into specific outputs (3D coordinate positions).

[0076] The visual sensor is fixedly installed outside the working space of the soft robotic arm to capture the three-dimensional spatial coordinate information of the reflective marking points on the soft robotic arm.

[0077] Specifically, the high-definition visual sensor is installed outside the working area of ​​the soft robotic arm and can be fixed in a stable position to ensure the capture of the complete movement of the soft arm. The visual sensor detects the three-dimensional position of the soft arm in space through pre-set reflective markers. The number and position of the markers are set according to the structure and length of the soft robotic arm to ensure that there are enough reflective points to effectively capture the movement of the robotic arm; each time the robotic arm moves, the visual sensor will synchronously obtain the three-dimensional coordinate data of all the markers, which will be used as output labels in the training of the convolutional neural network model to establish the inverse kinematics model.

[0078] The loss function of the convolutional neural network uses the mean square error to calculate the difference between the three-dimensional position coordinates output by the network and the actual measured position coordinates, and updates the network parameters through the back-propagation algorithm.

[0079] Specifically, during the training process of the convolutional neural network, the curvature and three-dimensional coordinate data are first normalized to ensure the numerical stability of the input and output data during the training process. Normalization can eliminate the differences between different data distributions, making the model training process more efficient; the network optimizes the parameters through the back-propagation algorithm, and the loss function uses the mean square error (MSE) to calculate the error between the output three-dimensional coordinates and the actual three-dimensional coordinates, and updates the network parameters through the gradient descent method. The division of the training set and the validation set will be used to monitor the convergence and generalization ability of the network to avoid overfitting.

[0080] The convolution kernel of the convolutional neural network extracts multi-dimensional curvature information features in each convolution operation through a parameter sharing mechanism.

[0081] Specifically, in a convolutional neural network, the convolution kernel of each layer slides on the input feature map through a parameter sharing mechanism to reduce the number of parameters in the network. This mechanism not only improves the training efficiency of the network, but also effectively reduces the risk of overfitting, and is particularly suitable for processing multi-dimensional data (such as curvature and deflection angle).

[0082] The fiber Bragg grating sensor detects the bending curvature of the soft robot arm and calculates the curvature information through the wavelength drift. The calculation process includes the formula:

[0083]

[0084] where λB Indicates the wavelength of reflected light, Δλ B is the center wavelength shift, P e is the elastic-optic coefficient, ε is the axial strain on the optical fiber, ξ is the thermo-optic coefficient, α T is the coefficient of thermal expansion, and ΔT is the change in ambient temperature.

[0085] Without considering torsion, a section of arc length ds in the optical fiber is selected to construct a pure bending model, and the relationship between the optical fiber strain and curvature is:

[0086]

[0087] Among them, ds is the arc length of the optical fiber, dl is the arc length of the neutral plane, k is the curvature, ρ is the radius of curvature, and δ is the distance from the paraxial core to the neutral axis.

[0088] Specifically, the wavelength drift detected by the fiber Bragg grating sensor can accurately reflect the bending curvature of the soft robotic arm. According to the formula:

[0089]

[0090] By using the wavelength drift value obtained by the sensor, the deformation at the corresponding position can be calculated, and then the curvature information of the soft arm in different bending states can be obtained.

[0091] When training the convolutional neural network, the input curvature data and three-dimensional coordinate data are normalized.

[0092] Specifically, in order to ensure the data stability of the convolutional neural network during the training process, the curvature data and three-dimensional coordinate data collected by the sensor are normalized. Normalization can limit the input data to a specific range, thereby accelerating the convergence of the model and avoiding the problem of gradient disappearance or explosion.

[0093] The kinematic modeling method of soft robotic arms based on convolutional neural networks is suitable for soft robotic arms with different geometric shapes and flexible materials, and can be generalized to different types of soft robotic arm kinematic modeling scenarios.

[0094] Specifically, the convolutional neural network modeling method of the present invention is applicable to soft robotic arms of different geometric shapes and materials. Regardless of the size, shape or material composition of the soft arm, the proposed model can be trained by adjusting the arrangement of sensors and the structure of the convolutional network, thereby generalizing to different types of soft arm scenarios.

[0095] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A kinematic modeling method for a soft robotic arm based on a convolutional neural network, characterized in that: The following steps are involved: Step 1: Fiber curvature reconstruction The curvature and curvature deflection angle of the soft robotic arm are detected by the fiber Bragg grating sensor. When the soft robotic arm is bent and deformed, the central wavelength of the fiber Bragg grating sensor will also drift. The curvature and angle information of the optical fiber are calculated based on the drift amount. The relationship between the curvature and the central wavelength drift amount is: where λ B Indicates the wavelength of reflected light, Δλ B is the center wavelength shift, P e is the elastic-optic coefficient, ε is the axial strain on the optical fiber, ξ is the thermo-optic coefficient, α T is the coefficient of thermal expansion, and ΔT is the change in ambient temperature. Without considering torsion, a section of arc length ds in the optical fiber is selected to construct a pure bending model, and the relationship between the optical fiber strain and curvature is: Where ds is the arc length of the optical fiber, dl is the arc length of the neutral plane, k is the curvature, ρ is the radius of curvature, and δ is the distance from the paraxial core to the neutral axis; Step 2: Get the 3D position coordinates of the software arm Arrange several reflective marking points on the soft robotic arm, and obtain the three-dimensional coordinate information of the soft robotic arm through the visual sensor; Step 3: Training of Convolutional Neural Network Based on the curvature obtained by the optical fiber sensor and the three-dimensional coordinate information obtained by the visual sensor, a convolutional neural network is used for model training to establish the kinematic model of the soft robotic arm; Step 4: Prediction of the kinematics of the soft robotic arm Use the trained convolutional neural network model to predict the spatial position of the soft robotic arm by inputting curvature information.

2. The method for kinematic modeling of a soft robotic arm based on a convolutional neural network according to claim 1, characterized in that: The fiber Bragg grating sensors are arranged at multiple bending positions of the soft robotic arm to detect the curvature changes at different positions and obtain the overall deformation information of the soft robotic arm.

3. The method for kinematic modeling of a soft robotic arm based on a convolutional neural network according to claim 1, characterized in that: The convolutional neural network includes at least one convolution layer and one pooling layer, wherein the convolution layer is used to extract local features from the input curvature and curvature deflection angle, and the pooling layer is used to perform dimensionality reduction processing on the features to reduce computational complexity and avoid overfitting.

4. The method for kinematic modeling of a soft robotic arm based on a convolutional neural network according to claim 1, characterized in that: The visual sensor is fixedly installed outside the working space of the soft robotic arm and is used to capture the three-dimensional spatial coordinate information of the reflective marking points on the soft robotic arm.

5. The method for kinematic modeling of a soft robotic arm based on a convolutional neural network according to claim 1, characterized in that: The loss function of the convolutional neural network uses mean square error to calculate the difference between the three-dimensional position coordinates output by the network and the actual measured position coordinates, and updates the network parameters through a back-propagation algorithm.

6. The method for kinematic modeling of a soft robotic arm based on a convolutional neural network according to claim 1, characterized in that: The convolution kernel of the convolutional neural network extracts multi-dimensional curvature information features in each convolution operation through a parameter sharing mechanism.

7. The method for kinematic modeling of a soft robotic arm based on a convolutional neural network according to claim 1, characterized in that: The fiber Bragg grating sensor detects the bending curvature of the soft robot arm and calculates the curvature information through the wavelength drift. The calculation process includes the formula: where λ B Indicates the wavelength of reflected light, Δλ B is the center wavelength shift, P e is the elastic-optic coefficient, ε is the axial strain on the optical fiber, ξ is the thermo-optic coefficient, α T is the coefficient of thermal expansion, and ΔT is the change in ambient temperature. Without considering torsion, a section of arc length ds in the optical fiber is selected to construct a pure bending model, and the relationship between the optical fiber strain and curvature is: Among them, ds is the arc length of the optical fiber, dl is the arc length of the neutral plane, k is the curvature, ρ is the radius of curvature, and δ is the distance from the paraxial core to the neutral axis.

8. The method for kinematic modeling of a soft robotic arm based on a convolutional neural network according to claim 1, characterized in that: When the convolutional neural network is trained, the input curvature data and three-dimensional coordinate data are normalized.

9. The method for kinematic modeling of a soft robotic arm based on a convolutional neural network according to claim 1, characterized in that: The soft robotic arm kinematic modeling method based on convolutional neural network is suitable for soft robotic arms with different geometric shapes and flexible materials, and can be generalized to different types of soft robotic arm kinematic modeling scenarios.

Images