Soft robot shape sensing method, apparatus, device and computer storage medium

Abstract

The application discloses a soft robot shape perception method, device, equipment and computer storage medium, relates to the shape perception technical field, and discloses a soft robot shape perception method, comprising: acquiring sensor collection data, wherein the sensor collection data includes acceleration, angular velocity and node length change value of the soft robot node;According to the acceleration, angular velocity and preset Kalman filter algorithm, the compensation attitude quaternion is determined, and the compensation attitude quaternion, the acceleration and the node length change value are trained in the preset neural network model, and the target neural network model is obtained;According to the target neural network model, the shape of the soft robot is perceived.The application improves the accuracy of soft robot shape perception.

Description

Technical Field

[0001] This application relates to the field of shape perception technology, and in particular to a shape perception method, apparatus, device, and computer storage medium for soft robots. Background Technology

[0002] With the development of shape perception technology and the application of soft robots, users have put forward higher requirements for the shape perception of soft robots.

[0003] Currently, inertial sensors are widely used in 3D motion tracking. They can be placed at various nodes of a soft robot to capture motion posture data in order to reconstruct deformable surfaces. However, this method of shape perception for soft robots has significant drawbacks. As the soft robot shrinks in length during actual deformation, a single IMU (Inertial Measurement Unit) sensor cannot capture all the deformation parameters. In other words, this method of shape perception for soft robots suffers from low accuracy because a single IMU sensor cannot capture all the deformation parameters.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this application is to provide a method, apparatus, device, and computer storage medium for shape perception of soft robots, aiming to solve the technical problem of low accuracy in shape perception of soft robots.

[0006] To achieve the above objectives, this application provides a soft robot shape perception method, which includes:

[0007] Acquire sensor data, wherein the sensor data includes the acceleration, angular velocity and inter-node length change values ​​of the soft robot nodes;

[0008] The compensation attitude quaternion is determined based on the acceleration, angular velocity, and a preset Kalman filter algorithm. The target neural network model is then trained in a preset neural network model based on the compensation attitude quaternion, the acceleration, and the change in length between nodes.

[0009] The soft robot performs shape perception based on the target neural network model.

[0010] In one embodiment, the step of determining the compensation attitude quaternion based on the acceleration, angular velocity, and a preset Kalman filter algorithm includes:

[0011] The angular velocity is preprocessed to obtain angular velocity processing data, and a quaternion attitude solution is performed based on the angular velocity processing data to obtain a state vector;

[0012] The acceleration is preprocessed to obtain acceleration processing data, which is used as a measurement. The compensation attitude quaternion is determined based on the state vector, the measurement, and a preset Kalman filter algorithm.

[0013] In one embodiment, the step of determining the compensation attitude quaternion based on the state vector, the measurement, and a preset Kalman filter algorithm includes:

[0014] The state equation in the preset Kalman filter algorithm is determined, and the output result of the state vector and the preset initial state vector in the state equation is determined as the predicted state vector.

[0015] The measurement equation in the Kalman filter algorithm is determined, and the output of the measurement and the predicted state vector in the measurement equation is used as the compensation attitude quaternion.

[0016] In one embodiment, the sensor data acquisition further includes actual position information acquired by the optical motion capture system, and the step of training a target neural network model in a preset neural network model based on the compensated attitude quaternion, the acceleration, and the inter-node length change value includes:

[0017] The compensated attitude quaternion, the acceleration, and the inter-node length change value are extracted and fused in a preset neural network model to obtain the predicted position information;

[0018] The neural network model is trained based on the predicted location information and the actual location information to obtain the target neural network model.

[0019] In one embodiment, the step of training the neural network model based on the predicted location information and the actual location information to obtain a target neural network model includes:

[0020] The loss error value between the predicted location information and the actual location information is determined based on a preset error calculation method, and the parameters of the neural network model are updated based on the loss error value.

[0021] Based on the updated parameters, the neural network model is used to extract and fuse the compensation posture quaternion, the acceleration, and the change in length between nodes in the preset neural network model to obtain the predicted position information. This process continues until the neural network model converges, at which point the updated neural network model is determined as the target neural network model.

[0022] In one embodiment, the step of performing shape perception on the soft robot based on the target neural network model includes:

[0023] The real-time predicted position of each node is determined in the target neural network model based on the real-time sensing data of each node of the soft robot.

[0024] The three-dimensional shape reconstruction of the soft robot is obtained by interpolation based on the real-time predicted position of the node.

[0025] In one embodiment, the real-time sensing data includes acceleration, angular velocity, and inter-node length change values. The step of determining the real-time predicted position of each node in the target neural network model based on the real-time sensing data of each node of the soft robot includes:

[0026] The target compensation attitude quaternion is determined based on the acceleration, the angular velocity, and the Kalman filter algorithm;

[0027] The outputs of the target compensation attitude quaternion, the acceleration, and the inter-node length change value in the target neural network model are determined as the real-time predicted positions of the nodes.

[0028] Furthermore, to achieve the above objectives, this application also provides a soft robot shape sensing device, the soft robot shape sensing device comprising:

[0029] The data acquisition module is used to acquire sensor-collected data, wherein the sensor-collected data includes the acceleration, angular velocity, and inter-node length change values ​​of the soft robot nodes;

[0030] The model training module is used to determine the compensation attitude quaternion based on the acceleration, angular velocity and the preset Kalman filter algorithm, and to train the target neural network model in the preset neural network model based on the compensation attitude quaternion, the acceleration and the length change value between nodes.

[0031] A shape perception module is used to perform shape perception on the soft robot based on the target neural network model.

[0032] In addition, to achieve the above objectives, this application also provides a soft robot shape perception device, including a processor, a memory, and a soft robot shape perception program stored in the memory that can be executed by the processor, wherein when the soft robot shape perception program is executed by the processor, it implements the steps of the soft robot shape perception method as described above.

[0033] This application also provides a computer storage medium storing a soft robot shape perception program, wherein when the soft robot shape perception program is executed by a processor, it implements the steps of the soft robot shape perception method as described above.

[0034] This application provides a shape perception method for soft robots. The method involves acquiring sensor data, including the acceleration, angular velocity, and inter-node length changes of the soft robot nodes. A compensation posture quaternion is determined based on the acceleration, angular velocity, and a preset Kalman filter algorithm. This compensation posture quaternion, along with the acceleration and inter-node length changes, is then used to train a target neural network model within a preset neural network model. The soft robot's shape is then perceived based on this target neural network model. This method avoids the problem of a single IMU sensor being unable to capture complete deformation parameters due to length contraction during actual deformation. This soft robot shape perception method, on the one hand, determines the compensation posture quaternion through acceleration, angular velocity, and a preset Kalman filter algorithm, thereby suppressing the problem of random error drift in gyroscope posture calculation. This improves the accuracy of soft robot shape perception by increasing parameter accuracy. On the other hand, it determines the shape of the soft robot based on the parameters of the soft robot at this time, rather than using a single IMU sensor (at least the length changes between nodes of the input target neural network model and the compensation posture quaternion are not directly acquired by the IMU sensor) for shape perception, thus improving the accuracy of soft robot shape perception. Attached Figure Description

[0035] Figure 1 This is a flowchart illustrating the first embodiment of the soft robot shape perception method of this application;

[0036] Figure 2 This is a flowchart illustrating the soft robot shape perception method of this application;

[0037] Figure 3 This is a schematic diagram of a training process for the soft robot shape perception method of this application;

[0038] Figure 4 This is a schematic diagram of the device module of the soft robot shape perception device of this application;

[0039] Figure 5 This is a schematic diagram of the hardware operating environment involved in the device in this application;

[0040] Figure 6 This is a schematic diagram of the sensor arrangement in the soft robot shape perception scheme of this application;

[0041] Figure 7 This is another schematic diagram of the sensor arrangement in the soft robot shape perception scheme of this application.

[0042] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0043] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0044] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0045] With the development of robotics technology, traditional rigid robots, due to their complex structure, limited flexibility, and the need for high safety and reliability, pose significant challenges in certain scenarios, such as grasping complex and fragile objects or operating in confined spaces. In recent years, with the development of new smart materials and 3D printing technology, soft robots have made breakthrough progress. Their bodies are processed using soft or flexible materials, possessing unlimited degrees of freedom, continuous deformation, and a large deformation range. Their inherent safety and compliance compensate for the shortcomings of rigid robots. Therefore, to achieve operation and interaction with soft robots, a core issue is shape perception, which relies on sensing technology to accurately measure and reconstruct the shape of the soft robot to achieve closed-loop control.

[0046] Currently, inertial sensors are widely used in 3D motion tracking due to their small size, light weight, portability, strong independence, and lack of site limitations and external interference. They can be placed at various nodes of soft robots to form an IMU sensor network to capture motion attitude data and reconstruct deformable surfaces. However, capturing motion attitude data based on inertial sensors has at least the following drawbacks: 1. Attitude calculations using gyroscopes in inertial sensors are easily affected by random error drift, resulting in poor stability of estimation accuracy; 2. Soft robots have infinite degrees of freedom and a large range of variability, making it difficult to establish accurate motion models to estimate the positions of each node; 3. Soft robots shrink in length during actual deformation, and a single inertial sensor cannot capture all deformation parameters.

[0047] Therefore, based on the above problems, this application proposes a soft robot shape perception method. By introducing multi-sensor fusion and combining it with grid-like spatial features, the accuracy of soft robot shape perception and reconstruction is improved. The main solution of this application is: to determine the compensation posture quaternion based on acceleration, angular velocity and a preset Kalman filter algorithm, and then to train the target neural network model based on the compensation posture quaternion, acceleration and the length change value between nodes in the neural network model to obtain the target neural network model. The soft robot's shape is then perceived based on the target neural network model, that is, the soft robot's parameters are input into the target neural network model to determine the shape of the soft robot. This soft robot shape perception method, on the one hand, determines the compensation posture quaternion through acceleration, angular velocity, and a preset Kalman filter algorithm, thereby suppressing the random error drift problem in gyroscope posture calculation. This improves the accuracy of soft robot shape perception by increasing parameter accuracy. On the other hand, it determines the shape of the soft robot based on the parameters of the soft robot at this time, rather than using a single IMU sensor (at least the length change values ​​between nodes of the input target neural network model and the compensation posture quaternion are not directly acquired by the IMU sensor) for shape perception, thus improving the accuracy of soft robot shape perception.

[0048] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or a device capable of performing the above functions, such as a soft robot shape sensing device. The following description uses a soft robot shape sensing device as an example to illustrate this embodiment and the subsequent embodiments.

[0049] Based on this, embodiments of this application provide a shape perception method for a soft robot, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the soft robot shape perception method of this application.

[0050] Reference Figure 1 This application provides a shape perception method for a soft robot, the shape perception method for a soft robot comprising:

[0051] Step S10: Acquire sensor data, wherein the sensor data includes the acceleration, angular velocity and length change between nodes of the soft robot node;

[0052] In this embodiment, sensor data is acquired, processed, and then used to train a neural network. After training, the neural network can perform shape perception on the soft robot based on input parameters. The sensor data includes the acceleration, angular velocity, and inter-node length changes of the soft robot's nodes. Acceleration and angular velocity refer to the three-axis acceleration and angular velocity of each node, while the inter-node length changes refer to the values ​​indicating contraction or stretching of the inter-node lengths during deformation. Training is then performed based on these acceleration, angular velocity, and inter-node length changes, and subsequent shape perception is also based on these values ​​after training. This avoids the problem of incomplete deformation parameters when using a single IMU sensor, thus improving the accuracy of the soft robot's shape perception.

[0053] In one embodiment, an IMU sensor network is deployed at each node of the soft robot or on its surface to collect acceleration and angular velocity data. Each module of the IMU sensor network may include a microcontroller unit (MCU) and an MPU6050. The MPU6050 includes a three-axis accelerometer and a three-axis gyroscope to collect X, Y, and Z-axis acceleration and angular velocity data. Simultaneously, strain sensors are used to collect the length changes between adjacent nodes of the soft robot. Each strain sensor is positioned between adjacent nodes. When the length between nodes contracts or stretches during deformation, the strain sensor outputs the corresponding length change as the inter-node length change value. (Refer to...) Figure 6 , Figure 6 This is a schematic diagram of the sensor arrangement in the soft robot shape perception scheme of this application. The IMU and strain sensor are arranged in a one-dimensional manner on the soft robot. Further, refer to... Figure 7 , Figure 7 This is another schematic diagram of the sensor arrangement in the soft robot shape perception scheme of this application. The IMU and strain sensor are arranged in a two-dimensional manner on the soft robot. The above are only two ways of arranging sensors in the soft robot shape perception scheme. Other arrangements are also possible and are not limited here.

[0054] In one embodiment, neural network training can be performed based on the acceleration, angular velocity, and inter-node length change of a single node, or it can be performed based on the acceleration, angular velocity, and inter-node length change of multiple nodes. For example, when training based on the quaternion and acceleration of node A, the inter-node length change of node A relative to other nodes should be selected, rather than an irrelevant inter-node length change.

[0055] Step S20: Determine the compensation attitude quaternion based on the acceleration, angular velocity and the preset Kalman filter algorithm, and train the target neural network model in the preset neural network model based on the compensation attitude quaternion, the acceleration and the change value between the node lengths;

[0056] In this embodiment, after determining the sensor-collected data, a compensation attitude quaternion is determined based on the acceleration, angular velocity, and a preset Kalman filter algorithm from the sensor-collected data. The resulting compensation attitude quaternion, acceleration, and inter-node length change values ​​are then used to train a preset neural network model to obtain the target neural network model. Here, the preset Kalman filter algorithm refers to a custom extended Kalman filter model (hereinafter referred to as EKF), the compensation attitude quaternion refers to the quaternion output by the acceleration and angular velocity in the preset Kalman filter algorithm, and the target neural network model refers to the model obtained by training based on the compensation attitude quaternion, acceleration, and inter-node length change values. The node positions are then determined based on the trained model, and the entire soft robot's 3D shape is reconstructed based on these determined positions, ensuring the accuracy of the soft robot's shape perception.

[0057] In one embodiment, the soft robot shape perception method is specifically referred to as the soft robot shape perception method that fuses IMU and strain sensors. On the one hand, it improves the accuracy of soft robot shape perception by fusing the acceleration, angular velocity, and changes in the connection length between nodes to obtain complete three-dimensional deformation parameters. On the other hand, for the information fusion of multi-source multimodal sensors (IMU and strain sensors), a neural network model is used to overcome the difficulty of establishing accurate motion models for soft robots, and effectively utilizes the spatial connection relationships and temporal characteristics of each node.

[0058] Step S30: Perform shape perception on the soft robot according to the target neural network model.

[0059] In this embodiment, after determining the target neural network model after training, shape perception is performed on the soft robot based on the target neural network model. Specifically, the compensated posture quaternions, acceleration, and length changes of each node are input into the target neural network model. The model then outputs the positions of each node, and interpolation is performed based on these determined node positions to achieve shape perception for the soft robot. Furthermore, the acceleration, angular velocity, and changes in connection length between nodes can be fused to obtain complete three-dimensional deformation parameters. The use of a neural network model for shape perception improves the accuracy of shape perception for the soft robot.

[0060] In one embodiment, reference is made to Figure 2 , Figure 2This is a flowchart illustrating the soft robot shape perception method of this application. After obtaining the attitude quaternion, acceleration, and length change based on the collected sensor information, the neural network is trained on the above data to obtain the required neural network (i.e., the target neural network). In this embodiment, the attitude quaternion is determined based on the node's acceleration, angular velocity, and Kalman filtering algorithm. When shape perception is required, new attitude quaternions, acceleration, and length change are input to the required neural network. The output value of the neural network at this time is the position of each node of the soft robot corresponding to the new attitude quaternion, acceleration, and length change. After determining the node positions, interpolation is performed to obtain the robot shape. That is, the node positions are connected according to the actual situation to obtain the shape of the entire soft robot, thereby realizing the soft robot shape perception. At the same time, it effectively integrates multi-source multimodal sensors (IMU and strain sensor), thereby improving the accuracy of soft robot shape perception.

[0061] Furthermore, based on the first embodiment of this application described above, a second embodiment of the soft robot shape perception method of this application is proposed. In this embodiment, step S20, the step of determining the compensation attitude quaternion based on the acceleration, angular velocity, and a preset Kalman filter algorithm, includes:

[0062] Step S201: Preprocess the angular velocity to obtain angular velocity processing data, and perform quaternion attitude calculation based on the angular velocity processing data to obtain the state vector;

[0063] Step S202: The acceleration is preprocessed to obtain acceleration processing data. The acceleration processing data is used as a measurement, and the compensation attitude quaternion is determined according to the state vector, the measurement and the preset Kalman filter algorithm.

[0064] In this embodiment, when determining the compensation attitude quaternion, the acceleration and angular velocity information in the velocity information are preprocessed first to obtain acceleration / angular velocity processed data. This preprocessing includes filtering and noise reduction. Angular velocity processed data refers to the data obtained after preprocessing angular velocity, and acceleration processed data refers to the data obtained after preprocessing acceleration. Then, the angular velocity processed data is used to perform quaternion attitude calculation to obtain a state vector. Simultaneously, the acceleration processed data is used as a measurement. The state vector is the vector obtained from the quaternion attitude calculation of the angular velocity processed data, and the measurement is the acceleration processed data. Finally, the compensation attitude quaternion is determined based on the state vector, the measurement, and a preset Kalman filter algorithm. This ensures the accuracy of the final shape perception by guaranteeing the accuracy of the input data.

[0065] In one embodiment, reference is made to Figure 3 , Figure 3This is a schematic diagram of the soft robot shape perception device of this application. The entire shape perception process does not directly process the data collected by the IMU to obtain the shape of the soft robot. Instead, it trains a neural network based on the data collected by the IMU, the processed data collected by the IMU, and the data from the strain sensor. After training, the neural network is used for shape perception. Figure 3 As shown, after determining the IMU data measurement, on the one hand, the pre-processed acceleration of the IMU is directly selected for neural network training; on the other hand, the acceleration and angular velocity collected by the IMU are pre-processed and fused using Kalman filtering. The angular velocity is used to calculate the quaternion attitude to obtain the state vector, and the acceleration is used as a measurement. Then, based on the state vector and the measurement, they are fused in the EKF to achieve drift error compensation output attitude quaternion. That is, the gyroscope and accelerometer are fused in the EKF to obtain the compensated attitude quaternion. This can avoid the problem that the attitude calculation of the gyroscope in the inertial sensor is easily affected by random error drift, resulting in poor stability of the estimation accuracy. By fusing the gyroscope and accelerometer in the EKF to achieve drift error compensation, the accuracy of shape perception of the soft robot is improved.

[0066] Further, the step of determining the compensation attitude quaternion based on the state vector, the measurement, and the preset Kalman filter algorithm includes:

[0067] Step S203: Determine the state equation in the preset Kalman filter algorithm, and determine the output result of the state vector and the preset initial state vector in the state equation as the predicted state vector.

[0068] Step S204: Determine the measurement equation in the Kalman filter algorithm, and determine the output results of the measurement and the predicted state vector in the Kalman filter as the compensation attitude quaternion.

[0069] In this embodiment, an IMU-EKF system model that fuses gyroscope and accelerometer data to calculate attitude is established to achieve drift error compensation. The established IMU-EKF system model is as follows:

[0070] x k+1 =F k x k +w k (1)

[0071] y k+1 =h k+1 (x k+1 )+v k+1 (2)

[0072] w k ∈(0,Q K(3)

[0073] v k ∈(0,R K )

[0074] Wherein, the state vector x k It is a 4*1 dimensional vector, x k+1 It is the state vector at time k+1, and F is the pose quaternion of each node. k It is the state transition matrix, y k+1 The measurement is taken at time k+1, with process noise w. k and measurement noise v k It is zero-mean Gaussian white noise, and the covariances are Q... k and R k Measurement equation It is a nonlinear function, where formula (3) represents a Gaussian distribution with zero mean noise.

[0075] In one embodiment, the extended Kalman filter algorithm (IMU-EKF system model) is used to estimate the quaternions and attitude angles of each node. By determining the state vector formula (formula (1)) in the preset Kalman filter algorithm, the output result of the state vector and the preset initial state vector in the state vector formula is used as the predicted state vector. The initial state vector is the set initial state vector. At this time, the initial state vector and the state vector, the measurement estimated covariance matrix, i.e. Q k and R k The estimation can be performed separately or together. The estimation method can be the same as that used in existing Kalman filtering, or it can be based on a unique covariance matrix corresponding to different state vectors and measurements. Then, the state vector at the next moment is determined based on the initial state vector as input, and finally, R is determined. k The measurement formula then determines the measurement value at the next moment, and thus the output value is used as the compensation attitude quaternion. This process can be repeated to determine the optimal value. By initializing the state variables and the covariance matrix of the state estimate, the optimal estimate of the system state is obtained after time and measurement updates based on the covariance matrix, serving as the compensation attitude quaternion. Accelerometer measurements are then used to compensate for the drift error in the gyroscope attitude calculation. The attitude quaternion obtained after EKF fusion of the IMU system is used as the input to the neural network model, effectively improving the accuracy of shape perception in the soft robot.

[0076] Furthermore, based on the first and / or second embodiments of this application described above, a third embodiment of the soft robot shape perception method of this application is proposed. In this embodiment, step S20, the step of training a target neural network model in a preset neural network model based on the compensated posture quaternion, the acceleration, and the inter-node length change value, includes:

[0077] Step S211: Extract and fuse the compensated attitude quaternion, the acceleration, and the inter-node length change value in a preset neural network model to obtain the predicted position information;

[0078] Step S212: Train the neural network model based on the predicted location information and the actual location information to obtain the target neural network model.

[0079] In this embodiment, the compensated posture quaternions, acceleration, and inter-node length change values ​​are extracted and fused in a preset neural network model to obtain predicted position information. Then, the neural network model is trained based on the predicted and actual position information to obtain the target neural network model. In other words, the neural network model is continuously trained to reduce the error between the predicted and actual position information, thus completing the neural network training. Predicted position information refers to the position information output by the neural network based on the input information. This target neural network model can then be used for shape perception of the soft robot, thereby improving the accuracy of shape perception.

[0080] In one embodiment, reference is made to Figure 3 The neural network model is a multi-input multi-output (MIMO) model. The input layer contains the acceleration of all nodes in the soft robot, the pose quaternions obtained by EKF fusion, and the length changes of all node connections. The output layer contains the position information of the entire soft robot's motion deformation at different times. By using the acceleration, pose quaternions obtained by EKF fusion, and length changes as inputs to the neural network, a multi-layer neural network (two layers for extraction as shown in the figure) is used to extract and train features from multiple sensors separately. One layer fuses the multi-modal sensor data (three layers for fusion as shown in the figure). After forward propagation through a fully connected layer, the predicted position information is output to the output layer. Simultaneously, because the sensor information includes labels of the soft robot's deformation (actual position information) obtained through the optical motion capture system, the network parameters are iteratively optimized by updating the backpropagation based on the difference between the actual position information and the predicted position information.

[0081] In one embodiment, the neural network fusion shown in the figure uses a network model such as LSTM, GNN, or a hybrid model such as GNN+LSTM or CNN+LSTM. The data from the three sensors are first processed through two layers of neural networks to extract corresponding features, effectively utilizing the connections and temporal dependencies between nodes. Then, these three neural network branches are concatenated and fused through a layer. The fused information is further processed through multiple layers of neural networks to capture higher-level features. Finally, a fully connected layer outputs the predicted location information.

[0082] Furthermore, the step of training the neural network model based on the predicted location information and the actual location information to obtain the target neural network model includes:

[0083] Step S213: Determine the loss error value between the predicted location information and the actual location information based on a preset error calculation method, and update the parameters of the neural network model based on the loss error value;

[0084] Step S214: Based on the neural network model after parameter update, the process of extracting and fusing the compensation posture quaternion, the acceleration, and the length change value between nodes in the preset neural network model to obtain the predicted position information is performed until the neural network model converges. Then, the neural network model after parameter update is determined as the target neural network model.

[0085] In this embodiment, the real-time position of the soft robot's deformation captured by the optical motion capture system is used as the label for neural network training during training. Then, the loss error value between the predicted position information and the actual position information is determined based on a preset error calculation method. The parameters of the neural network model are then updated based on the loss error value. The preset error calculation method can be the mean square error (MSE) calculation method, which can calculate the loss error between the predicted position information and the actual position information. The preset error calculation method for calculating the loss error value can be the same as commonly used calculation methods. Then, the parameters of the neural network model are updated based on the loss error value. For example, if the loss error value is large, a certain parameter in the neural network model is decreased, and another parameter is increased. This is achieved by backpropagating an optimization algorithm to the hidden layers of the neural network, continuously iterating and updating the network model's parameters. The parameter update algorithm chosen is Adam (other commonly used update methods can also be used, but are not limited here). Based on the parameter-updated neural network model, the compensated posture quaternion, the first acceleration, and the node length change value are extracted and fused within a preset neural network model to obtain predicted position information. This parameter update continues to determine the predicted position information until the neural network model converges. The parameter-updated neural network model is then determined as the target neural network model. The convergence of the neural network model can be defined as the loss error value being less than a certain value, or the number of parameter updates reaching a certain number, thus enabling the training of the neural network and providing a basis for subsequent shape perception in the soft robot.

[0086] Furthermore, based on the first, second, and / or third embodiments of this application described above, a fourth embodiment of the soft robot shape perception method of this application is proposed. In this embodiment, step S30, the step of performing shape perception on the soft robot according to the target neural network model, includes:

[0087] Step S31: Based on the real-time sensing data of each node of the soft robot, determine the real-time predicted position of the node in the target neural network model in sequence.

[0088] Step S32: Based on the real-time predicted position of the node, interpolation is performed to construct the three-dimensional shape reconstruction of the soft robot.

[0089] In this embodiment, after obtaining the target neural network model, real-time sensor data of the nodes whose positions need to be determined are input to determine the real-time predicted positions of the nodes. The real-time predicted position of the nodes refers to the output position of the target neural network model based on the real-time sensor data input. That is, the motion pattern of the soft robot is captured in real time and three-dimensional shape reconstruction is performed according to the trained neural network model to achieve shape perception (that is, three-dimensional shape reconstruction of the soft robot). After determining the real-time predicted positions of each node based on the real-time sensor data of each node, interpolation is performed based on the real-time predicted positions of the nodes to obtain the three-dimensional shape reconstruction (that is, the shape perception information of the soft robot; the interpolation method can be the interpolation method of existing robots, which is not limited here). This realizes the three-dimensional shape reconstruction of the soft robot, while avoiding the shortcomings of the prior art and improving the accuracy of the soft robot's shape perception.

[0090] In one embodiment, considering the characteristics of soft robots—infinite degrees of freedom, a large range of variability, and the difficulty in establishing accurate motion models—EKF and neural networks are used to fuse IMU and strain sensors to achieve shape perception and reconstruction of the soft robot. Simultaneously, the acceleration, angular velocity, and changes in connection length between nodes are fused to obtain complete three-dimensional deformation parameters. A multi-layer neural network is employed to achieve multimodal sensor information fusion, effectively utilizing the spatial and temporal characteristics of each node. Furthermore, accelerometer measurements are used to compensate for drift errors in gyroscope attitude calculations, effectively improving the accuracy of shape perception for the soft robot.

[0091] Furthermore, the real-time sensing data includes acceleration, angular velocity, and inter-node length change values. The step of determining the real-time predicted position of each node in the target neural network model based on the real-time sensing data of each node of the soft robot includes:

[0092] Step S33: Determine the target compensation attitude quaternion based on the acceleration, the angular velocity, and the Kalman filter algorithm;

[0093] Step S34: Determine the output of the target compensation attitude quaternion, the acceleration, and the inter-node length change value in the target neural network model as the real-time predicted position of the node.

[0094] In this embodiment, determining the real-time predicted position of a node requires determining the target compensation posture quaternion based on acceleration, angular velocity, and the Kalman filter algorithm. This is analogous to determining the compensation posture quaternion using acceleration, angular velocity, and a preset Kalman filter algorithm, as described in the previous embodiment. However, in this case, the input to the trained neural network model is used, while the input to the model to be trained is used. The target compensation posture quaternion, the acceleration, and the change in length between nodes are then used as the output of the target neural network model as the real-time predicted position of the node. In other words, the real-time predicted position of the node can be used directly without neural network training. It is worth noting that the acceleration here can be the data used in the previous training or new data. Furthermore, the acceleration, angular velocity, and change in connection length between nodes can be fused to obtain complete three-dimensional deformation parameters. A multi-layer neural network is then used to achieve multi-modal sensor information fusion, effectively utilizing the spatial and temporal characteristics of each node and improving the accuracy of shape perception for the soft robot.

[0095] This application also provides a shape sensing device for a soft robot; please refer to [reference needed]. Figure 4 The soft robot shape sensing device includes:

[0096] The data acquisition module A10 is used to acquire sensor data, wherein the sensor data includes the acceleration, angular velocity and inter-node length change values ​​of the soft robot nodes;

[0097] The model training module A20 is used to determine the compensation attitude quaternion based on the acceleration, angular velocity and the preset Kalman filter algorithm, and to train the target neural network model in the preset neural network model based on the compensation attitude quaternion, the acceleration and the length change value between nodes.

[0098] The shape perception module A30 is used to perform shape perception on the soft robot based on the target neural network model.

[0099] The soft robot shape sensing device provided in this application, employing the soft robot shape sensing method in the above embodiments, can solve the technical problem of low accuracy in soft robot shape sensing. Compared with the prior art, the beneficial effects of the soft robot shape sensing device provided in this application are the same as those of the soft robot shape sensing method provided in the above embodiments, and other technical features in the soft robot shape sensing device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0100] This application provides a soft robot shape perception device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the soft robot shape perception method in Embodiment 1 above.

[0101] The following is for reference. Figure 5 The diagram illustrates a structural schematic suitable for implementing the soft robot shape sensing device in the embodiments of this application. The soft robot shape sensing device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The illustrated soft robot shape sensing device is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0102] like Figure 5As shown, the soft robot shape sensing device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the soft robot shape sensing device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following devices can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the soft robot shape sensing device to communicate wirelessly or wiredly with other devices to exchange data. Although a soft robot shape sensing device with various devices is shown in the figure, it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0103] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0104] The soft robot shape perception device provided in this application, employing the soft robot shape perception method in the above embodiments, can solve the technical problem of low accuracy in soft robot shape perception. Compared with the prior art, the beneficial effects of the soft robot shape perception device provided in this application are the same as those of the soft robot shape perception method provided in the above embodiments, and other technical features in this soft robot shape perception device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0105] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0106] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0107] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the soft robot shape perception method in the above embodiments.

[0108] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor devices, apparatuses, or components, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution apparatus, device, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0109] The aforementioned computer-readable storage medium may be included in the soft robot shape sensing device; or it may exist independently and not assembled into the soft robot shape sensing device.

[0110] The aforementioned computer-readable storage medium carries one or more programs that, when executed by the soft robot shape sensing device, cause the soft robot shape sensing device to:

[0111] Acquire sensor data, wherein the sensor data includes the acceleration, angular velocity and inter-node length change values ​​of the soft robot nodes;

[0112] The compensation attitude quaternion is determined based on the acceleration, angular velocity, and a preset Kalman filter algorithm. The target neural network model is then trained in a preset neural network model based on the compensation attitude quaternion, the acceleration, and the change in length between nodes.

[0113] The soft robot performs shape perception based on the target neural network model.

[0114] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0115] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using dedicated hardware-based apparatus to perform the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0116] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0117] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described soft robot shape perception method, thereby solving the technical problem of low accuracy in soft robot shape perception. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the soft robot shape perception method provided in the above embodiments, and will not be repeated here.

[0118] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the soft robot shape perception method described above.

[0119] The computer program product provided in this application can solve the technical problem of low accuracy in shape perception of soft robots. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the soft robot shape perception method provided in the above embodiments, and will not be repeated here.

[0120] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A shape perception method for a soft robot, characterized in that, The soft robot shape perception method includes: Acquire sensor data, wherein the sensor data includes the acceleration, angular velocity and inter-node length change values ​​of the soft robot nodes; The compensation attitude quaternion is determined based on the acceleration, angular velocity, and a preset Kalman filter algorithm, wherein the step of determining the compensation attitude quaternion based on the acceleration, angular velocity, and preset Kalman filter algorithm includes: The angular velocity is preprocessed to obtain angular velocity processing data. Quaternion attitude calculation is performed based on the angular velocity processing data to obtain a state vector. The acceleration is preprocessed to obtain acceleration processing data, which is used as a measurement. A state equation in a preset Kalman filter algorithm is determined, and the output of the state vector and a preset initial state vector in the state equation is used as a predicted state vector. A measurement equation in the Kalman filter algorithm is determined, and the output of the measurement and the predicted state vector in the measurement equation is used as a compensated attitude quaternion. The model is trained in a preset neural network model based on the compensated attitude quaternion, the acceleration, and the inter-node length change value to obtain a target neural network model. The sensor acquisition data also includes actual position information acquired by an optical motion capture system. The step of training the target neural network model in a preset neural network model based on the compensated attitude quaternion, the acceleration, and the inter-node length change value includes: The compensated attitude quaternion, the acceleration, and the inter-node length change value are extracted and fused in a preset neural network model to obtain predicted position information; the neural network model is trained based on the predicted position information and the actual position information to obtain the target neural network model. The soft robot performs shape perception based on the target neural network model.

2. The soft robot shape perception method as described in claim 1, characterized in that, The step of training the neural network model based on the predicted location information and the actual location information to obtain the target neural network model includes: The loss error value between the predicted location information and the actual location information is determined based on a preset error calculation method, and the parameters of the neural network model are updated based on the loss error value. Based on the updated parameters, the neural network model is used to extract and fuse the compensation posture quaternion, the acceleration, and the length change between nodes in the preset neural network model to obtain the predicted position information. This process continues until the neural network model converges, at which point the updated neural network model is determined as the target neural network model.

3. The soft robot shape perception method according to any one of claims 1 to 2, characterized in that, The step of performing shape perception on the soft robot based on the target neural network model includes: The real-time predicted position of each node is determined in the target neural network model based on the real-time sensing data of each node of the soft robot. The three-dimensional shape reconstruction of the soft robot is obtained by interpolation based on the real-time predicted position of the node.

4. The soft robot shape perception method as described in claim 3, characterized in that, The real-time sensing data includes acceleration, angular velocity, and inter-node length changes. The step of determining the real-time predicted position of each node in the target neural network model based on the real-time sensing data of each node of the soft robot includes: The target compensation attitude quaternion is determined based on the acceleration, the angular velocity, and the Kalman filter algorithm; The outputs of the target compensation attitude quaternion, the acceleration, and the inter-node length change value in the target neural network model are determined as the real-time predicted positions of the nodes.

5. A soft robot shape sensing device, characterized in that, The soft robot shape sensing device includes: The data acquisition module is used to acquire sensor-collected data, wherein the sensor-collected data includes the acceleration, angular velocity, and inter-node length change values ​​of the soft robot nodes; The model training module is used to determine the compensated attitude quaternion based on the acceleration, angular velocity, and a preset Kalman filter algorithm. The step of determining the compensated attitude quaternion based on the acceleration, angular velocity, and the preset Kalman filter algorithm includes: preprocessing the angular velocity to obtain angular velocity processed data; performing quaternion attitude calculation based on the angular velocity processed data to obtain a state vector; preprocessing the acceleration to obtain acceleration processed data, using the acceleration processed data as a measurement, and determining the state equation in the preset Kalman filter algorithm, and determining the output result of the state vector and the preset initial state vector in the state equation as the predicted state vector; determining the measurement equation in the Kalman filter algorithm, and determining the measurement and the predicted state vector in the measurement equation. The output of the sensor is used as a compensation attitude quaternion, and the target neural network model is trained in a preset neural network model based on the compensation attitude quaternion, the acceleration, and the change in length between nodes. The sensor data also includes actual position information acquired by the optical motion capture system. The step of training the target neural network model in the preset neural network model based on the compensation attitude quaternion, the acceleration, and the change in length between nodes includes: extracting and fusing the compensation attitude quaternion, the acceleration, and the change in length between nodes in the preset neural network model to obtain predicted position information; and training the neural network model based on the predicted position information and the actual position information to obtain the target neural network model. A shape perception module is used to perform shape perception on the soft robot based on the target neural network model.

6. A soft robot shape sensing device, characterized in that, The soft robot shape perception device includes a processor, a memory, and a soft robot shape perception program stored in the memory that can be executed by the processor, wherein when the soft robot shape perception program is executed by the processor, it implements the steps of the soft robot shape perception method as described in any one of claims 1 to 4.

7. A computer storage medium, characterized in that, The computer storage medium stores a soft robot shape perception program, wherein when the soft robot shape perception program is executed by a processor, it implements the steps of the soft robot shape perception method as described in any one of claims 1 to 4.

Images