Control method, device and equipment for soft mechanical arm and storage medium

Abstract

The invention discloses a control method, device and equipment for a soft mechanical arm and a storage medium, and belongs to the technical field of mechanical arm control. The method comprises the steps that a target movement track of the soft mechanical arm is obtained; generating predicted driving pressure based on the target motion track through an inverse kinematics network model; generating a predicted motion track based on the predicted driving pressure through a forward kinematics network model; obtaining a trajectory error based on the target motion trajectory and the predicted motion trajectory; and a driving pressure correction value is generated based on the track error through the driving pressure correction model, and the soft mechanical arm is controlled to move according to the driving pressure correction value and the predicted driving pressure. Self-closed-loop driving pressure correction can be achieved through the inverse kinematics network model, the forward kinematics network model and the driving pressure correction model, the accuracy of the actual motion trail of the soft mechanical arm is improved, and then accurate motion control over the soft mechanical arm is achieved.

Description

Technical Field

[0001] This application relates to the field of robotic arm control technology, and in particular to a control method, device, equipment and storage medium for a soft robotic arm. Background Technology

[0002] A soft robotic arm is a type of robotic arm that mimics the characteristics of biological soft tissue. Unlike traditional rigid robotic arms, soft robotic arms are typically made of flexible or deformable materials and can perform various tasks and actions by deforming and adjusting their posture. Different types of soft robotic arms have different actuation methods; for example, their movement can be controlled by actuating air pressure.

[0003] In related technologies, when controlling the movement of a soft robotic arm, the pose data sequence and driving air pressure of the soft robotic arm are mainly obtained by integrating high-precision sensors (such as air pressure sensors or fiber optic sensors). Then, mathematical modeling is performed on the pose data sequence and driving air pressure to obtain the mapping relationship between the pose data sequence and driving air pressure. The driving air pressure for driving the movement of the soft robotic arm is determined through the mapping relationship, thereby realizing the motion control of the soft robotic arm.

[0004] However, the method of determining the driving air pressure of the soft robotic arm based on the mapping relationship in related technologies is limited by the complexity of the mathematical model. When the motion trajectory (indicating pose data sequence) of the soft robotic arm is relatively complex, it will lead to low motion accuracy of the soft robotic arm. Summary of the Invention

[0005] This application provides a control method, apparatus, device, and storage medium for a soft robotic arm, the technical solution of which is as follows:

[0006] According to one aspect of this application, a control method for a soft robotic arm is provided, the method comprising:

[0007] The target motion trajectory of the soft robotic arm is obtained, and the target motion trajectory is used to indicate the target pose data sequence of the soft robotic arm;

[0008] A predicted driving pressure is generated based on the target predicted trajectory using an inverse kinematics network model. The predicted driving pressure is the driving pressure value required to predict the movement of the soft robotic arm.

[0009] A predicted motion trajectory is generated based on the predicted driving pressure using a positive kinematics network model. The predicted motion trajectory is used to indicate the predicted pose data sequence of the soft robotic arm.

[0010] The trajectory error is obtained based on the target motion trajectory and the predicted motion trajectory. The trajectory error is used to indicate the pose data sequence error between the target pose data sequence and the predicted pose data sequence.

[0011] The driving pressure correction value is generated based on the trajectory error by the driving pressure correction model, and the movement of the soft robotic arm is controlled according to the driving pressure correction value and the predicted driving pressure.

[0012] According to one aspect of this application, a control device for a soft robotic arm is provided, the device comprising:

[0013] The acquisition module is used to acquire the target motion trajectory of the soft robotic arm, and the target motion trajectory is used to indicate the target pose data sequence of the soft robotic arm;

[0014] The generation module is used to generate a predicted driving pressure based on the target predicted trajectory through an inverse kinematics network model. The predicted driving pressure is a driving pressure value required to predict the movement of the soft robotic arm.

[0015] The generation module is used to generate a predicted motion trajectory based on the predicted driving pressure using a forward kinematics network model. The predicted motion trajectory is used to indicate the predicted pose data sequence of the soft robotic arm.

[0016] The generation module is used to obtain a trajectory error based on the target motion trajectory and the predicted motion trajectory, wherein the trajectory error is used to indicate the pose data sequence error between the target pose data sequence and the predicted pose data sequence;

[0017] The control module is used to generate a drive pressure correction value based on the trajectory error through a drive pressure correction model, and to control the movement of the soft robotic arm according to the drive pressure correction value and the predicted drive pressure.

[0018] According to another aspect of this application, a computer device is provided, comprising: a processor and a memory, wherein the memory stores at least one computer program, the at least one computer program being loaded and executed by the processor to implement the control method of the soft robotic arm as described above.

[0019] According to another aspect of this application, a computer storage medium is provided, wherein at least one computer program is stored in the computer-readable storage medium, and the at least one computer program is loaded and executed by a processor to implement the control method of the soft robotic arm as described above.

[0020] According to another aspect of this application, a computer program product is provided, comprising a computer program stored in a computer-readable storage medium; the computer program is read from and executed by a processor of a computer device from the computer-readable storage medium, causing the computer device to perform the control method of the soft robotic arm as described above.

[0021] The beneficial effects of the technical solution provided in this application include at least the following:

[0022] This application's embodiments are based on the combined action of an inverse kinematics network model, a forward kinematics network model, and a drive pressure correction model. After obtaining the desired motion trajectory (target motion trajectory) of the soft robotic arm, the inverse kinematics network model first generates a predicted drive pressure, then the forward kinematics network model generates a predicted motion trajectory, and finally the drive pressure correction model generates a drive pressure correction value based on the trajectory error (the error between the target motion trajectory and the predicted motion trajectory). The soft robotic arm's motion is then controlled jointly based on the drive pressure correction value and the predicted drive pressure. This control method for the soft robotic arm, by generating drive pressure correction values ​​through the drive pressure correction model to reduce trajectory errors, can achieve self-closed-loop drive pressure correction, improve the accuracy of the actual motion trajectory of the soft robotic arm, and thus achieve precise motion control of the soft robotic arm. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of the structure of a soft robotic arm provided in an exemplary embodiment of this application;

[0025] Figure 2 This is a schematic diagram of the architecture of a computer system provided in an exemplary embodiment of this application;

[0026] Figure 3 This is a schematic diagram of a control method for a soft robotic arm provided in an exemplary embodiment of this application;

[0027] Figure 4 This is a flowchart of a control method for a soft robotic arm provided in an exemplary embodiment of this application;

[0028] Figure 5 This is a flowchart of a control method for a soft robotic arm provided in an exemplary embodiment of this application;

[0029] Figure 6 This is a schematic diagram of a driving pressure correction model provided in an exemplary embodiment of this application;

[0030] Figure 7 This is a schematic diagram of an inverse kinematics network model provided in an exemplary embodiment of this application;

[0031] Figure 8 This is a schematic diagram of a forward kinematics network model provided in an exemplary embodiment of this application;

[0032] Figure 9 This is a flowchart of a control method for a soft robotic arm provided in an exemplary embodiment of this application;

[0033] Figure 10 This is a flowchart of a control method for a soft robotic arm provided in an exemplary embodiment of this application;

[0034] Figure 11 This is a schematic diagram of the motion modes of a soft robotic arm provided in an exemplary embodiment of this application;

[0035] Figure 12 This is a flowchart of a control method for a soft robotic arm provided in an exemplary embodiment of this application;

[0036] Figure 13 This is a schematic diagram of the motion trajectory of a soft robotic arm provided in an exemplary embodiment of this application;

[0037] Figure 14 This is a structural block diagram of the control device for a soft robotic arm provided in an exemplary embodiment of this application;

[0038] Figure 15 This is a schematic diagram of the structure of a computer device provided in an exemplary embodiment of this application. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0040] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0041] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0042] It should be understood that although the terms first, second, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, a first parameter may also be referred to as a second parameter without departing from the scope of this disclosure, and similarly, a second parameter may also be referred to as a first parameter. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0043] It should be noted that all data involved in this application (including but not limited to data used for analysis, training, and prediction) is information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of such data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the information regarding settings and operations involved in this application was obtained with full authorization.

[0044] Before introducing the technical solutions of this application, some terms involved in this application will be explained. The following related explanations are optional solutions and can be arbitrarily combined with the technical solutions of the embodiments of this application, all of which fall within the protection scope of the embodiments of this application. The embodiments of this application include at least some of the following contents.

[0045] Soft robotic arms: Soft robotic arms are a novel type of robotic arm that mimics the characteristics of biological soft tissue, exhibiting flexibility, adaptability, and high flexibility. Compared to traditional rigid robotic arms, soft robotic arms are typically constructed from flexible or deformable materials, allowing them to change shape or adapt to irregular working environments as needed. Optionally, soft robotic arms include at least one of pneumatically driven, hydraulically driven, electrically driven, magnetically driven, and thermally driven types. The embodiments of this application are applicable to the motion control of fluid-driven continuous soft robotic arms, such as pneumatically driven and hydraulically driven soft robotic arms.

[0046] Inverse kinematics refers to the process of calculating the motion parameters (driving pressure) required to achieve a given pose (position and orientation) of the end effector or end-effector of a soft robotic arm. For example, in the case of a pneumatically driven soft robotic arm, inverse kinematics can be understood as predicting the required driving air pressure value based on the desired pose (target motion trajectory) of the end effector or end-effector of the soft robotic arm.

[0047] Forward kinematics refers to the process of calculating the pose of the end effector or end-effector of a soft robotic arm based on its motion parameters (driving pressure). For example, in the case of a pneumatically driven soft robotic arm, forward kinematics can be understood as predicting the desired pose (target motion trajectory) of the end effector or end-effector based on the input driving air pressure value.

[0048] Deep Neural Network (DNN): Deep neural networks are a technique in the field of machine learning. A deep neural network is a multi-layer unsupervised neural network that can use the output features of the previous layer as the input of the next layer for feature learning. Through layer-by-layer feature mapping, the features of existing spatial samples are mapped to another feature space, thereby learning a better feature representation of the existing input.

[0049] Long Short-Term Memory (LSTM) network models are a special type of DNN designed to address the vanishing and exploding gradient problems inherent in traditional recurrent neural networks. Optionally, LSTM networks are time-recurrent neural networks suitable for processing and predicting important events with long intervals and delays in time series. They effectively control the flow of data through gating mechanisms (input gate, forget gate, and output gate), learning long-term dependencies within the data.

[0050] In this embodiment, the inverse kinematics network model, the forward kinematics network model, and the driving pressure correction model are network models trained based on a general LSTM model. Optionally, the inverse kinematics network model, the forward kinematics network model, and the driving pressure correction model have the same LSTM network architecture, but they are trained with different network parameters to achieve their respective functions, such as predicting driving pressure through the inverse kinematics network model and predicting motion trajectory through the forward kinematics network model.

[0051] This application mainly uses a pneumatically driven soft robotic arm (continuous soft robotic arm) as an example for illustration. It should be noted that the control method of the soft robotic arm proposed in this application is also applicable to other fluid-driven continuous soft robotic arms.

[0052] For example, Figure 1 The structure of a common pneumatically driven soft robotic arm (hereinafter referred to as a soft robotic arm) is shown (taking a single soft joint 1 and a soft robotic arm 2 as examples). Optionally, the soft robotic arm 2 is typically composed of a series of consecutive single soft joints 1, that is, the soft robotic arm includes multiple soft joints, and each soft joint 1 includes soft actuators arranged in parallel and evenly. For example, a single soft joint 1 includes a first soft actuator, a second soft actuator, and a third soft actuator arranged in parallel and evenly. "Evenly arranged" means that the axes of the three soft actuators are parallel, and the distance between any two axes is the same or similar.

[0053] Figure 1 Part (1) shows the structure of a single soft joint 1, which is the basic unit of the soft robotic arm 2, mainly composed of a soft actuator 1A and a rigid end plate 1B. The soft actuator 1A is a material or structure that can deform when air pressure is applied. Its shape is similar to a bellows, and it can expand or contract under air pressure. By controlling the change of air pressure, the soft actuator 1A can bend or extend, thereby driving the movement of the soft robotic arm 2. The rigid end plate 1B is fixed at both ends of the soft actuator 1A, providing the necessary support and connection points. The rigid end plate 1B ensures that the deformation of the soft actuator 1A can be effectively transmitted to the rest of the soft robotic arm 2.

[0054] Figure 1 Section (2) shows the structure of the soft robotic arm 2, which is a continuous multi-joint soft robotic arm composed of multiple individual soft joints 1 connected in series (for example, the soft robotic arm 2 is composed of 6 individual soft joints 1 connected in series). The individual soft joints 1 are connected by air passages to form a continuous soft robotic arm 2. Each individual soft joint 1 is a combination of a soft actuator and two rigid end plates. The series structure allows the entire soft robotic arm 2 to bend and extend flexibly in all directions. Optionally, the soft actuators in one path are connected through internal air passages. Figure 1 The soft robotic arm 2 shown in part (2) can be considered as having three soft actuators. The soft robotic arm 2 is controlled by adjusting the air pressure. For example, by applying positive pressure, the soft actuators expand and the soft robotic arm 2 extends; by applying negative pressure, the soft actuators contract and the soft robotic arm 2 bends.

[0055] It should be noted that the above is merely an exemplary description of the soft robotic arm structure and does not constitute a limitation on the soft robotic arm structure. For example, the number of individual soft joints of the soft robotic arm may be increased or decreased, and the number of soft actuators of the soft robotic arm may be increased or decreased. This application does not limit these aspects.

[0056] Figure 2A schematic diagram of the architecture of a computer system provided in one embodiment of this application is shown. The computer system may include: a terminal 100 and a server 200.

[0057] Terminal 100 can be an electronic device such as a mobile phone, tablet computer, vehicle terminal (vehicle system), wearable device, personal computer (PC), or vehicle terminal. A client application for the target application can be installed and run on terminal 100. This target application can be an application that supports the control of a soft robotic arm, or other applications that provide control of a soft robotic arm; this application does not limit the specific application. Furthermore, this application does not limit the form of the target application, including but not limited to applications (Apps), mini-programs, clients, etc., installed on terminal 100, and can also be in web page form.

[0058] Server 200 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services such as cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and basic cloud computing services such as big data and artificial intelligence handheld image recognition platforms. Server 200 can be a backend server for the aforementioned target application, used to provide backend services to the clients of the target application.

[0059] Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. Based on the cloud computing business model, cloud technology encompasses network technology, information technology, integration technology, management platform technology, and application technology. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will require robust system support, which can only be achieved through cloud computing.

[0060] In some embodiments, the server described above can also be implemented as a node in a blockchain system. Blockchain is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and cryptographic algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer.

[0061] Optionally, the soft robotic arm and terminal 100 and server 200 can communicate via a network, such as a wireless network or a wired network.

[0062] The control method for the soft robotic arm provided in this application embodiment can be executed by a computer device, which refers to an electronic device with data computing, processing, and storage capabilities. Figure 2 Taking the computer system shown as an example, the control method of the soft robotic arm can be executed by the terminal 100 (such as the client of the target application installed and running in the terminal 100 executing the control method of the soft robotic arm), or by the server 200, or by the interaction and cooperation between the terminal 100 and the server 200. This application does not limit this.

[0063] For example, Figure 3 A schematic diagram of a control method for a soft robotic arm provided in an exemplary embodiment of this application is shown. The method is executed by a computer device, which may be... Figure 2 The terminal 100 and / or server 200 shown.

[0064] In related technologies, controlling the movement of a soft robotic arm primarily involves acquiring its pose data sequence and driving air pressure using integrated high-precision sensors (such as barometric pressure sensors or fiber optic sensors). A model is then created to establish a mapping relationship between the pose data sequence and the driving air pressure. This mapping relationship is used to determine the driving air pressure required to move the soft robotic arm, thus achieving motion control. However, this method of determining the driving air pressure based on a mapping relationship cannot accurately capture the driving air pressure under arbitrary motion trajectories, resulting in low motion accuracy for the soft robotic arm.

[0065] To address the aforementioned issues, this application proposes a control method for a soft robotic arm. First, a predicted driving pressure is generated based on the target motion trajectory using an inverse kinematics network model. Second, a predicted motion trajectory is generated based on the predicted driving pressure using a forward kinematics network model. Then, a driving pressure correction value is generated by using a driving pressure correction model to minimize the trajectory error (the error between the target motion trajectory and the predicted motion trajectory). This enables self-closed-loop driving pressure correction, thereby reducing the error between the actual motion trajectory and the target motion trajectory of the soft robotic arm, and ultimately achieving precise motion control of the soft robotic arm.

[0066] For example, in conjunction with reference Figure 1 After acquiring the target motion trajectory 10 of the soft robotic arm, the target motion trajectory 10 is input into the inverse kinematics network model 20. The inverse kinematics network model 20 generates a predicted driving pressure 30 based on the target predicted trajectory 10. The target motion trajectory 10 is the expected position and posture of the soft robotic arm (typically referring to the center of the rigid endplate at the end of the soft robotic arm or the end effector of the soft robotic arm) when performing a task. Optionally, the target motion trajectory 10 is used to indicate the target pose data sequence of the soft robotic arm. The predicted driving pressure 30 is calculated by the inverse kinematics network model 20 and is the driving pressure value required for the soft robotic arm to move along the target motion trajectory 10. The inverse kinematics network model 20 is a network model trained based on an LSTM model (or a general LSTM model). Optionally, when training the inverse kinematics network model 20 based on the target motion trajectory, the actual motion trajectory can be determined by a motion capture system on the soft robotic arm; or, the actual motion trajectory can be determined by a camera vision sensor on the soft robotic arm.

[0067] In some embodiments, the predicted driving pressure 30 is input into the forward kinematics network model 40, which generates a predicted motion trajectory 50 based on the predicted driving pressure 30. The predicted motion trajectory 50 is calculated by the forward kinematics network model 40 and is the expected motion path or pose of the soft robotic arm inferred from the predicted driving pressure 30. Optionally, the predicted motion trajectory 50 is used to indicate the predicted pose data sequence of the soft robotic arm. The forward kinematics network model 40 is a network model trained based on an LSTM model. Optionally, when training the forward kinematics network model 40 based on the predicted driving pressure, the predicted driving pressure can be determined by a pressure sensor on the soft robotic arm.

[0068] In some embodiments, a trajectory error 60 is obtained based on the target motion trajectory 10 and the predicted motion trajectory 50. The trajectory error 60 is the error between the target motion trajectory 10 and the predicted motion trajectory 50, used to indicate the pose data sequence error between the target pose data sequence and the predicted pose data sequence. Optionally, the trajectory error 60 is input into a driving pressure correction model 70, which generates a driving pressure correction value 80 based on the trajectory error 60. The driving pressure correction value 80 is used to correct the predicted driving pressure 50 to reduce the trajectory error 60. The driving pressure correction model 70 is a network model trained based on an LSTM model. Optionally, the corrected driving pressure can be obtained based on the driving pressure correction value 80 and the predicted driving pressure 50, and the movement of the soft robotic arm is controlled according to the corrected driving pressure.

[0069] Figure 4 This is a flowchart illustrating a control method for a soft robotic arm provided in an exemplary embodiment of this application. The method can be executed by a computer device, which may be a terminal or a server. The method includes:

[0070] Step 210: Obtain the target motion trajectory of the soft robotic arm;

[0071] In some embodiments, a soft robotic arm is a novel type of robotic arm that mimics the characteristics of biological soft tissue, exhibiting softness, flexibility, and high compliance. Compared to traditional rigid robotic arms, soft robotic arms are typically constructed from flexible or deformable materials, allowing them to change shape or adapt to irregular working environments as needed.

[0072] Optionally, based on the driving method of the soft robotic arm, the types of soft robotic arms include at least one of the following: pneumatically driven soft robotic arms, hydraulically driven soft robotic arms, electrically driven soft robotic arms, magnetically driven soft robotic arms, and thermally driven soft robotic arms, but are not limited thereto.

[0073] In some embodiments, the control method proposed in this application is applicable to the motion control of fluid-driven continuous soft robotic arms, such as pneumatically driven and hydraulically driven soft robotic arms. Specifically, a pneumatically driven soft robotic arm controls the inflow and outflow of gas to change the air pressure within the soft actuator, thereby achieving bending or straightening of the soft robotic arm. A hydraulically driven soft robotic arm utilizes liquid pressure to drive a piston within the soft actuator, achieving movement of the soft robotic arm.

[0074] In some embodiments, the target motion trajectory of the soft robotic arm can be understood as the target motion trajectory at the center of the rigid endplate of the soft robotic arm, or as the target motion trajectory of the end effector of the soft robotic arm. The target motion trajectory is used to indicate the target pose data sequence of the soft robotic arm.

[0075] In some embodiments, the target motion trajectory is used to indicate a target pose data sequence at the center of the rigid endplate of the soft robotic arm, or the target motion trajectory is used to indicate a target pose data sequence of the end effector of the soft robotic arm. Optionally, the target pose data is represented based on the three-dimensional spatial coordinates of the center of the rigid endplate of the soft robotic arm or the end effector of the soft robotic arm. Here, the end effector refers to an actuator capable of performing a first function, such as an end effector capable of performing a grasping function. The embodiments of this application do not specifically limit the function of the end effector.

[0076] In one possible implementation, the target motion trajectory can also be used to indicate the target motion trajectory of the entire soft manipulator. For example, the target motion trajectory indicates a sequence of target pose data at the center of the rigid endplate of each soft joint in the soft manipulator, with the target pose data represented by the three-dimensional spatial coordinates of the center of the rigid endplate of each soft joint; or, the target motion trajectory indicates a sequence of target pose data at the center of the rigid endplate of the central soft joint of the soft manipulator, with the target pose data represented by the three-dimensional spatial coordinates of the center of the rigid endplate of the central soft joint. This application does not limit this to a specific implementation.

[0077] It should be noted that, unless otherwise specified, the target motion trajectory of the soft robotic arm mentioned below can be understood as the target motion trajectory at the center of the rigid end plate at the end of the soft robotic arm, or the target motion trajectory of the end effector of the soft robotic arm.

[0078] In some embodiments, the target motion trajectory of the soft robotic arm is the motion path or trajectory that the soft robotic arm is expected to achieve when performing a task. Optionally, the target motion trajectory is used to indicate a sequence of target pose data for the soft robotic arm; that is, the target motion trajectory is used to indicate multiple target pose data of the soft robotic arm. Each target pose data defines the target position and orientation of the soft robotic arm at a certain point in time (or a certain moment). Through a series of continuous target pose data, the target path of the soft robotic arm from the starting point to the ending point is depicted. Optionally, the target pose data sequence typically consists of a series of three-dimensional coordinate points, indicating the target motion trajectory of the soft robotic arm.

[0079] In some embodiments, the target motion trajectory is a target path used to describe how the soft robotic arm moves toward, rotates, or tilts. Optionally, the target motion trajectory can take various forms, depending on the task requirements and operating environment.

[0080] For example, the target motion trajectory includes at least one of a straight-line trajectory, a curved trajectory, a circular arc trajectory, and a custom trajectory, but is not limited thereto. A straight-line trajectory is where the soft robotic arm moves from a starting point to an ending point along a straight line, suitable for simple handling or linear motion tasks; a curved trajectory is where the soft robotic arm moves along a predetermined curved path, suitable for tasks requiring obstacle avoidance or complex path planning; a circular arc trajectory is where the soft robotic arm moves along a circular arc path, suitable for rotational handling or circular motion tasks; and a custom trajectory is the motion trajectory of the soft robotic arm customized according to specific task requirements and environmental constraints.

[0081] Step 220: Generate predicted driving pressure based on the target predicted trajectory using an inverse kinematics network model;

[0082] The predicted drive pressure is the drive pressure value used to predict the movement required to move the soft robotic arm. The drive pressure value refers to the magnitude of the pressure applied to the soft robotic arm, determining its bending degree and direction of movement. For example, in a gas-driven soft robotic arm, the predicted drive pressure is the drive air pressure value used to predict the movement required to move the soft robotic arm. Changes in the drive air pressure will cause changes in the shape of the soft robotic arm, thus affecting its movement.

[0083] In some embodiments, the predicted driving pressure refers to the driving pressure value calculated by an inverse kinematics network model, used to drive the soft robotic arm to achieve the target motion trajectory.

[0084] In some embodiments, inverse kinematics refers to the process of calculating the driving pressure required to achieve a given pose of the rigid endplate or end effector of a soft robotic arm. An inverse kinematics network model is a network model used to solve inverse kinematics problems, that is, predicting the required driving pressure based on the target motion trajectory of the soft robotic arm. Optionally, the inverse kinematics network model is a network model trained based on a general LSTM model.

[0085] In some embodiments, a predicted driving pressure can be generated based on the target predicted trajectory using an inverse kinematics network model. Optionally, the inverse kinematics network model takes the target motion trajectory as input and outputs the predicted driving pressure, the predicted driving pressure indicating the driving pressure value being able to drive the soft robotic arm to reach the target motion trajectory.

[0086] Optionally, the target motion trajectory is used to indicate the target pose data sequence (multiple target pose data) of the soft robotic arm, the driving pressure value corresponds to each target pose data of the target motion trajectory in time, and the predicted driving pressure indicates the driving pressure value that needs to be applied at each time point to ensure that the robotic arm can move along the path of the target motion trajectory.

[0087] For example, suppose a soft robotic arm needs to move along a straight line from point A to point B, and this movement is discretized into multiple time segments. Each time segment corresponds to a set of target pose data, and the set of target pose data includes multiple target pose data. Optionally, the set of target pose data includes multiple target pose data at the center of the rigid endplate of the soft robotic arm, or the target pose data of the end effector of the soft robotic arm. For each moment in each time segment, the inverse kinematics network model calculates the driving pressure value corresponding to the target pose data at the current moment. For example, the target pose data is represented as coordinates in a three-dimensional coordinate system. The coordinates of the first target pose data at time t1 are (x1, y1, z1); the coordinates of the second target pose data at time y2 are (x2, y2, z2); and the coordinates of the third target pose data at time t3 are (x3, y3, z3). Accordingly, at time t1, the driving pressure value corresponding to the first target pose data is P1; at time t2, the driving pressure value corresponding to the second target pose data is P2; and at time t3, the driving pressure value corresponding to the third target pose data is P3.

[0088] Step 230: Generate a predicted motion trajectory based on the predicted driving pressure using a forward kinematics network model;

[0089] Among them, the predicted motion trajectory is used to indicate the predicted pose data sequence of the soft robotic arm.

[0090] In some embodiments, the predicted motion trajectory of the soft robotic arm can be understood as the predicted motion trajectory at the center of the rigid endplate of the soft robotic arm, or as the predicted motion trajectory of the end effector of the soft robotic arm. The predicted motion trajectory is used to indicate the predicted pose data sequence of the soft robotic arm. Optionally, the predicted motion trajectory is used to indicate the predicted pose data sequence at the center of the rigid endplate of the soft robotic arm, or the target motion trajectory is used to indicate the predicted pose data sequence of the end effector of the soft robotic arm. Optionally, the predicted pose data is represented based on the three-dimensional spatial coordinates of the center of the rigid endplate of the soft robotic arm or the end effector of the soft robotic arm.

[0091] In one possible implementation, the predicted motion trajectory can also be used to indicate the predicted motion trajectory of the entire soft manipulator. For example, the predicted motion trajectory indicates a sequence of predicted pose data at the center of the rigid endplate of each soft joint in the soft manipulator, with the predicted pose data represented by the three-dimensional spatial coordinates of the center of the rigid endplate of each soft joint; or, the predicted motion trajectory indicates a sequence of predicted pose data at the center of the rigid endplate of the central soft joint of the soft manipulator, with the predicted pose data represented by the three-dimensional spatial coordinates of the center of the rigid endplate of the central soft joint. This application does not limit this to a specific implementation.

[0092] It should be noted that, unless otherwise specified, the predicted motion trajectory of the soft robotic arm mentioned below can be understood as the predicted motion trajectory at the center of the rigid end plate at the end of the soft robotic arm, or the predicted motion trajectory of the end effector of the soft robotic arm.

[0093] In some embodiments, the predicted motion trajectory of the soft robotic arm is a motion path or trajectory corresponding to the predicted driving pressure. Optionally, the predicted motion trajectory is used to indicate a sequence of predicted pose data for the soft robotic arm; that is, the predicted motion trajectory is used to indicate multiple predicted pose data of the soft robotic arm. Each predicted pose data defines the predicted position and orientation of the soft robotic arm at a certain point in time. Through a series of consecutive predicted pose data, the predicted path of the soft robotic arm from the starting point to the ending point is depicted. Optionally, the sequence of predicted pose data typically consists of a series of three-dimensional coordinate points indicating the predicted motion trajectory of the soft robotic arm.

[0094] In some embodiments, the predicted motion trajectory refers to the predicted motion trajectory calculated based on the predicted driving pressure using a forward kinematics network model, which is used to drive the soft robotic arm to reach the predicted driving pressure.

[0095] In some embodiments, forward kinematics refers to the process of calculating the pose of the end effector or end effector of a soft robotic arm when the driving pressure is reached. A forward kinematics network model is a network model used to solve forward kinematics problems, that is, to obtain the predicted motion trajectory of a soft robotic arm based on the predicted driving pressure. Optionally, the forward kinematics network model is a network model trained based on a general LSTM model.

[0096] In some embodiments, a forward kinematics network model can generate a predicted motion trajectory based on the predicted driving pressure. Optionally, the forward kinematics network model takes the predicted motion pressure as input and outputs a predicted motion trajectory, which indicates the motion trajectory corresponding to the arrival of the predicted driving pressure. Optionally, the predicted motion trajectory is used to indicate a sequence of predicted pose data (multiple predicted pose data) of the soft robotic arm. The predicted motion trajectory is a predicted path that the soft robotic arm will traverse after the predicted driving pressure is applied, including the predicted pose data at each time point.

[0097] For example, suppose a soft robotic arm needs to move along a straight line from point A to point B, spanning multiple time segments. The inverse kinematics network model calculates the predicted driving pressure corresponding to the target pose data at a certain moment in each time segment, while the forward kinematics network model calculates the predicted motion path, or predicted trajectory, of the soft robotic arm based on the driving pressure value indicated by the predicted driving pressure. For instance, if the driving pressure value at time t1 is P1, the first predicted pose data calculated by the forward kinematics network model based on P1 is (x′1, y′1, z′1); if the driving pressure value at time t2 is P2, the second predicted pose data calculated by the forward kinematics network model based on P2 is (x′2, y′2, z′2); and if the driving pressure value at time t3 is P3, the third predicted pose data calculated by the forward kinematics network model based on P3 is (x′3, y′3, z′3).

[0098] Step 240: Obtain the trajectory error based on the target trajectory and the predicted trajectory;

[0099] In some embodiments, trajectory error refers to the difference between the target motion trajectory and the predicted motion trajectory. The target motion trajectory is used to indicate the target pose data sequence of the soft robotic arm, the predicted motion trajectory is used to indicate the predicted pose data sequence of the soft robotic arm, and the trajectory error is used to indicate the pose data sequence error between the target pose data sequence and the predicted pose data sequence.

[0100] In some embodiments, trajectory error reflects the deviation between the predicted motion trajectory and the target motion trajectory achieved by the center of the rigid endplate or the end effector of the soft robotic arm during task execution.

[0101] In some embodiments, trajectory errors can arise from various causes. For example, external disturbances such as friction, changes in gravity, and external resistance can affect the movement of the soft robotic arm, causing its predicted trajectory to deviate from the target trajectory. Alternatively, limitations in the soft robotic arm's drive system, such as the controller in the drive system potentially failing to accurately execute the predicted drive pressure, can lead to a difference between the predicted and target trajectories. This application does not limit the causes of trajectory errors.

[0102] For example, suppose a soft robotic arm needs to move along a straight line from point A to point B, passing through multiple time points. The inverse kinematics network model calculates the predicted driving pressure corresponding to the target pose data at each time segment, while the forward kinematics network model calculates the predicted motion trajectory of the soft robotic arm based on the driving pressure value indicated by the predicted driving pressure. For example, the coordinates of the first target pose data at time t1 are (x1, y1, z1), and the driving pressure value at time t1 is P1. The first predicted pose data calculated by the forward kinematics network model based on P1 is (x′1, y′1, z′1), and the trajectory error can be expressed as (x1-x′1, y1-y′1, z1-z′1). The coordinates of the second target pose data at time t2 are (x2, y2, z2), and the driving pressure value at time t2 is P2. The forward kinematics network model calculates the predicted motion trajectory of the soft robotic arm based on P1. The second predicted pose data calculated is (x′2, y′2, z′2), and the trajectory error can be expressed as (x2-x′2, y2-y′2, z2-z′2). The coordinates of the third target pose data at time t3 are (x3, y3, z3), and the driving pressure value at time t3 is P3. The third predicted pose data calculated by the forward kinematics network model based on P3 is (x′3, y′3, z′3), and the trajectory error can be expressed as (x3-x′3, y3-y′3, z3-z′3).

[0103] In some embodiments, trajectory error is typically calculated by comparing a target pose data sequence with a predicted pose data sequence. Various methods can be used to calculate trajectory error, such as Euclidean distance, angular difference, or other correlation measures. This application does not limit the method used to calculate trajectory error.

[0104] Step 250: Generate driving pressure correction values ​​based on trajectory errors using the driving pressure correction model, and control the movement of the soft robotic arm based on the driving pressure correction values ​​and predicted driving pressure.

[0105] Among them, the drive pressure correction value is a compensation pressure value used to correct the movement of the robotic arm in the drive control software.

[0106] In some embodiments, the drive pressure correction value is a compensation pressure value calculated by the drive pressure correction model based on trajectory error. Optionally, the drive pressure correction value is used to adjust the predicted drive pressure to achieve more precise motion control of the soft robotic arm and ensure that the actual motion trajectory of the soft robotic arm is closer to the target predicted trajectory.

[0107] In some embodiments, the trajectory error is input into the driving pressure correction model, the driving pressure model generates a driving pressure correction value, and the driving pressure correction value is added to the predicted driving pressure to generate the actual driving pressure for controlling the movement of the soft robotic arm.

[0108] In some embodiments, the driving pressure correction model is a network model that generates driving pressure correction values ​​to correct the motion of the soft robotic arm based on the input trajectory error. Optionally, the driving pressure correction model is a network model trained based on a general LSTM model.

[0109] In summary, the method provided in this application is based on the combined action of an inverse kinematics network model, a forward kinematics network model, and a drive pressure correction model. After obtaining the desired motion trajectory (target motion trajectory) of the soft robotic arm, it first generates a predicted drive pressure based on the target motion trajectory using the inverse kinematics network model, then generates a predicted motion trajectory based on the predicted drive pressure using the forward kinematics network model, and finally generates a drive pressure correction value based on the trajectory error (the error between the target motion trajectory and the predicted motion trajectory) using the drive pressure correction model. The soft robotic arm's motion is then controlled based on the drive pressure correction value and the predicted drive pressure. This control method for the soft robotic arm generates the drive pressure correction value by minimizing the trajectory error through the drive pressure correction model, achieving self-closed-loop drive pressure correction, improving the accuracy of the actual motion trajectory of the soft robotic arm, and thus realizing precise motion control of the soft robotic arm.

[0110] The driving pressure correction model generates driving pressure correction values ​​based on trajectory errors.

[0111] In some embodiments, the trajectory error is input into the driving pressure correction model, the driving pressure model generates a driving pressure correction value, and the driving pressure correction value is added to the predicted driving pressure to generate the actual driving pressure for controlling the movement of the soft robotic arm.

[0112] Figure 5 A flowchart of a control method for a soft robotic arm provided in an exemplary embodiment of this application is shown, wherein step 250 can be replaced by step 251.

[0113] Step 251: With the goal of minimizing trajectory error, the driving pressure correction value is generated based on the trajectory error through the driving pressure correction model, and the soft robotic arm is controlled to move according to the driving pressure correction value and the predicted driving pressure.

[0114] The trajectory error is the difference between the target motion trajectory and the predicted motion trajectory, used to indicate the pose data sequence error between the target pose data sequence and the predicted pose data sequence.

[0115] In some embodiments, with the goal of minimizing trajectory error, the drive pressure correction model is controlled to generate drive pressure correction values ​​based on the trajectory error. Minimizing trajectory error allows the actual motion trajectory of the soft robotic arm to be as close as possible to the target motion trajectory, reducing pose deviations.

[0116] In some embodiments, the drive pressure correction model generates a drive pressure correction value based on the trajectory error. This drive pressure correction value is used to adjust the predicted drive pressure driving the soft robotic arm's motion, correcting the current motion state of the soft robotic arm to make it closer to the target motion trajectory. By minimizing the trajectory error, the drive pressure correction value generated by the drive pressure correction model can better correct the predicted drive pressure. By combining the generated drive pressure correction value with the predicted drive pressure, the motion of the soft robotic arm is adjusted to be as close as possible to the ideal target motion trajectory.

[0117] In this embodiment, a driving pressure correction value is generated by a driving pressure correction model, and the driving pressure correction value is added to the predicted driving pressure to form an adjusted actual driving pressure, thereby adjusting the movement of the soft robotic arm. This method allows the actual movement trajectory of the soft robotic arm to be estimated more closely to the target movement trajectory, thus improving the accuracy of the soft robotic arm's movement.

[0118] In some embodiments, the driving pressure correction model is a network model trained based on a general LSTM model. For example, Figure 6 This is a schematic diagram of the structure of a driving pressure correction model provided in one embodiment of this application.

[0119] like Figure 6 As shown, the driving pressure correction model 600 includes a first input layer 601, a first memory layer 602, a first fully connected layer 603, and a first regression output layer 604, which are cascaded in sequence. The first memory layer 602 is an LSTM layer and includes a first number of hidden units. The first fully connected layer 603 includes fully connected layer A and fully connected layer B; optionally, the first fully connected layer 603 also includes a first dropout layer. Optionally, the input of the driving pressure correction model 600 is the trajectory error, and the output is the driving pressure correction value. Optionally, step 251 above can be replaced by the following steps:

[0120] 1) The trajectory error is input into the first input layer 601, and the first input layer 601 encodes the trajectory error into trajectory error features;

[0121] Trajectory error can also be called pose data sequence error, and trajectory error characteristics can also be called pose data sequence error characteristics.

[0122] Optionally, the trajectory error is used to indicate the pose data sequence error between the target pose data sequence and the predicted pose data sequence, and the trajectory error includes multiple pose data errors. The first input layer 601 encodes the pose data sequence errors to obtain pose data sequence error features, and each pose data error feature is a characteristic vector representation of a pose data error. For example, the trajectory error is input into the first input layer 601, and the first input layer 601 encodes the trajectory to obtain trajectory error feature 1 (i.e., pose data error feature 1), trajectory error feature 2 (i.e., pose data error feature 2)... trajectory error feature n (i.e., pose data error feature n).

[0123] 2) Input the trajectory error features into the first memory layer 602, and obtain the hidden state trajectory error features after processing based on the first number of hidden units;

[0124] The first memory layer 602 is an LSTM layer, which includes a first number of hidden units. For example, the first memory layer 602 includes m LSTM units, which are connected sequentially. Optionally, each LSTM unit receives the input trajectory error features at different times and updates its internal state. The input of each LSTM unit includes the trajectory error features at the current time, the hidden state at the previous time, and the unit state at the previous time. The output of each LSTM unit includes the hidden state at the current time and the unit state at the current time.

[0125] Optionally, the first memory layer 602 takes the trajectory error features as input, that is, the first memory layer 602 takes the pose data sequence error features as input, and after processing by a first number of hidden units (which can also be understood as LSTM units), the first memory layer 602 outputs the corresponding hidden state trajectory error features.

[0126] 3) Input the hidden state trajectory error features into the first fully connected layer 603. After performing a fully connected operation based on the first fully connected layer 603, the linearly changed hidden state trajectory error features are obtained.

[0127] The first fully connected layer 603 includes a fully connected layer A and a fully connected layer B. Optionally, the first fully connected layer 603 also includes a first dropout layer.

[0128] In one possible implementation, the first fully connected layer 603 includes a fully connected layer A, a first discard layer, and a fully connected layer B cascaded in sequence. Optionally, the hidden state trajectory error features are input into the first fully connected layer 603, the fully connected layer A performs a first fully connected layer operation, the first discard layer performs a discard operation on the features after the first fully connected layer operation, and the fully connected layer B performs a second fully connected layer operation to obtain the linearly changed hidden state trajectory error features.

[0129] In some embodiments, the hidden state trajectory error features are input into the first fully connected layer 603. The fully connected layer A performs a first linear transformation and activation function processing on the hidden state trajectory error features. The first dropout layer performs a dropout operation, which prevents overfitting by randomly discarding (setting to zero) the output of a portion of neurons. After passing through the first dropout layer, the hidden state trajectory error features enter the fully connected layer B for a second linear transformation and activation function processing. After further extraction and transformation of features, the linearly transformed hidden state trajectory error features are obtained.

[0130] 4) Input the linearly transformed hidden state trajectory error features into the first regression output layer 604 to obtain the driving pressure correction value.

[0131] In some embodiments, the linearly transformed latent state trajectory error features are input into the first regression output layer 604, which generates the final driving pressure correction value. The driving pressure correction value is generated based on the trajectory error, and different driving pressure correction values ​​correspond to different pose data sequence errors.

[0132] In this embodiment, a driving pressure correction model is used to generate a predicted pressure correction value based on the trajectory error. The driving pressure correction model is a network model trained based on a first general LSTM model. By using the trajectory error as the input of the LSTM model, the dependency between the trajectory error and the predicted pressure correction value can be learned. Since the LSTM model can update its internal parameters in real time, it can dynamically adjust the prediction result (predicted pressure correction value) according to the latest trajectory error.

[0133] Inverse kinematics network models generate and predict driving pressure based on target motion trajectories.

[0134] In some embodiments, the target motion trajectory is input into the inverse kinematics network model, and the inverse kinematics network generates a corresponding predicted driving pressure based on the target pose data sequence indicated by the target motion trajectory.

[0135] In some embodiments, the inverse kinematics network model is a network model trained based on a second general LSTM model. For example, Figure 7 This is a schematic diagram of the structure of an inverse kinematics network model provided in one embodiment of this application.

[0136] like Figure 7As shown, the inverse kinematics network model 700 includes a second input layer 701, a second memory layer 702, a second fully connected layer 703, and a second regression output layer 704, all cascaded sequentially. The second memory layer 702 is an LSTM layer, containing a second number of hidden units. The second fully connected layer 703 includes fully connected layers C and D; optionally, it also includes a second dropout layer. Optionally, the input to the inverse kinematics network model 700 is the target motion trajectory, and the output is the predicted driving pressure. The steps of the inverse kinematics network model 700 in generating the predicted driving pressure based on the target motion trajectory are as follows:

[0137] 1) Input the target motion trajectory into the second input layer 701, and the second input layer 701 encodes the target motion trajectory into target pose sequence features;

[0138] The target motion trajectory is used to indicate the target pose data sequence of the soft robotic arm; that is, the target motion trajectory is used to indicate multiple target pose data of the soft robotic arm. The target pose sequence feature is the feature vector representation of the encoded target pose data sequence (multiple target pose data).

[0139] Optionally, the target motion trajectory is input into the second input layer 701, that is, the target pose data sequence (multiple target pose data) is input into the second input layer 701. The second input layer 701 encodes the target pose data sequence to obtain target pose sequence features, that is, multiple target pose features are obtained. For example, the target motion trajectory is input into the second input layer 701, and the second input layer 701 encodes the target motion trajectory to obtain target pose feature 1, target pose feature 2, ... target pose feature n.

[0140] 2) Input the target pose sequence features into the second memory layer 702, and process them based on the second number of hidden units to obtain the hidden state target pose sequence features;

[0141] The second memory layer 702 is an LSTM layer, containing a second number of hidden units. For example, the second memory layer 702 includes m LSTM units, which are connected sequentially. Optionally, each LSTM unit receives the input target pose features at different times and updates its internal state. The input of each LSTM unit includes the target pose features at the current time, the hidden state at the previous time, and the unit state at the previous time. The output of each LSTM unit includes the hidden state at the current time and the unit state at the current time.

[0142] Optionally, the second memory layer 702 takes the target pose sequence features as input. That is, the second memory layer 702 takes the target pose sequence features as input, and after processing by the second number of hidden units (which can also be understood as LSTM units), the second memory layer 702 outputs the corresponding hidden state target pose sequence features.

[0143] 3) Input the hidden state target pose sequence features into the second fully connected layer 703. After performing a fully connected operation based on the second fully connected layer 703, the linearly transformed hidden state target pose sequence features are obtained.

[0144] The second fully connected layer 703 includes a fully connected layer C and a fully connected layer D. Optionally, the second fully connected layer 703 also includes a second dropout layer.

[0145] In one possible implementation, the second fully connected layer 703 includes a fully connected layer C, a second dropout layer, and a fully connected layer D cascaded together. Optionally, the hidden state target pose sequence features are input into the second fully connected layer 703. The fully connected layer C performs a first fully connected layer operation, the second dropout layer performs a dropout operation on the features after the first fully connected layer operation, and the fully connected layer D performs a second fully connected layer operation to obtain the linearly transformed hidden state target pose sequence features.

[0146] In some embodiments, the hidden state target pose sequence features are input into the second fully connected layer 703. The fully connected layer C performs a first linear transformation and activation function processing on the hidden state target pose sequence features. The second dropout layer performs a dropout operation, randomly discarding (setting to zero) the output of a portion of neurons to prevent overfitting. After passing through the second dropout layer, the hidden state target pose sequence features enter the fully connected layer D for a second linear transformation and activation function processing. After further extraction and transformation of features, the linearly transformed hidden state target pose sequence features are obtained.

[0147] 4) Input the linearly transformed hidden state target pose sequence features into the second regression output layer 704 to obtain the predicted driving pressure.

[0148] In some embodiments, the linearly transformed latent state target pose sequence features are input into the second regression output layer 704, which generates the final predicted driving pressure. Optionally, the target pose sequence features include multiple target pose features, and the second regression output layer 704 generates the predicted driving pressure corresponding to each target pose feature.

[0149] In this embodiment, the predicted driving pressure is generated based on the target motion trajectory using an inverse kinematics network model. The inverse kinematics network model is a network model trained based on the second general LSTM model. By using the target motion trajectory as the input of the LSTM model, the relationship between the target motion trajectory and the driving pressure can be learned. Since the LSTM model can update its internal parameters in real time, it can dynamically adjust the prediction result (predicted driving pressure) according to the latest target motion trajectory.

[0150] The positive kinematic network model generates predicted motion trajectories based on predicted driving pressure.

[0151] In some embodiments, the predicted driving pressure is input into the forward kinematics network model, which generates a corresponding predicted pose data sequence based on the predicted driving pressure, and obtains the predicted motion trajectory of the soft robotic arm based on the predicted pose data sequence (multiple predicted pose data).

[0152] In some embodiments, the inverse kinematics network model is a network model trained based on a third general LSTM model. For example, Figure 8 This is a schematic diagram of the structure of a forward kinematics network model provided in one embodiment of this application.

[0153] like Figure 8 As shown, the forward kinematics network model 800 includes a third input layer 801, a third memory layer 802, a third fully connected layer 803, and a third regression output layer 804, all cascaded sequentially. The third memory layer 802 is an LSTM layer, containing a third number of hidden units. The third fully connected layer 803 includes fully connected layers E and F; optionally, it also includes a third dropout layer. Optionally, the input to the forward kinematics network model 800 is the predicted driving pressure, and the output is the predicted motion trajectory. The steps of the forward kinematics network model 800 in generating the predicted motion trajectory based on the predicted driving pressure are as follows:

[0154] 1) The predicted driving pressure is input into the third input layer 801, and the third input layer 801 encodes the predicted driving pressure into driving pressure features;

[0155] The predicted driving pressure is the driving pressure value used to predict the movement required to drive the soft robotic arm. The driving pressure value refers to the magnitude of the pressure applied to the soft robotic arm, determining its bending degree and direction of movement. The driving pressure feature is represented by a feature vector encoded from the driving pressure value.

[0156] Optionally, the predicted driving pressure is input to the third input layer 801, that is, the driving pressure value indicated by the predicted driving pressure is input to the third input layer 801. The third input layer 801 encodes the driving pressure value indicated by the predicted driving pressure to obtain the corresponding driving pressure feature. For example, the predicted driving pressure corresponding to different target pose data sequences is input to the third input layer 801. The third input layer 801 encodes the different predicted driving pressures to obtain driving pressure feature 1, driving pressure feature 2, ... driving pressure feature n.

[0157] 2) Input the driving pressure features into the third memory layer 802, and obtain the hidden state driving pressure features after processing based on the third number of hidden units;

[0158] The third memory layer 802 is an LSTM layer, containing a third number of hidden units. For example, the third memory layer 802 includes m LSTM units, which are connected sequentially. Optionally, each LSTM unit receives the input driving pressure features at different times and updates its internal state. The input to each LSTM unit includes the driving pressure features at the current time, the hidden state at the previous time, and the unit state at the previous time. The output of each LSTM unit includes the hidden state at the current time and the unit state at the current time.

[0159] Optionally, the third memory layer 802 takes the driving pressure features as input, processes them through a third number of hidden units (which can also be understood as LSTM units), and then outputs the corresponding hidden state driving pressure features.

[0160] 3) Input the hidden state driving pressure features into the third fully connected layer 803. After performing a fully connected operation based on the third fully connected layer 803, the linearly changed hidden state driving pressure features are obtained.

[0161] The third fully connected layer 803 includes a fully connected layer E and a fully connected layer F. Optionally, the third fully connected layer 803 also includes a third dropout layer.

[0162] In one possible implementation, the third fully connected layer 803 includes a fully connected layer E, a third discard layer, and a fully connected layer F cascaded in sequence. Optionally, the hidden state driving pressure features are input into the third fully connected layer 803, the fully connected layer E performs a first fully connected layer operation, the third discard layer performs a discard operation on the features after the first fully connected layer operation, and the fully connected layer F performs a second fully connected layer operation to obtain the linearly changed hidden state driving pressure features.

[0163] In some embodiments, the hidden state driving pressure features are input into the third fully connected layer 803. The fully connected layer E performs a first linear transformation and activation function processing on the hidden state driving pressure features. The third dropout layer performs a dropout operation, which prevents overfitting by randomly discarding (setting zero) the output of a portion of neurons. After passing through the second dropout layer, the hidden state driving pressure features enter the fully connected layer F for a second linear transformation and activation function processing. After further extraction and transformation of features, the linearly transformed hidden state driving pressure features are obtained.

[0164] 4) Input the linearly transformed latent state driving pressure features into the third regression output layer 804 to obtain the predicted motion trajectory.

[0165] In some embodiments, the linearly transformed latent state driving pressure features are input into the third regression output layer 804, which generates the final predicted motion trajectory. Optionally, the target pose data sequence generates different predicted driving pressures, with each target pose data corresponding to one predicted driving pressure. The third regression output layer 804 generates predicted pose data corresponding to each driving pressure feature, and a predicted pose data sequence is obtained based on multiple predicted pose data, thus obtaining the predicted motion trajectory.

[0166] In this embodiment, a predicted motion trajectory is generated based on the predicted driving pressure using a forward kinematics network model. The forward kinematics network model is a network model trained based on a third general LSTM model. By using the predicted driving pressure as the input of the LSTM model, the relationship between the predicted driving pressure and the predicted motion trajectory can be learned. Since the LSTM model can update its internal parameters in real time, it can dynamically adjust the prediction result (predicted motion trajectory) according to the latest predicted driving pressure.

[0167] In the motion control of the soft robotic arm, a control method based on a driving pressure correction model, an inverse kinematics network model, and a forward kinematics network model is implemented. Training samples can be obtained in advance to train the driving pressure correction model, the inverse kinematics network model, and the forward kinematics network model. These three models are trained using reinforcement learning. By continuously optimizing the network parameters of the models, the accuracy of the actual driving pressure on the soft robotic arm will be higher, and the actual motion estimate of the soft robotic arm will be closer to the target motion trajectory.

[0168] In some embodiments, the driving pressure correction model, inverse kinematics network model, and forward kinematics network model are trained based on an LSTM model with the same network architecture. Optionally, the network parameters of the driving pressure correction model, inverse kinematics network model, and forward kinematics network model are different. That is, the driving pressure correction model, inverse kinematics network model, and forward kinematics network model are LSTM models with the same network architecture but different network parameters.

[0169] Before introducing how to train the model, let's first introduce how the data in the training samples are collected.

[0170] Collection of motion data (data indicating motion trajectory)

[0171] In some embodiments, the actual motion trajectory is determined based on a motion capture system on the soft robotic arm; or, the actual motion trajectory is determined based on a camera vision sensor on the soft robotic arm.

[0172] In this embodiment, the actual motion trajectory of the soft robotic arm can be monitored in real time through a motion capture system or a camera vision sensor, providing instant motion data for training the inverse motion network model.

[0173] In some embodiments, the soft robotic arm is composed of multiple individual soft joints connected in series; that is, the soft robotic arm includes multiple soft joints, and each soft joint includes soft actuators arranged in parallel and evenly. For example, a single soft joint includes a first soft actuator, a second soft actuator, and a third soft actuator arranged in parallel and evenly. Even arrangement means that the axes of the three soft actuators are parallel, and the distance between any two axes is the same or similar. Optionally, the soft robotic arm includes multiple soft joints, and the soft actuators along the entire path are connected via an internal air passage; that is, the driving pressure of the soft actuators along the entire path of the soft robotic arm is the same.

[0174] In some embodiments, to simplify the actual motion trajectory acquisition process of the soft robotic arm and improve the usability of the acquired data, several motion modes are designed according to the structure of the soft robotic arm. Each of the several motion modes is one of n basic motion modes. Optionally, a soft joint of the soft robotic arm is randomly driven to perform one of the n basic motion modes, which is taken as the current motion mode among the several motion modes. Based on the motion capture system on the soft robotic arm, the motion trajectory of the soft robotic arm in the current motion mode is acquired. After a preset time, the step of randomly driving a joint of the soft robotic arm to perform one of the n basic motion modes as the current motion mode is repeated.

[0175] For example, the n basic motion modes include at least one of a first basic motion mode, a second basic motion mode, a third basic motion mode, and a fourth basic motion mode. For instance, a soft joint in a soft robotic arm is randomly driven to perform a first basic motion mode, which is then used as the current motion mode. The current first basic motion mode is captured by a motion capture system on the soft robotic arm. After a 1-second pause, the fourth basic motion mode is repeatedly performed by randomly driving a joint in the soft robotic arm, and the motion is captured again by the motion capture system.

[0176] In this embodiment, the soft joints of the soft robotic arm are randomly driven to execute different basic motion modes, ensuring the diversity and richness of the collected data. Motion trajectory acquisition based on a motion capture system can accurately record the actual motion trajectory of the soft robotic arm in each motion mode, guaranteeing the accuracy and reliability of the data. Repeating the motion mode after a preset duration can capture dynamic changes in continuous motion, enhancing the temporal consistency and coherence of the data.

[0177] In some embodiments, the actual motion trajectory includes a number of sequentially ordered motion modes, each of which is one of n basic motion modes.

[0178] In one possible implementation, n basic motion modes respectively realize the bending of the soft joint forward and backward around the X-axis and left and right around the Y-axis.

[0179] For example, in conjunction with reference Figure 9 , Figure 9 The image shows a single soft joint of a soft robotic arm achieving four motion modes: forward and backward rotation around the X-axis, and left and right rotation around the Y-axis. For example... Figure 9 As shown in part (1) of the figure, the soft robotic arm has three soft drivers, namely the first soft driver 1, the second soft driver 2 and the third soft driver 3. Figure 9 The first basic motion mode (turning left around the Y-axis) shown in part (1) of the figure includes the first software actuator 1 in an increasing pressure state, and the second software actuator 2 and the third software actuator 3 in a decreasing pressure state. Figure 9 As shown in part (2) of the figure, the soft robotic arm has three soft drivers, namely the first soft driver 1, the second soft driver 2 and the third soft driver 3. Figure 9 The second basic motion mode (rightward bend around the Y-axis) shown in part (2) of the figure is driven by the first software actuator 1 in a pressure reduction state, and the second software actuator 2 and the third software actuator 3 in a pressure increase state. Figure 9 As shown in part (3) of the figure, the soft robotic arm has three soft drivers, namely the first soft driver 1, the second soft driver 2 and the third soft driver 3. Figure 9 The diagram in part (3) shows the third basic motion mode (backward bending around the X-axis). The driving methods of the third basic motion mode include the first software actuator 1 being in a constant pressure state, the second software actuator 2 being in a reduced pressure state, and the third software actuator 3 being in a increased pressure state. Figure 9As shown in part (4) of the figure, the soft robotic arm has three soft drivers, namely the first soft driver 1, the second soft driver 2 and the third soft driver 3. Figure 9 The fourth basic motion mode (forward bending around the X-axis) shown in part (4) of the figure includes the first software actuator 1 in a constant pressure state, the second software actuator 2 in an increased pressure state, and the third software actuator 3 in a decreased pressure state.

[0180] In this embodiment, several motion modes are combined from various basic motion modes, enabling the collected motion data (motion trajectory indication data) to cover various motion states and changes of the soft robotic arm, thereby improving the comprehensiveness of the motion data used for training.

[0181] For driving pressure data acquisition

[0182] In some embodiments, the actual driving pressure is determined based on pressure sensors on the soft robotic arm. The pressure sensors provide real-time driving pressure data, enabling the acquisition of real-time driving pressure data when training the forward kinematics network model.

[0183] In one possible implementation, when training the soft robotic arm by acquiring actual driving pressure, a pressure sensor can be placed in each of the soft actuators of the soft robotic arm to measure the pressure value in each soft actuator. Optionally, the data (actual driving pressure) collected by the pressure sensor is collected at a preset frequency, for example, at a frequency of 20 Hz.

[0184] After introducing the collection of training data, the training of the inverse kinematics network model, the forward kinematics network model, and the driving pressure correction model will be introduced in turn.

[0185] For inverse kinematics network models

[0186] In some embodiments, the inverse kinematics network model is a network model trained based on the second general LSTM model.

[0187] Figure 10 A flowchart of a control method for a soft robotic arm provided in an exemplary embodiment of this application is shown, the method including steps 310, 320 and 330.

[0188] Step 310: Obtain the second training sample;

[0189] The second training sample includes at least two target prediction trajectories and a second label, whereby the second label is used to indicate the target driving pressure corresponding to the target prediction trajectory.

[0190] In some embodiments, a second training sample is obtained, which is a set of samples used to train the inverse kinematics network model. Optionally, the second training sample includes at least two target prediction trajectories, each of which is the motion path or trajectory that the soft robotic arm is expected to achieve when performing a task. Optionally, the target motion trajectory is used to indicate the target pose data sequence of the soft robotic arm; that is, the target motion trajectory is used to indicate multiple target pose data of the soft robotic arm. Each target pose data defines the target position and orientation of the soft robotic arm at a certain point in time. Through a series of continuous target pose data, the target path of the soft robotic arm from the starting point to the ending point is depicted. Optionally, a second label corresponding to each target prediction trajectory is used to indicate the target driving pressure corresponding to the target motion trajectory.

[0191] In some embodiments, when training the inverse kinematics network model, the target motion trajectory can be collected as the actual motion trajectory. Optionally, the actual motion trajectory can be determined based on the motion capture system on the soft robotic arm; or, the actual motion trajectory can be determined based on the camera vision sensor on the soft robotic arm.

[0192] In some embodiments, the inverse kinematics network model is an LSTM model trained based on a second training sample. The second training sample includes the actual motion trajectory collected by the motion capture system and the corresponding driving pressure data. The actual motion trajectory collected by the motion capture system serves as the training data in the second training sample, and the driving pressure data collected by the pressure sensor serves as the second label.

[0193] In some embodiments, the target motion trajectory (here, the target motion trajectory can be regarded as the collected motion data) can be based on the above-mentioned " Collection of motion data (data indicating motion trajectory) The data is collected in the manner indicated by the chapter, and the collected motion data is used as the second training sample input to the inverse kinematics network model; the target driving pressure can be based on the above. For driving pressure data acquisition The data is collected in the manner indicated by the chapter, and the collected driving pressure data is used as the second label corresponding to the second training sample.

[0194] Step 320: Input the second training sample into the second general LSTM model to obtain the prediction driving pressure for each target prediction trajectory;

[0195] In some embodiments, the second training samples are input into the second general LSTM model, that is, each target predicted trajectory is used as the input of the second general LSTM model, and the output is the predicted driving pressure corresponding to the target pose data sequence indicated by each target predicted trajectory. The model structure of the second general LSTM model can be referred to... Figure 7 The inverse kinematics network model 700 is shown.

[0196] Step 330: Adjust the network parameters of the second general LSTM model based on the sample loss of the second training samples to obtain the trained inverse kinematics network model.

[0197] The sample loss of the second training sample is used to indicate the loss between the predicted driving pressure and the target driving pressure.

[0198] In some embodiments, the output of the second general LSTM model is the predicted driving pressure, and comparing the predicted driving pressure with the target driving pressure yields the sample loss of the second training samples. The sample loss of the second training samples is used to indicate the error between the predicted driving pressure and the target driving pressure.

[0199] Optionally, backpropagation can be used to adjust the network parameters of the second general LSTM model, making the prediction-driven pressure and the target-driven pressure of the second general LSTM model closer. Optionally, the second general LSTM model can be optimized using a second loss function. The second loss function can be any type of loss function, such as the cross-entropy loss function or the mean squared error loss function. After the above training, an inverse kinematics network model can be obtained.

[0200] It should be noted that the training of the inverse kinematics network model provided in this embodiment is only illustrative and does not constitute a limitation on the training method of the inverse kinematics network model.

[0201] In this embodiment, an inverse kinematics network model is obtained by training a second general LSTM model. This second general LSTM model can capture the long-term and short-term dependencies of the target motion trajectory, thereby more accurately predicting the driving pressure. This method can effectively reduce the difference between the predicted driving pressure and the target driving pressure, improving the prediction accuracy of the inverse kinematics network model. By adjusting the network parameters of the second general LSTM model based on the sample loss of the second training samples, it is ensured that the second general LSTM model can be gradually optimized, ultimately obtaining an inverse kinematics network model that can accurately predict the driving pressure. This automated adjustment process can improve the adaptability and robustness of the inverse kinematics network model.

[0202] For positive kinematic network models

[0203] In some embodiments, the forward kinematics network model is a network model trained based on the third general LSTM model.

[0204] Figure 11 A flowchart of a control method for a soft robotic arm provided in an exemplary embodiment of this application is shown, the method including steps 410, 420 and 430.

[0205] Step 410: Obtain the third training sample;

[0206] The third training sample includes at least two predicted driving pressures and a third label, which is used to indicate the actual predicted motion trajectory corresponding to the predicted driving pressure.

[0207] In some embodiments, a third training sample is obtained, which is a set of samples used to train the forward kinematics network model. Optionally, the third training sample includes at least two predicted driving pressures, which are driving pressure values ​​required to predict the motion of the soft robotic arm. The driving pressure value refers to the magnitude of the pressure applied to the soft robotic arm, determining the degree of bending and direction of motion of the soft robotic arm. Optionally, a third label corresponding to each predicted driving pressure is used to indicate the actual predicted motion trajectory corresponding to the predicted driving pressure.

[0208] In some embodiments, when training the forward kinematics network model, the predicted driving pressure can be treated as the actual driving pressure and collected. Optionally, the actual driving pressure can be determined based on a pressure sensor on the soft robotic arm. The pressure sensor can provide real-time driving pressure data, enabling the acquisition of real-time driving pressure data during the training of the forward kinematics network model.

[0209] In some embodiments, the forward kinematics network model is an LSTM model trained based on a third training sample. The third training sample includes driving pressure data collected by a pressure sensor and the actual motion trajectory corresponding to the driving pressure data. The driving pressure data collected by the pressure sensor serves as the training data in the third training sample, and the actual motion trajectory collected by the motion capture system serves as the third label.

[0210] In some embodiments, the predicted driving pressure (here, the predicted driving pressure can be considered as the driving pressure data collected by the pressure sensor) can be based on the above-mentioned " For driving pressure data acquisition The data is collected in the manner indicated by the chapter, and the collected driving pressure data is used as the third training sample input to the positive kinematics network model; the predicted motion trajectory (the predicted driving pressure here can be regarded as the collected motion data) can be based on the above. For motion data (data indicating motion trajectory) collection The data is collected in the manner indicated by the chapter, and the collected motion data is used as the third label corresponding to the third training sample.

[0211] Step 420: Input the third training sample into the third general LSTM model to obtain the predicted motion trajectory for each predicted driving pressure;

[0212] In some embodiments, the third training sample is input into the third general LSTM model, that is, each predicted driving pressure is used as the input of the third general LSTM model, and the output is the predicted motion trajectory corresponding to the driving pressure value indicated by each predicted driving pressure. The model structure of the third general LSTM model can be referred to... Figure 8 The positive kinematic network model 800 is shown.

[0213] Step 430: Adjust the network parameters of the third general LSTM model based on the sample loss of the third training sample to obtain the trained forward kinematics network model.

[0214] The sample loss of the third training sample is used to indicate the loss between the predicted motion trajectory and the actual predicted motion trajectory.

[0215] In some embodiments, the output of the third general LSTM model is a predicted motion trajectory. Comparing the predicted motion trajectory with the actual predicted motion trajectory yields the sample loss of the third training sample. The sample loss of the third training sample indicates the error between the predicted motion trajectory and the actual predicted motion trajectory.

[0216] Optionally, backpropagation can be used to adjust the network parameters of the third general LSTM model, making the predicted motion trajectory of the third general LSTM model closer to the actual predicted motion trajectory. Optionally, the third general LSTM model can be optimized using a third loss function. The third loss function can be any loss function, for example, it can be the cross-entropy loss function or the mean squared error loss function. After the above training, a forward kinematics network model can be obtained.

[0217] It should be noted that the training of the forward kinematics network model provided in this embodiment is only illustrative and does not constitute a limitation on the training method of the forward kinematics network model.

[0218] In this embodiment, a forward kinematics network model is obtained by training a third general LSTM model. This third general LSTM model can capture the long-term and short-term dependencies of the predicted driving pressure, thus predicting the motion trajectory more accurately. This method effectively reduces the difference between the predicted motion trajectory and the actual predicted motion trajectory, improving the prediction accuracy of the forward kinematics network model. By adjusting the network parameters of the third general LSTM model based on the sample loss from the third training samples, the third general LSTM model is gradually optimized, ultimately resulting in a forward kinematics network model that can accurately predict the motion trajectory. This automated adjustment process enhances the adaptability and robustness of the forward kinematics network model.

[0219] For the driving pressure correction model

[0220] In some embodiments, the driving pressure correction model is a network model trained based on a first general LSTM model.

[0221] Figure 12 A flowchart of a control method for a soft robotic arm provided in an exemplary embodiment of this application is shown, the method including steps 510, 520 and 530.

[0222] Step 510: Obtain the first training sample;

[0223] The first training sample includes at least two trajectory errors and a first label, which is used to indicate the actual driving pressure correction value corresponding to the trajectory error.

[0224] In some embodiments, the driving pressure compensation is based on an LSTM model trained on a first training sample. The first training sample includes a first difference and a second difference corresponding to the first difference. The first difference is the difference between the predicted motion trajectory generated by the forward kinematics network model and the target motion trajectory, and the second difference is the difference between the predicted driving pressure generated by the inverse kinematics network model and the driving pressure data collected by the pressure sensor.

[0225] In some embodiments, a first training sample is obtained, which is a set of samples used to train the driving pressure correction model. Optionally, the first training sample includes at least two trajectory errors, each trajectory error being a pose data sequence error used to indicate the difference between the target pose data sequence and the predicted pose data sequence. Optionally, a first label corresponding to each trajectory error is used to indicate the actual driving pressure correction value corresponding to the trajectory error.

[0226] In some embodiments, after training the inverse kinematics network model and the forward kinematics network model, the predicted driving pressure generated by the inverse kinematics network model is input into the forward kinematics network model. The forward kinematics network model can generate a predicted motion trajectory. Based on the predicted motion trajectory and the target motion trajectory (here, the target motion trajectory refers to the expected trajectory of the soft robotic arm), trajectory errors can be obtained. Based on multiple target motion trajectories and multiple predicted motion trajectories, multiple trajectory errors can be obtained. These multiple trajectory errors are used as the first training samples for training the driving pressure correction model. The difference between the predicted driving pressure generated by the inverse kinematics network model based on the target motion trajectory and the driving pressure data collected by the pressure sensor is used as the first label corresponding to the first training sample.

[0227] Step 520: Input the first training sample into the first general LSTM model to obtain the driving pressure correction value for each trajectory error;

[0228] In some embodiments, the first training samples are input into the first general LSTM model, that is, each trajectory error is used as the input of the first general LSTM model, and the output is a driving pressure correction value for each trajectory error. Here, each trajectory error refers to the difference between the target motion trajectory and the predicted motion trajectory. The model structure of the first general LSTM model can be referenced from... Figure 6 The driving pressure correction model 600 is shown.

[0229] Optionally, the driving pressure correction value can be expressed as:

[0230] ΔP=f(Err traj )

[0231] Where ΔP represents the driving pressure correction value, Err traj Let f represent the trajectory error, and let f represent the first general LSTM model, which can also be understood as f representing the driving pressure correction model.

[0232] In some embodiments, the actual driving pressure correction value cannot be determined by experimental measurement during the training of the driving pressure correction model. Therefore, the actual driving pressure correction value can be determined by the gradient descent optimization method.

[0233] Step 530: Adjust the network parameters of the first general LSTM model based on the sample loss of the first training sample to obtain the trained driving pressure correction model.

[0234] The sample loss of the first training sample is used to indicate the loss between the actual driving pressure correction value and the predicted driving pressure correction value.

[0235] In some embodiments, the output of the first general LSTM model is a predicted drive pressure correction value. Comparing the predicted drive pressure correction value with the actual drive pressure correction value yields the sample loss of the first training sample. The sample loss of the first training sample indicates the error between the actual drive pressure correction value and the actual drive pressure correction value.

[0236] Optionally, the backpropagation algorithm is used to adjust the network parameters of the first general LSTM model, making the predicted driving pressure correction value of the first general LSTM model closer to the actual driving pressure correction value. Optionally, the first general LSTM model is optimized using a first loss function. The first loss function can be any loss function, for example, it can be the CE loss (Cross Entropy loss) function or the Mean Square Error (MSE) loss function. After the above training, the driving pressure correction model can be obtained.

[0237] It should be noted that the training of the driving pressure correction model provided in this embodiment is only illustrative and does not constitute a limitation on the training method of the driving pressure correction model.

[0238] In this embodiment, a driving pressure correction model is obtained by training a first general LSTM model. This first general LSTM model can capture the long-term and short-term dependencies of trajectory errors, thereby more accurately predicting the driving pressure correction value. This method effectively reduces trajectory errors and improves the prediction accuracy of the driving pressure correction model. By adjusting the network parameters of the first general LSTM model based on the sample loss of the first training samples, the first general LSTM model is gradually optimized, ultimately resulting in a driving pressure correction model that can accurately predict the driving pressure correction value. This automated adjustment process enhances the adaptability and robustness of the driving pressure correction model.

[0239] In some embodiments, the driving pressure correction model, inverse kinematics network model, and forward kinematics network model are trained based on an LSTM model with the same network architecture. Optionally, the network parameters of the driving pressure correction model, inverse kinematics network model, and forward kinematics network model are different. That is, the driving pressure correction model, inverse kinematics network model, and forward kinematics network model are LSTM models with the same network architecture but different network parameters.

[0240] In this embodiment, the driving pressure correction model, inverse kinematics network model, and forward kinematics network model are trained using LSTM models with the same network architecture but different network parameters. Using the same network architecture simplifies the model design process, eliminating the need to design different architectures for each task, thus saving development time and resources. Each model uses independent network parameters, allowing for optimization for specific tasks, thereby improving the performance of each model.

[0241] For general LSTM models, the number of hidden units in the memory layer (LSTM layer), the size of the fully connected layer, and the initial learning rate of the model are three important parameters that determine the training effect of the model. The optimal network parameters are different for different dimensions of input and output data. Parameter tuning can be determined through multiple experiments, Bayesian optimization, etc. Table 1 shows a set of better network parameters.

[0242] Table 1

[0243] Number of hidden units in LSTM layer Fully connected layer size Initial learning rate Forward kinematic network model 164 70 0.015 Inverse kinematics network model 160 196 0.015 Driven pressure correction model 128 64 0.02

[0244] It should be noted that the network parameters in Table 1 above are merely illustrative examples.

[0245] Beneficial effects of this plan

[0246] The proposed soft robotic arm motion and driving pressure prediction system can train a neural network (inverse kinematics network model, forward kinematics network model, and driving pressure correction model) for soft robotic arm motion control with a small amount of training data, and can greatly improve the motion trajectory accuracy of the soft robotic arm compared with a single inverse kinematics network. Figure 13 The images show comparisons between the predicted trajectories of the soft robotic arm and the target motion trajectory, using circular, rhomboid, and hexagonal trajectories with and without air pressure compensation networks. For example, Figure 13 Figure (1) shows the motion trajectory of the soft robotic arm when the circular trajectory is used. a1 represents the path of the target trajectory (target motion trajectory), b1 represents the path of the predicted trajectory (predicted motion trajectory), and c1 represents the path of the correction trajectory (motion trajectory jointly controlled by the drive pressure correction value and the predicted drive pressure). Figure 13 Figure (2) shows the motion trajectory of the soft robotic arm when the rhomboid trajectory is formed. a2 represents the path of the target trajectory (target motion trajectory), b2 represents the path of the predicted trajectory (predicted motion trajectory), and c2 represents the path of the correction trajectory (motion trajectory jointly controlled by the correction value of the driving pressure and the predicted driving pressure). Figure 13 Figure (3) shows the motion trajectory of the soft robotic arm when using a hexagonal trajectory. a3 represents the path of the target trajectory (target motion trajectory), b3 represents the path of the predicted trajectory (predicted motion trajectory), and c3 represents the path of the corrected trajectory (motion trajectory jointly controlled by the drive pressure correction value and the predicted drive pressure). Figure 13 It can be seen that the motion trajectory after the driving pressure correction value is closer to the target motion trajectory.

[0247] Table 2 calculates the root mean square error in the X and Y directions for the three trajectories. The calculation results show that the embodiments of this application can significantly improve the accuracy of the output predicted motion trajectory, thereby improving the accuracy of motion control of the soft robotic arm.

[0248] Table 2

[0249] Root mean square error without pressure compensation Root mean square error with pressure correction round (0.0051，0.0042) (0.0015，0.0009) diamond (0.0050，0.0041) (0.0020，0.0013) hexagon (0.0055，0.0040) (0.0019，0.0013)

[0250] It should be noted that the root mean square error in Table 2 above, with and without pressure compensation, is only an illustrative example.

[0251] Figure 14 This diagram illustrates a structural block diagram of a control device for a soft robotic arm according to an embodiment of this application. This control device for the soft robotic arm has the functionality to implement the control method example described above. The functionality can be implemented in hardware or by hardware executing corresponding software. The device can be the server described above, or it can be located within a server. Figure 14As shown, the device 1400 may include: an acquisition module 1410, a generation module 1420, and a control module 1430.

[0252] The acquisition module 1410 is used to acquire the target motion trajectory of the soft robotic arm, and the target motion trajectory is used to indicate the target pose data sequence of the soft robotic arm;

[0253] The generation module 1420 is used to generate a predicted driving pressure based on the target predicted trajectory through an inverse kinematics network model. The predicted driving pressure is a driving pressure value required to predict the movement of the soft robotic arm.

[0254] The generation module 1420 is used to generate a predicted motion trajectory based on the predicted driving pressure using a positive kinematics network model. The predicted motion trajectory is used to indicate the predicted pose data sequence of the soft robotic arm.

[0255] The generation module 1420 is used to obtain a trajectory error based on the target motion trajectory and the predicted motion trajectory, wherein the trajectory error is used to indicate the pose data sequence error between the target pose data sequence and the predicted pose data sequence;

[0256] The control module 1430 is used to generate a drive pressure correction value based on the trajectory error through a drive pressure correction model, and to control the movement of the soft robotic arm according to the drive pressure correction value and the predicted drive pressure.

[0257] In some embodiments, the control module 1430 includes a control submodule.

[0258] In an optional example, a control submodule is configured to generate a drive pressure correction value based on the trajectory error using the drive pressure correction model with the goal of minimizing the trajectory error, and control the movement of the soft robotic arm based on the drive pressure correction value and the predicted drive pressure.

[0259] In some embodiments, the control submodule includes an input unit.

[0260] In an optional example, an input unit is used to input the trajectory error into the first input layer to obtain trajectory error features;

[0261] The input unit is used to input the trajectory error features into the first memory layer, and obtain the hidden state trajectory error features after processing based on the first number of hidden units;

[0262] The input unit is used to input the hidden state trajectory error features into the first fully connected layer, and after performing a fully connected operation based on the first fully connected layer, obtain the linearly changed hidden state trajectory error features.

[0263] The input unit is used to input the linearly changed hidden state trajectory error features into the first regression output layer to obtain the driving pressure correction value.

[0264] In an optional example, the input unit is used to input the target motion trajectory into the second input layer to obtain the target pose sequence features;

[0265] The input unit is used to input the target pose sequence features into the second memory layer, and obtain the hidden state target pose sequence features after processing based on the second number of hidden units;

[0266] The input unit is used to input the hidden state target pose sequence features into the second fully connected layer, and after performing a fully connected operation based on the second fully connected layer, the linearly transformed hidden state target pose sequence features are obtained.

[0267] The input unit is used to input the linearly transformed hidden state target pose sequence features into the second regression output layer to obtain the predicted driving pressure.

[0268] In an optional example, an input unit is used to input the predicted driving pressure into the third input layer to obtain driving pressure features;

[0269] The input unit is used to input the driving pressure feature into the third memory layer, and obtain the hidden state driving pressure feature after processing based on the third number of hidden units;

[0270] The input unit is used to input the hidden state driving pressure features into the third fully connected layer, and after performing a fully connected operation based on the third fully connected layer, the linearly changed hidden state driving pressure features are obtained.

[0271] The input unit is used to input the linearly changed latent state driving pressure features into the third regression output layer to obtain the predicted motion trajectory.

[0272] In some embodiments, the device 1400 further includes an input module and an adjustment module.

[0273] In an optional example, the acquisition module 1410 is used to acquire a first training sample, the first training sample including at least two trajectory errors and a first label, the first label being used to indicate the actual driving pressure correction value corresponding to the trajectory error;

[0274] The input module is used to input the first training sample into the first general LSTM long short-term memory network model to obtain the driving pressure correction value for each trajectory error;

[0275] The adjustment module is used to adjust the network parameters of the first general LSTM model based on the sample loss of the first training sample to obtain the trained driving pressure correction model. The sample loss of the first training sample is used to indicate the loss between the actual driving pressure correction value and the predicted driving pressure correction value.

[0276] In an optional example, the acquisition module 1410 is used to acquire a second training sample, the second training sample including at least two target prediction trajectories and a second label, the second label being used to indicate the target driving pressure corresponding to the target prediction trajectory;

[0277] The input module is used to input the second training sample into the second general LSTM model to obtain the prediction driving pressure for each target prediction trajectory;

[0278] The adjustment module is used to adjust the network parameters of the second general LSTM model based on the sample loss of the second training sample to obtain the trained inverse kinematics network model. The sample loss of the second training sample is used to indicate the loss between the predicted driving pressure and the target driving pressure.

[0279] In an optional example, module 1410 is used to acquire a third training sample, the third training sample including at least two predicted driving pressures and a third label, the third label being used to indicate the actual motion trajectory corresponding to the predicted driving pressure;

[0280] The input module is used to input the third training sample into the third general LSTM model to obtain the predicted motion trajectory for each predicted driving pressure.

[0281] The adjustment module is used to adjust the network parameters of the third general LSTM model based on the sample loss of the third training sample to obtain the trained forward kinematics network model. The sample loss of the third training sample is used to indicate the loss between the predicted motion trajectory and the actual motion trajectory.

[0282] In an optional example, the driving pressure correction model, the inverse kinematics network model, and the forward kinematics network model are LSTM models with the same network architecture but different network parameters.

[0283] In some embodiments, the apparatus 1400 further includes a determining module.

[0284] In one optional example, the determining module is used to determine the actual motion trajectory based on the motion capture system on the soft robotic arm; or, based on the camera vision sensor on the soft robotic arm, to determine the actual motion trajectory.

[0285] In some embodiments, the determining module includes a driving submodule and an acquisition submodule.

[0286] In an optional example, the soft robotic arm includes multiple soft joints, each soft joint including a first soft actuator, a second soft actuator, and a third soft actuator arranged side by side and evenly, and the actual motion trajectory includes several motion modes ordered in sequence, each of the several motion modes being one of n basic motion modes;

[0287] The driving submodule is used to randomly drive one of the soft joints of the soft robotic arm to perform one of the n basic motion modes, which is the current motion mode among the several motion modes;

[0288] The acquisition submodule acquires the motion trajectory of the soft robotic arm in the current motion mode based on the motion capture system on the soft robotic arm;

[0289] The driving submodule is used to repeatedly execute the step of randomly driving one joint of the soft robotic arm to perform one of the n basic motion modes after a preset time, as the current motion mode among the plurality of motion modes.

[0290] In an optional example, the n basic action modes include at least one of the following: a first basic action mode, a second basic action mode, a third basic action mode, and a fourth basic action mode:

[0291] The driving mode of the first basic action mode includes the first software driver being in an increasing pressure state, and the second software driver and the third software driver being in a decreasing pressure state;

[0292] The driving mode of the second basic action mode includes the first software driver being in a pressure reduction state, and the second software driver and the third software driver being in a pressure increase state;

[0293] The driving mode of the third basic action mode includes the first software driver being in a constant pressure state, the second software driver being in a reduced pressure state, and the third software driver being in a increased pressure state.

[0294] The driving mode of the fourth basic action mode includes the first software driver being in a constant pressure state, the second software driver being in a pressure-increasing state, and the third software driver being in a pressure-decreasing state.

[0295] In an optional example, a determination module is used to determine the actual driving pressure based on a pressure sensor on the soft robotic arm.

[0296] It should be noted that the specific limitations of the control device for the one or more soft robotic arms provided above can be found in the limitations of the control method for soft robotic arms mentioned above, and will not be repeated here. Each module of the above-mentioned purification device can be implemented entirely or partially through software, hardware, or a combination thereof. Each module can be embedded in the processor of the computer device in hardware form or independent of the processor, or it can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0297] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0298] Figure 15 A structural block diagram of a computer device 1500 illustrating an exemplary embodiment of this application is shown. This computer device can be used to implement the control method for the soft robotic arm provided in the above embodiments. The computer device 1500 includes a Central Processing Unit (CPU) 1501, a system memory 1504 including Random Access Memory (RAM) 1502 and Read-Only Memory (ROM) 1503, and a system bus 1505 connecting the system memory 1504 and the CPU 1501. The computer device 1500 also includes a basic input / output system (I / O system) 1506 to facilitate information transfer between various devices within the computer device, and a mass storage device 1507 for storing an operating system 1513, application programs 1514, and other program modules 1515.

[0299] The basic input / output system 1506 includes a display 1508 for displaying information and an input device 1509 for user input, such as a mouse or keyboard. Both the display 1508 and the input device 1509 are connected to the central processing unit 1501 via an input / output controller 1510 connected to the system bus 1505. The basic input / output system 1506 may also include the input / output controller 1510 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 1510 also provides output to a display screen, printer, or other types of output devices.

[0300] The mass storage device 1507 is connected to the central processing unit 1501 via a mass storage controller (not shown) connected to the system bus 1505. The mass storage device 1507 and its associated computer-readable storage media provide non-volatile storage for the terminal device 1500. That is, the mass storage device 1507 may include computer-readable storage media (not shown), such as a hard disk or a compact disc read-only memory (CD-ROM) drive.

[0301] Without loss of generality, the computer-readable storage medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable storage instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state storage technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer storage medium is not limited to the above-mentioned types. The system memory 1504 and mass storage device 1507 described above can be collectively referred to as memory.

[0302] The memory stores one or more programs, which are configured to be executed by one or more central processing units 1501. The one or more programs contain instructions for implementing the above method embodiments. The central processing unit 1501 executes the one or more programs to implement the control method of the soft robotic arm provided by the above method embodiments.

[0303] According to various embodiments of this application, the computer device 1500 can also be connected to a remote terminal device on a network, such as the Internet. That is, the computer device 1500 can be connected to a network 1512 via a network interface unit 1511 connected to the system bus 1505, or the network interface unit 1511 can be used to connect to other types of networks or remote terminal device systems (not shown).

[0304] The memory further includes one or more programs stored in the memory, and the one or more programs include steps for performing the control method of the soft robotic arm executed by the terminal device in the method provided in the embodiments of this application.

[0305] This application also provides a computer-readable storage medium storing at least one computer program, which is loaded and executed by a processor to implement the control method of the soft robotic arm provided in the above-described method embodiments.

[0306] This application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. The computer program is read from and executed by a processor of a computer device, causing the computer device to perform the control method of the soft robotic arm provided in the above-described method embodiments.

[0307] It is understood that, in the specific embodiments of this application, the data involved, historical data, and user data processing related to user identity or characteristics, such as profiles, require user permission or consent when applied to specific products or technologies. Furthermore, the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0308] It should be noted that, unless otherwise expressly defined herein, all terms used in the claims are to be interpreted according to their ordinary meaning in the technical field. Unless otherwise expressly stated, all references to "an element, device, component, apparatus, step, etc." are openly interpreted as referring to at least one instance of an element, device, component, apparatus, step, etc. Unless expressly stated otherwise, the steps of any method disclosed herein are not necessarily to be performed in the exact order disclosed.

[0309] It should be understood that "multiple" as used in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

Claims

1. A control method for a soft robotic arm, characterized in that, The method includes: The target motion trajectory of the soft robotic arm is obtained, and the target motion trajectory is used to indicate the target pose data sequence of the soft robotic arm; A predicted driving pressure is generated based on the target motion trajectory using an inverse kinematics network model. The predicted driving pressure is the driving pressure value required to predict the motion of the soft robotic arm. A predicted motion trajectory is generated based on the predicted driving pressure using a positive kinematics network model. The predicted motion trajectory is used to indicate the predicted pose data sequence of the soft robotic arm. The trajectory error is obtained based on the target motion trajectory and the predicted motion trajectory. The trajectory error is used to indicate the pose data sequence error between the target pose data sequence and the predicted pose data sequence. The driving pressure correction value is generated based on the trajectory error by the driving pressure correction model, and the movement of the soft robotic arm is controlled according to the driving pressure correction value and the predicted driving pressure.

2. The method according to claim 1, characterized in that, The step of generating a drive pressure correction value based on the trajectory error using a drive pressure correction model, and controlling the movement of the soft robotic arm according to the drive pressure correction value and the predicted drive pressure, includes: With the goal of minimizing the trajectory error, the driving pressure correction value is generated based on the trajectory error by the driving pressure correction model, and the movement of the soft robotic arm is controlled according to the driving pressure correction value and the predicted driving pressure.

3. The method according to claim 2, characterized in that, The driving pressure correction model includes a first input layer, a first memory layer, a first fully connected layer and a first regression output layer cascaded in sequence, wherein the first memory layer includes a first number of hidden units; The step of generating the drive pressure correction value based on the trajectory error using the drive pressure correction model, with the goal of minimizing the trajectory error, includes: The trajectory error is input into the first input layer to obtain the trajectory error features; The trajectory error features are input into the first memory layer and processed based on the first number of hidden units to obtain the hidden state trajectory error features. The hidden state trajectory error features are input into the first fully connected layer. After performing a fully connected operation based on the first fully connected layer, the linearly changed hidden state trajectory error features are obtained. The linearly transformed hidden state trajectory error features are input into the first regression output layer to obtain the driving pressure correction value.

4. The method according to any one of claims 1 to 3, characterized in that, The inverse kinematics network model includes a second input layer, a second memory layer, a second fully connected layer, and a second regression output layer cascaded in sequence, wherein the second memory layer includes a second number of hidden units. The step of generating predicted driving pressure based on the target predicted trajectory using the inverse kinematics network model, wherein the predicted driving pressure is a predicted pressure value used to drive the movement of the soft robotic arm, including: The target motion trajectory is input into the second input layer to obtain the target pose sequence features; The target pose sequence features are input into the second memory layer and processed based on the second number of hidden units to obtain the hidden state target pose sequence features. The hidden state target pose sequence features are input into the second fully connected layer. After performing a fully connected operation based on the second fully connected layer, the linearly transformed hidden state target pose sequence features are obtained. The linearly transformed hidden state target pose sequence features are input into the second regression output layer to obtain the predicted driving pressure.

5. The method according to any one of claims 1 to 3, characterized in that, The positive kinematics network model includes a third input layer, a third memory layer, a third fully connected layer, and a third regression output layer cascaded in sequence, wherein the third memory layer includes a third number of hidden units; The process involves generating a predicted motion trajectory based on the predicted driving pressure using the positive kinematics network model. This predicted motion trajectory indicates the predicted pose data sequence of the soft robotic arm, including: The predicted driving pressure is input into the third input layer to obtain the driving pressure characteristics; The driving pressure features are input into the third memory layer and processed based on the third number of hidden units to obtain the hidden state driving pressure features. The hidden state driving pressure feature is input into the third fully connected layer. After performing a fully connected operation based on the third fully connected layer, the linearly changed hidden state driving pressure feature is obtained. The linearly transformed latent state driving pressure features are input into the third regression output layer to obtain the predicted motion trajectory.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Obtain a first training sample, which includes at least two trajectory errors and a first label, wherein the first label is used to indicate the actual driving pressure correction value corresponding to the trajectory error; The first training sample is input into the first general LSTM long short-term memory network model to obtain the driving pressure correction value for each trajectory error; The network parameters of the first general LSTM model are adjusted based on the sample loss of the first training sample to obtain the trained driving pressure correction model. The sample loss of the first training sample is used to indicate the loss between the actual driving pressure correction value and the predicted driving pressure correction value.

7. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Obtain a second training sample, which includes at least two target prediction trajectories and a second label, wherein the second label is used to indicate the target driving pressure corresponding to the target prediction trajectory; The second training sample is input into the second general LSTM model to obtain the prediction driving pressure for each target prediction trajectory; The network parameters of the second general LSTM model are adjusted based on the sample loss of the second training sample to obtain the trained inverse kinematics network model. The sample loss of the second training sample is used to indicate the loss between the predicted driving pressure and the target driving pressure.

8. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Obtain a third training sample, which includes at least two predicted driving pressures and a third label, wherein the third label is used to indicate the actual predicted motion trajectory corresponding to the predicted driving pressure; The third training sample is input into the third general LSTM model to obtain the predicted motion trajectory for each predicted driving pressure. The network parameters of the third general LSTM model are adjusted based on the sample loss of the third training sample to obtain the trained forward kinematics network model. The sample loss of the third training sample is used to indicate the loss between the predicted motion trajectory and the actual predicted motion trajectory.

9. The method according to any one of claims 1 to 8, characterized in that, The driving pressure correction model, the inverse kinematics network model, and the forward kinematics network model are LSTM models with the same network architecture but different network parameters.

10. The method according to claim 7, characterized in that, The method further includes: The actual motion trajectory is determined based on the motion capture system on the soft robotic arm; or, The actual motion trajectory is determined based on the camera vision sensor on the soft robotic arm.

11. The method according to claim 10, characterized in that, The soft robotic arm includes multiple soft joints, each soft joint comprising a first soft actuator, a second soft actuator, and a third soft actuator arranged in parallel and evenly. The actual motion trajectory includes several sequentially ordered motion modes, each of which is one of n basic motion modes. The method further includes: Randomly drive one of the soft joints of the soft robotic arm to perform one of the n basic motion modes, which is used as the current motion mode among the plurality of motion modes; The motion trajectory of the soft robotic arm in the current motion mode is acquired based on the motion capture system on the soft robotic arm. After a preset time, the step of randomly driving one joint of the soft robotic arm to perform one of the n basic motion modes is repeated, serving as the current motion mode among the plurality of motion modes.

12. The method according to claim 10 or 11, characterized in that, The n basic action modes include at least one of the following: a first basic action mode, a second basic action mode, a third basic action mode, and a fourth basic action mode: The driving mode of the first basic action mode includes the first software driver being in an increasing pressure state, and the second software driver and the third software driver being in a decreasing pressure state; The driving mode of the second basic action mode includes the first software driver being in a pressure reduction state, and the second software driver and the third software driver being in a pressure increase state; The driving mode of the third basic action mode includes the first software driver being in a constant pressure state, the second software driver being in a reduced pressure state, and the third software driver being in a increased pressure state. The driving mode of the fourth basic action mode includes the first software driver being in a constant pressure state, the second software driver being in a pressure-increasing state, and the third software driver being in a pressure-decreasing state.

13. The method according to claim 8, characterized in that, The method further includes: The actual driving pressure is determined based on the pressure sensor on the soft robotic arm.

14. A control device for a soft robotic arm, characterized in that, The device includes: The acquisition module is used to acquire the target motion trajectory of the soft robotic arm, and the target motion trajectory is used to indicate the target pose data sequence of the soft robotic arm; The generation module is used to generate a predicted driving pressure based on the target predicted trajectory through an inverse kinematics network model. The predicted driving pressure is a driving pressure value required to predict the movement of the soft robotic arm. The generation module is used to generate a predicted motion trajectory based on the predicted driving pressure using a forward kinematics network model. The predicted motion trajectory is used to indicate the predicted pose data sequence of the soft robotic arm. The generation module is used to obtain a trajectory error based on the target motion trajectory and the predicted motion trajectory, wherein the trajectory error is used to indicate the pose data sequence error between the target pose data sequence and the predicted pose data sequence; The control module is used to generate a drive pressure correction value based on the trajectory error through a drive pressure correction model, and to control the movement of the soft robotic arm according to the drive pressure correction value and the predicted drive pressure.

15. A computer device, characterized in that, The computer device includes a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the control method of the soft robotic arm as described in any one of claims 1 to 13.

16. A computer storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to implement the control method of the soft robotic arm as described in any one of claims 1 to 13.

17. A computer program product, characterized in that, The computer program product includes a computer program stored in a computer-readable storage medium; the computer program is read from and executed by a processor of a computer device, causing the computer device to perform the control method of the soft robotic arm as described in any one of claims 1 to 13.

Images