Biologically inspired robotic fish swimming action learning method and device

Abstract

The invention provides a biologically-inspired robotic fish swimming action learning method and device, and the method comprises the steps: carrying out the extraction of fish key points based on collected biological fish swimming data, and obtaining a biologically-inspired experience database; determining a fish swimming action sequence by using a dynamic trajectory method according to the obtained positions of the joints of the robotic fish; pre-training a pre-constructed agent by using a biological heuristic experience database, and carrying out offline learning based on biological data to obtain a pre-trained agent; training a pre-trained agent based on the fish swimming action sequence, and performing online learning to obtain a robotic fish swimming action learning agent; wherein different states and action vectors are adopted in the online learning stage and the offline learning stage; and controlling the robotic fish to swim based on the robotic fish swim action learning agent. The robot fish swimming is controlled based on the robot fish swimming action learning agent, and the robot fish swimming action can be accurately controlled and optimized.

Description

Technical Field

[0001] The present invention relates to the technical field of underwater robots, and in particular to a method and device for learning the swimming motion of a bio-inspired robotic fish. Background Art

[0002] In nature, fish achieve various swimming behaviors by coordinating the movements of multiple fins with their bodies. Body-Caudal Fin (BCF) propulsion is the primary propulsion mode employed by most fish. This mode generates thrust through the coordinated wave motion of the body and caudal fin, enabling rapid launches while maintaining efficient cruising performance. Due to its significant advantages in maneuverability and propulsion efficiency, the BCF propulsion mode holds significant value as a biomimetic prototype for application research in underwater robotics.

[0003] With the deepening of research on underwater bionic robots, researchers are committed to improving the motion performance of underwater robots in BCF mode by combining bionic technology and control methods. Since the birth of the world's first bionic robotic fish, scientists have been constantly trying to establish a mathematical model of fish swimming motion and apply it to the motion control of underwater robotic fish. The fish body traveling wave equation theory proposed by Lighthill and its improved method provide a theoretical basis for the motion control of robotic fish. Multiple trajectory approximation methods have been proposed to approximate the traveling wave equation to obtain better robotic fish swimming effects. With the widespread application of the Central Pattern Generator (CPG) method in robots, CPG-based control methods have gradually been applied to the swimming control of robotic fish. The CPG method can generate rhythmic signals and is easy to calculate and adjust parameters.

[0004] Although these methods fit fish body waves based on or other functional forms, and often feature fixed periods and symmetrical motions, they do not fully address the flexibility and maneuverability of fish swimming. The swimming behavior of fish in nature reveals that fish can adjust their range and speed in real time according to their swimming state and maneuvering objectives, thereby exhibiting unique adaptive locomotion strategies in different behavioral scenarios. For example, during predation, fish employ large-amplitude, low-frequency body oscillations to generate an S-shaped start, achieving sudden high acceleration to capture prey. During sustained swimming, they switch to a small-amplitude, high-frequency oscillation pattern, which effectively reduces swimming resistance and maintains a stable high-speed cruising state. This ability of fish to dynamically adjust motion parameters based on behavioral needs enables them to optimize their locomotion performance in different scenarios. However, current research on robotic fish generally lacks attention to this adaptability, particularly regarding the impact of motion parameter adjustment on maneuverability, which remains underdeveloped. Summary of the Invention

[0005] The present invention provides a bio-inspired robotic fish swimming motion learning method and device to address the shortcomings of existing robotic fish swimming motions, such as low flexibility and maneuverability, and to improve the swimming performance of robotic fish in different states. The technical solutions proposed in the present invention are as follows: In a first aspect, the present invention provides a method for learning a bio-inspired robotic fish swimming motion, comprising: Based on the collected biological fish swimming data, fish key points are extracted to obtain a biologically inspired experience database; Obtain the position of each joint of the robotic fish, and use the dynamic trajectory method to determine the swimming action sequence of the fish based on the position of each joint of the robotic fish; Pre-training a pre-built intelligent agent using the biologically inspired experience database to perform offline learning based on biological data to obtain a pre-trained intelligent agent; Training the pre-trained agent based on the fish swimming motion sequence and performing online learning to obtain a robotic fish swimming motion learning agent; wherein different state vectors and action vectors are used in the online learning stage and the offline learning stage; Based on the swimming action of the robotic fish, an intelligent agent is learned to control the swimming of the robotic fish.

[0006] Optionally, the fish key points are extracted based on the collected biological fish swimming data to obtain a biologically inspired empirical database, including: Extracting key points of the fish from the collected biological fish swimming data using a pre-built skeleton extraction network to obtain coordinates of multiple key points of the fish; Extracting the fish swimming characteristics based on the coordinates of the multiple key points, and calculating the heading angle and joint angle at each moment; In each swing cycle, the swing amplitude of the heading angle and the maximum swing amplitude and instantaneous movement speed of each joint are determined according to the heading angle and the joint angles, thereby obtaining the bio-inspired experience database.

[0007] Optionally, the position of each joint of the robotic fish includes the current position of each joint and the position of the connecting rod at the previous moment; and determining the swimming motion sequence of the robotic fish by using a dynamic trajectory method according to the position of each joint of the robotic fish includes: Determine the angle of the joint at the current time step based on the current position of the joint and the joints before and after it; Get the instantaneous angular velocity of the joint, and determine the acceleration of the joint based on the joint and its front and rear joint connection positions and the instantaneous angular velocity of the joint; Determine the angle of the joint relative to the heading based on the angle of each joint; For the connecting rod composed of each joint, for the position point on the connecting rod, a relative coordinate system is established in the relative coordinate system of the connecting rod, and the velocity component of the position point on the connecting rod in the current connecting rod coordinate system is determined according to the angle of the joint relative to the heading, the position point on the connecting rod, the connecting rod position of the joint, and the connecting rod position of the next joint of the joint; the angle, acceleration, angle relative to the heading of each joint, and the velocity component of the position point on the connecting rod constitute the fish swimming action sequence.

[0008] Optionally, the motion vector in the offline learning stage includes the swing amplitude of each joint in a flapping cycle and the average value of the joint angular velocity in a flapping cycle, and the state vector includes the speed at the current moment, the bow pitch amplitude in the current cycle and the acceleration at the next moment; The action vector in the online learning stage includes the joint limit angles of each joint and the swing speed of the active joint, and the state vector includes the current robot fish speed, the bow swing amplitude in the current cycle, and the difference between the current robot fish speed and the target speed.

[0009] Optionally, the reward function for the offline learning phase is set to: in, represents the reward function in the offline learning phase, and are the coefficients for evaluating acceleration and pitch amplitude, represents the acceleration at the next moment, Indicates the heading amplitude in the current cycle. Indicates the allowable range of yaw.

[0010] Optionally, the reward function for the online learning phase is set to: in, represents the reward function in the online learning phase, and are the coefficients for evaluating speed and yaw amplitude, represents the target speed, Indicates the current speed of the robot fish. Indicates the heading amplitude in the current cycle. Indicates the allowable range of yaw.

[0011] In a second aspect, the present invention further provides a bio-inspired robotic fish swimming motion learning device, comprising the following modules: A key point extraction module is used to extract key points of fish based on the collected biological fish swimming data to obtain a biologically inspired experience database; The trajectory extraction module is used to obtain the position of each joint of the robotic fish and determine the swimming action sequence of the fish using the dynamic trajectory method based on the position of each joint of the robotic fish; An offline learning module, configured to pre-train a pre-built intelligent agent using the biologically inspired experience database, and perform offline learning based on biological data to obtain a pre-trained intelligent agent; An online learning module is used to train the pre-trained agent based on the fish swimming motion sequence, perform online learning, and obtain a robotic fish swimming motion learning agent; wherein different state vectors and action vectors are used in the online learning stage and the offline learning stage; A swimming control module is used to control the swimming of the robotic fish based on the swimming action learning agent of the robotic fish.

[0012] In a third aspect, the present invention further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the computer program, the bio-inspired robotic fish swimming motion learning method as described in the first aspect above is implemented.

[0013] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the bio-inspired robotic fish swimming motion learning method as described in the first aspect above.

[0014] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the bio-inspired robotic fish swimming motion learning method as described in the first aspect above.

[0015] Based on the above technical solution, the present invention has the following beneficial effects compared with the prior art: The bio-inspired robotic fish swimming motion learning method and device provided by the present invention captures the various motion patterns and posture changes of real fish swimming by collecting swimming data from biological fish and extracting key points. This biological data provides a rich reference for the robotic fish's swimming motions, helping to enhance the diversity and authenticity of the robotic fish's motions. Based on the positions of the robotic fish's joints, a dynamic trajectory method can be used to generate a series of motion sequences that emulate the characteristics of fish swimming. This method can simulate the continuity and smoothness of fish swimming, making the robotic fish's swimming motion more natural. The dynamic trajectory method also allows for real-time adjustments based on the robotic fish's actual motion state and environmental changes, thereby improving the flexibility and adaptability of the robotic fish's swimming. During the offline learning phase, the intelligent agent is pre-trained using a bio-inspired empirical database, enabling the agent to initially grasp the basic laws and characteristics of fish swimming. This lays a solid foundation for subsequent online learning. During the online learning phase, the pre-trained intelligent agent is further trained based on the fish's swimming motion sequences, enabling real-time adjustments and optimization based on the robotic fish's actual swimming state and environmental changes. This learning method helps improve the robotic fish's swimming maneuverability and responsiveness. The online and offline learning phases use different state and action vectors, enabling the agent to conduct targeted learning and optimization based on the needs of each phase, further improving the flexibility and accuracy of the swimming motion. By learning the agent's control over the robotic fish's swimming motion based on its swimming behavior, the agent can achieve precise control and optimization of the robotic fish's swimming motion. The agent can make real-time adjustments based on environmental changes and the robotic fish's actual state, making the robotic fish's swimming more flexible, natural, and efficient. Through continuous learning and optimization, the agent can also gradually master more complex swimming motions and strategies, further improving the robotic fish's swimming performance and maneuverability.

[0016] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or understood by practicing the present invention. The purposes and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0017] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 Schematic diagram of the flow of the biologically inspired robotic fish swimming motion learning method of the present invention.

[0020] Figure 2 Schematic diagram of the biological fish feature extraction method of the present invention; wherein, (a) is a schematic diagram of the method for collecting biological fish behavior data, and (b) is a diagram of the key point division situation.

[0021] Figure 3 Schematic diagram of the dynamic trajectory method of the robotic fish in the present invention; wherein, (a), (b), (c), (d), and (e) are schematic diagrams of changes in the joints of the robotic fish at different times, and (f) is a schematic diagram of changes in the joints at a specific time.

[0022] Figure 4 It is a schematic diagram of the two-stage learning framework in the present invention.

[0023] Figure 5 This is a timing diagram of the movement of the robot fish after learning is completed.

[0024] Figure 6 This is the action output diagram after the robot fish in the present invention completes learning.

[0025] Figure 7 It is a structural schematic diagram of the biologically inspired robotic fish swimming motion learning device of the present invention.

[0026] Figure 8 It is a structural schematic diagram of the electronic device in the present invention. DETAILED DESCRIPTION

[0027] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0028] The following combination Figure 1-Figure 7 The present invention describes a method and device for learning the swimming motion of a bio-inspired robotic fish.

[0029] This method adopts a two-stage swimming motion optimization strategy based on biological inspiration, which includes two stages: offline learning and online learning. This method aims to introduce valuable biological experience into the motion control of robotic fish by deeply exploring and imitating the swimming behavior of real fish, thereby further improving its motion performance. Figure 1 As shown, including the following: Step S110 : extracting fish key points based on the collected biological fish swimming data to obtain a biologically inspired experience database.

[0030] In the offline learning stage, the present invention first uses the swimming behavior of real fish as a learning paradigm. Figure 2 As shown in (a) of the figure, high-definition video equipment (such as cameras) or other underwater observation devices fixed to support frames on both sides of the water tank collect a wide range of swimming videos and images of real fish. This data not only covers the overall swimming trajectory of the fish, but also deeply explores the subtle movements of each key moving part.

[0031] To extract valuable features from these biological fish movement data, computer vision technology, specifically key point extraction algorithms, was applied. This technology can extract key motion feature points from the collected data, accurately capturing the fish's movement characteristics and state features, such as the movement trajectory and speed of the head, tail, and fins. This feature data was further organized and optimized to construct a bio-inspired empirical database.

[0032] Step S120: Obtain the position of each joint of the robotic fish, and determine the swimming motion sequence of the fish using a dynamic trajectory method based on the position of each joint of the robotic fish.

[0033] Based on the actual structure of the robotic fish, the positional information of its joints (such as the head, body segments, and tail) is obtained. Then, using the dynamic trajectory method, the fish's swimming motion sequence is simulated based on the extracted key point data and mapped to the robotic fish's joints. By determining the fish's swimming motion sequence, a target trajectory for mimicking fish swimming is provided for the robotic fish, guiding its learning of swimming movements.

[0034] Step S130: pre-training a pre-built intelligent agent using the biologically inspired experience database, performing offline learning based on biological data, and obtaining a pre-trained intelligent agent.

[0035] Based on the empirical database obtained in step S110, the present invention designs and pre-trains an intelligent agent (or motion model). This intelligent agent uses deep learning and other technologies to deeply learn and fit the data in the database, thereby initially grasping the basic laws and characteristics of fish swimming movements.

[0036] Specifically, an intelligent agent (such as a deep neural network) is constructed and pre-trained using the extracted biologically inspired experience database. This pre-training phase utilizes an offline learning approach, meaning the agent does not directly interact with the environment. Instead, it learns from historical fish swimming data to initially grasp the basic patterns of swimming movements. This pre-training gives the agent a preliminary understanding of fish swimming movements, laying the foundation for subsequent online learning.

[0037] Step S140: training the pre-trained agent based on the fish swimming motion sequence, performing online learning, and obtaining a robotic fish swimming motion learning agent; wherein, different state vectors and action vectors are used in the online learning stage and the offline learning stage.

[0038] Based on pre-training, the agent undergoes further training using the determined fish swimming motion sequence as a training target. This phase employs an online learning approach, where the agent interacts with the environment and adjusts its motion based on real-time feedback to approximate the target trajectory. Different state vectors and action vectors are used in the online and offline learning phases to accommodate real-time interaction. This online learning allows the agent to flexibly adjust its swimming motion based on the actual environment, achieving accurate imitation of the fish's swimming motion.

[0039] Specifically, after entering the online learning phase, the pre-trained agent is transferred to the physical robotic fish platform. In a real underwater environment, the robotic fish swims according to the control instructions of the pre-trained agent, and uses sensors to collect real-time information about the environment and its own status. This information is fed back to the agent for online adjustment and optimization of its swimming movements. During this phase, the robotic fish not only imitates the basic swimming movements of fish but also makes adaptive adjustments based on real-time feedback. Through continuous trial and error and learning, the robotic fish gradually learns how to maintain a stable swimming posture under different environmental conditions, achieving more efficient and flexible swimming.

[0040] Step S150: Based on the swimming action of the robotic fish, the learning agent controls the swimming of the robotic fish.

[0041] After pre-training and online learning, the intelligent agent has mastered the basic principles of fish swimming and the ability to adjust in real time. At this point, the intelligent agent can be deployed on a robotic fish to control its swimming movements. Based on real-time environmental information and the robotic fish's status, the intelligent agent outputs corresponding control instructions to drive the robotic fish's joint movements. Under the control of the intelligent agent, the robotic fish can swim as flexibly as a real fish, with higher swimming efficiency and maneuverability.

[0042] The bio-inspired robotic fish swimming motion learning method provided by the present invention captures the various motion patterns and posture changes of real fish swimming by collecting swimming data from biological fish and extracting key points. This biological data provides a rich reference for the robotic fish's swimming motions, helping to enhance the diversity and authenticity of the robotic fish's motions. Based on the positions of the robotic fish's joints, a dynamic trajectory method can be used to generate a series of motion sequences that emulate the characteristics of fish swimming. This method can simulate the continuity and smoothness of fish swimming, making the robotic fish's swimming motion more natural. The dynamic trajectory method also allows for real-time adjustments based on the robotic fish's actual motion state and environmental changes, thereby improving the flexibility and adaptability of the robotic fish's swimming. In the offline learning phase, the intelligent agent is pre-trained using a bio-inspired empirical database, enabling the agent to initially grasp the basic laws and characteristics of fish swimming. This lays a solid foundation for subsequent online learning. In the online learning phase, the pre-trained intelligent agent is further trained based on the fish swimming motion sequences, enabling real-time adjustments and optimization based on the robotic fish's actual swimming state and environmental changes. This learning method helps improve the robotic fish's swimming maneuverability and responsiveness. The online and offline learning phases use different state and action vectors, enabling the agent to conduct targeted learning and optimization based on the needs of each phase, further improving the flexibility and accuracy of the swimming motion. By learning the agent's control over the robotic fish's swimming motion based on its swimming behavior, the agent can achieve precise control and optimization of the robotic fish's swimming motion. The agent can make real-time adjustments based on environmental changes and the robotic fish's actual state, making the robotic fish's swimming more flexible, natural, and efficient. Through continuous learning and optimization, the agent can also gradually master more complex swimming motions and strategies, further improving the robotic fish's swimming performance and maneuverability.

[0043] Furthermore, by constructing an intelligent agent and conducting pre-training and online learning, the present invention equips the robotic fish with the ability to autonomously learn and adapt to its environment. This enables the robotic fish to flexibly adjust its swimming movements and strategies based on real-time conditions to cope with various unknown and complex underwater environments. By mimicking the swimming movements of biological fish, the present invention enables the robotic fish to swim with more ideal fluid dynamics, significantly improving its swimming efficiency. This helps the robotic fish conserve energy and extend its endurance when performing long missions.

[0044] In an optional embodiment, before constructing a bio-inspired empirical database, it is first necessary to collect the swimming data of biological fish and accurately record the fish posture. In order to ensure the authenticity and naturalness of the data, it is usually filmed in an undisturbed pool environment. However, traditional methods such as pasting marking points will interfere with the swimming movements of fish, affect the accuracy of the data and interfere with the decision-making of fish. Therefore, the present invention adopts a deep learning method to extract the key points of the fish body. The above-mentioned step S110 extracts the key points of fish based on the collected biological fish swimming data to obtain a bio-inspired empirical database, including: S1101. Utilize a pre-built skeleton extraction network to extract key points of the fish from the collected biological fish swimming data to obtain coordinates of multiple key points of the fish.

[0045] In order to accurately capture the body movements of fish, refer to Figure 2 As shown in (b), the fish is divided into 6 key points from head to tail. These key points represent the front end (head), the end (tail) and the 4 main movement joints (first joint , second joint , third joint , fourth joint ). This key point division method helps to fully capture the swimming characteristics of fish.

[0046] Next, refer to Figure 2 As shown in (a), the six required fish key points are trained using DeepLabCut, an open-source animal skeleton extraction solution. DeepLabCut is a deep learning-based skeleton extraction tool that accurately extracts the coordinates of key points on the animal body. In this paper, the skeleton extraction network uses a convolutional and deconvolutional network based on ResNet50, which has strong feature extraction capabilities.

[0047] Using a labeled dataset consisting of approximately 200 fish photos with manually annotated key points, the network achieved usable performance after approximately 12,000 training rounds. When deployed in the same pool environment as the annotated samples, the network was able to effectively extract the coordinates of the fish's six key points with a high degree of confidence. This approach not only avoids the interference of traditional labeling methods on the fish's swimming movements, but also improves the accuracy and efficiency of key point extraction.

[0048] S1102: Extract the fish swimming characteristics based on the coordinates of the multiple key points, and calculate the heading angle and joint angle at each moment.

[0049] After extracting the coordinates of the fish's key points, the data needs to be further processed and analyzed to extract the characteristics of the fish's swimming. These characteristics include heading angle, joint angle, swing amplitude, maximum swing amplitude, and instantaneous movement speed.

[0050] The heading angle is an important parameter for the fish's swimming direction and can be obtained by calculating the angle between the head keypoint and the first joint coordinate. The joint angle reflects the relative position of the various parts of the fish's body and can be obtained by calculating the angle between adjacent keypoints.

[0051] In the present invention, the heading angle and joint angle at each moment are calculated using the key point coordinates. For example, the heading angle can be calculated using formula (1): (1) Where, is the heading angle, which indicates the direction in which the fish swims. are the horizontal and vertical coordinates of the key points of the head, are the horizontal and vertical coordinates of the first joint. is the inverse tangent function.

[0052] Similarly, the joint angle can also be calculated using a similar method, taking the first joint as an example: (2) Where, is the angle of the first joint, are the horizontal and vertical coordinates of the key points of the head, are the horizontal and vertical coordinates of the first joint, are the horizontal and vertical coordinates of the second joint (the joint after the first joint).

[0053] Second joint Angle , third joint Angle , fourth joint Angle The calculation method of is similar and will not be repeated here.

[0054] S1103. In each swing cycle, determine the swing amplitude of the heading angle and the maximum swing amplitude and instantaneous motion speed of each joint according to the heading angle and the joint angles, and obtain the bio-inspired experience database.

[0055] Based on the fish's swimming rhythm and caudal fin oscillation frequency, the video is divided into several oscillation cycles. Each oscillation cycle lasts from the moment the caudal fin begins to oscillate until it resumes. When analyzing the collected raw data, we focus on the changes in the heading angle and joint angle sequences. Within each complete oscillation cycle, we calculate the heading angle amplitude, the maximum oscillation amplitude of each joint, and the instantaneous motion velocity. These time series feature data reflect the dynamic characteristics of the fish's swimming and serve as important foundational data for subsequent offline model training.

[0056] Specifically, the heading angles at the beginning and end of each swing cycle were recorded, and the swing amplitude of the heading angle (i.e., the difference between the maximum and minimum heading angles) was calculated. The maximum and minimum angles of each joint within each swing cycle were recorded to obtain the maximum swing amplitude of each joint. The instantaneous speed of the fish within each swing cycle was calculated by analyzing the changes in key point coordinates between consecutive frames.

[0057] After extracting characteristic data about fish swimming, this data needs to be standardized. Standardization eliminates dimensional differences between different features, making the data more suitable for subsequent machine learning algorithm processing. In this invention, characteristic data such as heading angle, joint angle, swing amplitude, maximum swing amplitude, and instantaneous motion speed are standardized. Data such as the heading angle swing amplitude, maximum swing amplitude of each joint, and instantaneous motion speed within each swing cycle are organized into a table or database format. The empirical database should contain additional information such as fish species, swimming environment (such as water temperature and water quality), and swing cycle number to facilitate subsequent analysis and application. This bio-inspired empirical database provides parameter references for the design of robotic fish. By analyzing changes in heading angle and joint angles during fish swimming, the swimming efficiency and maneuverability of the robotic fish can be optimized. This empirical database contains rich information about fish swimming characteristics and can be used to train an intelligent agent to learn the patterns and strategies of fish swimming. The data in the database serves as training samples for training an intelligent agent (such as a reinforcement learning model) to control the swimming of the robotic fish. By simulating the heading adjustment and joint swing of fish when swimming, the robotic fish can swim autonomously in complex environments.

[0058] This invention uses a deep learning approach to extract key points from fish bodies, avoiding the interference of traditional marker methods on fish swimming movements and improving the accuracy and efficiency of key point extraction. By dividing six key points and calculating characteristic parameters such as heading angle and joint angle, the robot captures comprehensive information about fish swimming movements, providing rich data support for subsequent intelligent agent training. By training the intelligent agent based on a biologically inspired empirical database, the robot fish can learn the patterns and strategies of fish swimming, thereby enhancing its swimming flexibility and maneuverability.

[0059] In an optional embodiment, observations of biological fish swimming revealed that their swimming motion is not a completely symmetrical, periodic oscillation, but rather involves dynamically adjusting the coordination of the tail joints based on scenario requirements. As the tail joint swings, the motion trajectory of the rear joint tends to follow and overlap the swing path of the front joint, minimizing disturbance to the course while achieving thrust. Based on this, the present invention employs a dynamic trajectory method as a swimming control method to mimic biological motion. Compared to the rhythmic control associated with a central pattern generator (CPG), each joint of the robotic fish can adjust its position in real time based on the course state, eliminating the rhythmic relationships between individual joints. Therefore, by modifying coordination relationships such as joint limit angles, the dynamic trajectory method can generate a larger swing space and coordination relationships, providing a foundation for robotic fish motion learning. The present invention simulates the posture changes and dynamic characteristics of the robotic fish during swimming by calculating parameters such as joint angles, acceleration, and the velocity component of the position on the connecting rod. The position of each joint of the robotic fish includes the current position of each joint and the position of the connecting rod at the previous moment; the above-mentioned step S120 of determining the fish swimming action sequence by using the dynamic trajectory method according to the position of each joint of the robotic fish includes: S1201. Determine the angle of the joint at the current time step based on the current position of the joint and its front and rear joints.

[0060] Obtain the position information of each joint at the current time step, including the joint coordinates and the direction vector relative to a reference point. For each joint, calculate the angle between the vectors of the joint and the joints before and after it, i.e., the joint angle. This can be achieved using the dot product or cross product formula of the vectors. Store the calculated joint angles in a data structure for subsequent use.

[0061] In this embodiment, Figure 3 The motion sequence implemented by the dynamic trajectory method is shown, that is, the left swing sequence of the robotic fish. The black link represents the current joint position, and the blue dotted line represents the link position at the previous moment. The joint angle is determined by the angle between adjacent black links. Specifically, Figure 3 In (a), (b), and (c), the first joint acts as the active joint to perform the left swing, while the other three joints perform passive dynamic trajectory adjustment. Once the first joint reaches its limit, the second joint takes over the active swing task, and the other joints perform passive updates, such as Figure 3 As shown in (d) and (e). Figure 3 (f) in the figure shows the changes of joints at a specific moment as an example of the derivation of related calculations. According to the theory of dynamic trajectory method, the joints at the current time step Angle It can be expressed as: (3) in, 、 、 Represent three consecutive joints ( 、 、 )'s current location. Represents the inverse cosine function. Represents the magnitude of a vector.

[0062] S1202: Obtain the instantaneous angular velocity of the joint, and determine the acceleration of the joint based on the connecting rod positions of the joint and its front and rear joints at the previous moment and the instantaneous angular velocity of the joint.

[0063] Obtain the instantaneous angular velocity of each joint at the current time step. This can be obtained by time-differentiating the joint position or using sensor measurements. For each joint, calculate the change between its link position at the previous moment and its current moment, and the relationship between this change and the instantaneous angular velocity. This can be achieved through vector differentiation. Use physical laws (such as Newton's second law) to calculate the joint acceleration. Store the calculated joint acceleration in a data structure for subsequent use.

[0064] In this embodiment, the joint Acceleration It can be expressed as: (4) (5) in, Indicates joints The acceleration of Indicates joints The modulus of the acceleration, express The derivative of is the angular velocity transformation matrix of the rotation vector. Indicates joints The derivative of the angular velocity transformation matrix corresponding to the rotation vector. represents the instantaneous angular velocity of the joint, Indicates joints Relative to joints relative rotation. Indicates joints to the joints The connecting rod length, Indicates the angle of the joint. and Represents joints and The instantaneous speed.

[0065] S1203: Determine the angle of the joint relative to the heading according to the angle of each joint.

[0066] In order to understand the changes in the angles of each joint relative to the heading during the swimming process of the robotic fish, this application determines its angle relative to the heading based on the angles of each joint. The heading angle is defined as the angle between the head of the robotic fish or a reference point and the swimming direction. This can be achieved by calculating the vector angle between the head or reference point and a fixed point (such as a target point or starting point). For each joint, calculate its angle relative to the heading. This can be achieved by superimposing or converting the joint angle with the heading angle, depending on the required calculation accuracy and efficiency. The calculated joint angles relative to the heading are stored in a data structure for subsequent use.

[0067] In this embodiment, it is assumed that the joint The angle relative to the heading is , then . Represents any joint angle.

[0068] S1204. For the connecting rod composed of each joint and the position point on the connecting rod, a relative coordinate system is established in the relative coordinate system of the connecting rod. According to the angle of the joint relative to the heading, the position point on the connecting rod, the connecting rod position of the joint, and the connecting rod position of the next joint of the joint, the velocity component at the position point on the connecting rod in the current connecting rod coordinate system is determined to obtain the fish swimming action sequence.

[0069] For each joint in the linkage, a relative coordinate system is established within the linkage's relative coordinate system. The velocity component at the point on the linkage in the current linkage coordinate system is determined based on the joint's angle relative to the heading, the location on the linkage, the linkage position of the joint, and the linkage position of the next joint. This helps understand the velocity distribution and dynamic characteristics of each point on the linkage during the robot fish's swimming. The specific steps are as follows: For each link, establish its relative coordinate system. This can be achieved by selecting a reference point on the link as the origin and defining an x-axis parallel to the link and a y-axis perpendicular to the link. For each point on the link, calculate its velocity component in the current link coordinate system. This can be achieved by projecting the velocity vector of the point onto the x- and y-axes. Store the calculated velocity components of the point on the link in a data structure for subsequent use.

[0070] In this embodiment, for the connecting rod Position point on , establish a relative coordinate system in the relative coordinate system of the connecting rod, such as Figure 3As shown in (f) in the figure. The position point in the current link coordinate system The velocity component at can be expressed as: (6) Where, and Respectively represent the position points in the current link coordinate system The projection of the velocity component at on the x-axis and y-axis. Indicated by joints Angle relative to heading Determines the rotation matrix used to transform the velocity vector from the global coordinate system to the current link coordinate system. and Represents joints and The instantaneous speed.

[0071] Through the above steps, the dynamic trajectory method can be used to determine the motion sequence of the robotic fish during swimming. This includes parameters such as the angles, accelerations, angles relative to the heading, and velocity components of the points on the connecting rods, which constitute the fish's swimming motion sequence. These parameters can be used to guide the design and control of the robotic fish, improving its swimming efficiency and maneuverability.

[0072] The present invention uses a dynamic trajectory method to accurately simulate parameters such as joint angles, acceleration, and velocity components during the swimming of a robotic fish, thereby improving simulation accuracy. This helps to better understand the biomechanical mechanisms of fish swimming and provides strong support for the design and control of robotic fish. The fish swimming motion sequence determined using the dynamic trajectory method can guide the robotic fish to flexibly adjust its posture during swimming. By simulating the active swing and passive update processes of different joints, the robotic fish can achieve autonomous swimming and obstacle avoidance functions in complex environments.

[0073] In an optional embodiment, during the offline learning phase, the present invention utilizes fish swimming motion sequences for completely offline empirical learning. The goal of this phase is to extract key motion features from a large amount of fish swimming data and train an intelligent agent that can predict the motion state of the robotic fish based on these features.

[0074] The motion vector in the offline learning stage includes the swing amplitude of each joint in a flapping cycle and the average value of the joint angular velocity in a flapping cycle, and the state vector includes the speed at the current moment, the bow swing amplitude in the current cycle and the acceleration at the next moment.

[0075] Through step S110, over 7,000 straight swimming motion-state sequences meeting the training requirements were extracted from the swimming videos. The original 10ms sampling period data was downsampled and reconstructed into a 100ms sampling period time series. This step not only matches the robotic fish's control period but also eliminates most of the duplicate data within the same swing cycle, improving data processing efficiency.

[0076] The definition of action vector and state vector is as follows: Motion Vector ,in represents the swing amplitude of the four joints in one flapping cycle, represents the average angular velocity of the joints during a flapping cycle. These parameters reflect the motion characteristics of each joint when the fish swims.

[0077] State vector ,in, Indicates the speed at the current moment, Indicates the heading amplitude in the current cycle. Represents the acceleration at the next moment. The accelerated action of the fish causes the acceleration to increase at the next moment. Since the entire action sequence is known, As state variables, they can help the agent make effective decisions about current actions. These parameters reflect the current movement state and future movement trends of the fish.

[0078] In the offline phase, key information is extracted from a biologically inspired empirical database to construct the state vector and action vector. The following are the specific steps for obtaining the offline state vector and action vector using the heading angle swing amplitude, the maximum swing amplitude of each joint, and the instantaneous motion speed in the database: The action vector construction process is as follows: Extracting joint swing amplitudes: For each straight swimming sequence, extract the swing amplitudes of four key joints (assuming they are the head, middle of the body, front of the tail, and back of the tail; the specific joints can be adjusted according to actual conditions) within one flapping cycle from the empirical database. These swing amplitudes are expressed as Calculate the average angular velocity of the joints: For each joint, calculate its angular velocity in one flapping cycle and find the average value, which is expressed as This can be obtained by integrating and averaging the rate of change of the joint angle over time. Constructing the action vector: Combine the joint swing amplitude and the average value of the joint angular velocity extracted above to form the action vector The state vector construction process is as follows: Extract the current speed: From the experience database, for each action sequence at the current moment, extract the instantaneous movement speed of the fish, expressed as Calculate the yaw amplitude in the current cycle: The yaw amplitude (i.e., the range of the heading angle) can be obtained by calculating the difference between the maximum value θmax (i.e., the maximum swing amplitude obtained in step S1103) and the minimum value θmin of the heading angle in the current cycle. . Predict the acceleration at the next moment: the acceleration at the next moment Use the current velocity change rate to approximate (i.e. ≈ΔtΔv, where Δv is the change in speed and Δt is the time interval). Construct the state vector: Combine the current speed and heading amplitude extracted above and the predicted acceleration at the next moment to form the state vector Through the above steps, key information can be extracted from the biologically inspired experience database and the state vector and action vector for offline learning can be constructed. These vectors will serve as input to the machine learning algorithm to train the model to simulate the swimming behavior of fish.

[0079] In an optional embodiment, the action vector in the online learning stage includes the joint limit angles of each joint and the swing speed of the active joint, and the state vector includes the current robot fish speed, the bow swing amplitude in the current cycle, and the difference between the current robot fish speed and the target speed.

[0080] The online learning phase is based on the offline learning phase. In this phase, the present invention uses the pre-trained agent in the offline learning phase as the initial model and performs online adjustments and optimizations based on the actual motion state of the robotic fish. Specifically, the action vector and state vector are updated: The action vector is updated as ,in Indicates the joint limit angle of each joint, Indicates the swing speed of the active joint. These parameters reflect the motion characteristics of the robotic fish during actual motion.

[0081] The state vector is updated to , Indicates the current speed of the robot fish, Indicates the heading amplitude in the current cycle. Indicates the current speed of the robot fish With target speed These parameters reflect the current motion state of the robotic fish and the gap with the target state.

[0082] The model trained during the offline learning phase is used as the initial model, and online adjustments and optimizations are performed based on the robot fish's actual motion state. This step helps the robot fish quickly adapt to different environments and task requirements. The online learning phase still uses reinforcement learning for training, but now an exploration function is required to explore new motion strategies. Furthermore, the network's learning status needs to be monitored to prevent overfitting.

[0083] The present invention achieves an efficient and stable method for controlling the motion of a robotic fish by combining offline and online learning stages. Specifically, through the extraction of key motion features during the offline learning stage and the online adjustment and optimization during the online learning stage, the robotic fish can more accurately control its motion state and achieve smoother and faster swimming. The online learning stage enables the robotic fish to make adaptive adjustments based on environmental changes and task requirements, thereby improving its environmental adaptability. The extensive data preprocessing and reinforcement learning training during the offline learning stage enable the online learning stage to converge to the optimal solution more quickly, thereby reducing training costs.

[0084] In an optional embodiment, during the offline learning phase, the reward function for reinforcement learning training is set to: in, represents the reward function in the offline learning phase, and They represent the coefficients for evaluating acceleration and yaw amplitude, respectively, and are used to adjust the impact of acceleration and yaw amplitude on rewards. Indicates the acceleration at the next moment, reflecting the dynamic changes caused by the agent's actions. It represents the bow swing amplitude in the current cycle, that is, the swing degree of the agent in a certain period of time. Indicates the allowable range of yaw, which is a given value. The range is within the allowable range, and the reward The effect of is very small and can be ignored. When a large angle occurs, it causes The sharp drop causes the learning network to suppress this behavior, avoiding large angular fluctuations. The reason for rewarding the magnitude of the next-moment acceleration is that the sum of the accelerations in the sequence represents the rapidity of convergence to the target velocity. The acceleration at the next moment in the target state approaches 0, proving that the actions taken can quickly converge the velocity to the target value. The design of the reward function in this invention reflects the expectations for the agent's behavior: the desired acceleration is minimized (to reduce unnecessary power consumption or the intensity of the movements) while the desired bow roll amplitude is minimized (to maintain stability).

[0085] In the actual training process, the learning rate of the Actor and Critic networks is set to 0.0002, and the discount rate Set to 0.95. Since the state sequence is known and training is completely offline, no exploration function is set. Training is stopped when the network fully converges to prevent overfitting from affecting subsequent online transfer learning.

[0086] The offline learning process is as follows: Based on the collected action vectors and state vectors, a policy network or policy function is initialized. This policy will be used to select an action based on the current state. The initial policy can be random or based on some prior knowledge.

[0087] Using the collected action and state vectors and a defined reward function, the policy network is trained using an offline reinforcement learning algorithm (such as an offline variant of Q-learning or an offline version of Deep Deterministic Policy Gradient (DDPG)). During training, the algorithm continuously attempts to find a policy that maximizes the cumulative reward. Because this is done offline, real-time interaction with the environment is not required. The reward function R is calculated based on the agent's actions and the resulting state changes, guiding the direction of policy learning.

[0088] During or after training, the learned policy can be evaluated. This involves testing the policy's performance using a dataset that was not used in training. Based on the evaluation results, the policy can be adjusted or optimized. For example, the coefficients in the reward function can be adjusted. and You can change the emphasis on acceleration and pitch by adjusting the learned policy parameters. You can also try different offline reinforcement learning algorithms or adjust algorithm parameters to further improve the policy's performance. Once the policy has been fully trained and evaluated and achieves a satisfactory level of performance, it can be deployed to a real-world environment. During deployment, the agent will use the learned policy to select actions based on the current state, hoping to maximize the cumulative reward.

[0089] In the online learning stage, the variable parameter dynamic trajectory method deployed by the robot fish is used for online learning, such as Figure 4 The network structure and update method remain consistent with the offline learning phase, with minor adjustments to the state and motion vectors. During this phase, when using the variable-parameter dynamic trajectory method deployed by the robotic fish for online learning, the present invention adjusts the definitions of the state and motion vectors based on the actual motion state of the robotic fish and the parameters of the dynamic trajectory method. The following is the process of obtaining the online state and motion vectors from the fish's swimming motion sequence (or the actual motion data of the robotic fish): First, the action vector construction process is as follows: Dynamic trajectory method joint limit size: In the dynamic trajectory method, the joint limit angle refers to the maximum or minimum angle that the joint is allowed to reach during the movement. In the online learning stage, this parameter can be used as a fixed or adjustable input to the action vector, expressed as Swing speed of active joints: The swing speed of active joints refers to the change in angle of the actively controlled joints (such as the joints at the tail) per unit time during the swimming process of the robot fish. In the online learning stage, this speed is used as part of the action vector and is expressed as . Constructing action vector: The action vector in the online stage can be expressed as The state vector construction process is as follows: Current robot fish speed Through the above Indicates the bow shaking amplitude of the robot fish. It refers to the range of the heading angle of the robot fish during its swimming process. This parameter can be obtained by adjusting the heading angle of the robot fish. The target speed is the speed that the robot fish is expected to reach. During the online learning phase, the difference between the current speed and the target speed is used as part of the state vector to indicate the direction and magnitude of the speed that the robot fish needs to adjust. This difference can be expressed as In summary, the state vector in the online phase can be expressed as .

[0090] In an optional embodiment, the reward function in the online learning phase is set to: in, represents the reward function in the online learning phase, and are the coefficients for evaluating speed and yaw amplitude, represents the target speed, Indicates the current speed of the robot fish. Indicates the heading amplitude in the current cycle. Indicates the allowable range of yaw.

[0091] in, , representing the maximum deviation between the current and target headings within a cycle. Similar to the basic principle of the first stage, by reducing the sum of the speed differences from the target in the sequence and suppressing the magnitude of heading fluctuations, the robotic fish can adjust its movements in a timely manner based on the current speed and acceleration requirements. Represents the cycle, Indicates the current heading angle, Indicates the target heading angle.

[0092] In the above reward function, the speed evaluation part The reward reflects the difference between the robot fish's current speed and the target speed. As the robot fish approaches the target speed, the difference decreases, and the reward increases (because the absolute value of the negative difference decreases). Conversely, if the speed moves away from the target speed, the reward decreases. It is the coefficient for evaluating speed, which determines the impact of speed gap on the reward value.

[0093] Bow roll amplitude evaluation section The reward measures the ratio between the robot fish's bow pitch amplitude and its permitted range during the current cycle. When the pitch amplitude is small, the reward is high; when the pitch amplitude exceeds the permitted range, the reward is significantly reduced. It is the coefficient for evaluating the bow pitch amplitude, which determines the influence of the bow pitch amplitude on the reward value.

[0094] During the online learning phase, the variable parameter dynamic trajectory method deployed by the robotic fish is used for online learning. The online learning process is as follows: The robot fish first senses the current environmental status through sensors and other devices, including its own speed and the current robot fish speed. , the heading amplitude in the current cycle Etc. Based on the perceived state information, the robotic fish uses the above reward function to calculate the reward value in the current state.

[0095] Based on the calculated reward, the robotic fish adjusts its control strategy to maximize future cumulative reward. This involves optimizing control parameters such as propulsion force and steering angle. After adjusting its strategy, the robotic fish performs corresponding actions to change its state. These actions include acceleration, deceleration, and steering.

[0096] The above process will iterate continuously until the robot fish learns a strategy that effectively maximizes the cumulative reward. During the learning process, the robot fish will continuously trial and error, adjust its strategy, and gradually approach the optimal solution. After a certain number of iterations, if the robot fish's performance (such as cumulative reward) stabilizes or reaches a preset threshold, the learning process can be considered to have converged. At this point, the robot fish has learned a relatively effective control strategy.

[0097] In actual training, due to pre-training, the network learning rate for both the actor and critic was reduced to 0.0001, and the discount rate was maintained at 0.95. Because learning was conducted through online interaction, a high exploration rate was set to ensure effective transfer of fish movements to the robotic fish. The angular exploration range was set to 5°, and the angular velocity exploration rate was set to 10° / s.

[0098] Reference Figure 4As shown, the bio-inspired robotic fish swimming motion learning method provided by the present invention is a process that combines reinforcement learning and robotic control technology. The following is a detailed description of the method: This method builds a reinforcement learning framework and utilizes a target network consisting of an actor network and a critic network to simulate and optimize the swimming motions of a robotic fish. Guided by reinforcement learning, the robotic fish can learn how to swim more efficiently in a virtual reality environment to achieve pre-defined goals, such as increasing swimming speed, reducing energy consumption, or maintaining a stable swimming posture.

[0099] The actor network receives state information from the real environment (such as the robot fish's position, posture, and joint angles) and generates an action sequence, indicating how the robot fish should adjust its swimming motion. The output action sequence includes parameters such as swing speed, swing amplitude, joint limit angles, and swimming frequency. The real environment simulates the physical environment in which the robot fish swims, including water flow and obstacles. The robot fish model consists of the robot fish and the actuators (such as motors and servos) that drive the robot fish's swimming motion.

[0100] The policy network evaluates the effectiveness of the action sequence generated by the action network, calculating the reward value obtained by executing the action in the current state. It uses the Q-learning algorithm to estimate the value function Q(s,a) for the action, where s represents the state and a represents the action. It outputs a reward prediction to guide parameter updates in the action network.

[0101] During the offline learning phase, the action network and policy network are trained using a pre-collected biologically inspired empirical database. This dataset includes the robotic fish's swimming motions and corresponding rewards in different states. By iteratively updating the network parameters, the action network gradually learns to generate optimal action sequences in different states.

[0102] During the online learning phase, in a real-world environment, the robot fish uses the action network to generate an action sequence (S, a, r, S') based on its current state and executes these actions. The policy network calculates rewards based on the execution results and updates network parameters using a loss function to optimize future behavioral decisions. S and S' represent the state vectors, and r represents the reward.

[0103] Figure 5The results of the robotic fish's swimming after learning are shown. In actual swimming, the robotic fish, subject to its own performance limitations and water resistance, begins its movement from a stationary state until a certain point, when the power provided by the robot and the water resistance reach equilibrium, at which point the robot fish transitions from an accelerated state to a constant (i.e., stable) swimming state and maintains it. During the acceleration phase, the method described in this invention enables rapid start-up using large-amplitude, low-frequency movements. During the constant-speed phase, the method described in this invention achieves stable swimming by reducing the swing amplitude and increasing the swing frequency.

[0104] Figure 6 Shown in Figure 5 The output of motion variables during the acceleration process shows that the method of the present invention can adjust the motion output in a timely manner according to its own state, meet different swimming needs, and realize different motion strategies in the acceleration stage and the uniform speed stage (i.e., the speed stabilization stage). From the perspective of the limit angle (joint motion range), the robot fish tends to provide larger first and second joint limit angles at low speeds. After reaching the target speed, the second joint amplitude will be reduced and the third joint limit will be increased. From the perspective of joint angular velocity, in the early stage of the acceleration stage, a smaller angular velocity is used to increase the speed. After the acceleration is completed, a higher swing speed is used with a small swing amplitude to achieve high-frequency swimming.

[0105] The following describes the bio-inspired robotic fish swimming motion learning device provided by the present invention. The bio-inspired robotic fish swimming motion learning device described below and the bio-inspired robotic fish swimming motion learning method described above can refer to each other.

[0106] The bio-inspired robotic fish swimming motion learning device provided by the present invention refers to Figure 7 As shown, including: A key point extraction module 210 is used to extract key points of fish based on the collected biological fish swimming data to obtain a biologically inspired experience database; The trajectory extraction module 220 is used to obtain the position of each joint of the robotic fish and determine the swimming motion sequence of the fish using the dynamic trajectory method based on the position of each joint of the robotic fish; an offline learning module 230 for pre-training a pre-built intelligent agent using the biologically inspired experience database, performing offline learning based on biological data, and obtaining a pre-trained intelligent agent; An online learning module 240 is configured to train the pre-trained agent based on the fish swimming motion sequence, perform online learning, and obtain a robotic fish swimming motion learning agent; wherein different state vectors and action vectors are used in the online learning phase and the offline learning phase; The swimming control module 250 is used to control the swimming of the robotic fish based on the robotic fish swimming action learning agent.

[0107] Figure 8 An example of a physical structure diagram of an electronic device is shown below. Figure 8 As shown, the electronic device may include: a processor 310, a communications interface 320, a memory 330, and a communication bus 340. The processor 310, the communications interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 may call logic instructions in the memory 330 to execute the bio-inspired robotic fish swimming motion learning method.

[0108] Furthermore, the logic instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0109] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the bio-inspired robot fish swimming motion learning method provided by the above methods.

[0110] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the bio-inspired robotic fish swimming motion learning method provided by the above methods.

[0111] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0112] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A bio-inspired robotic fish swimming motion learning method, characterized by: include: Based on the collected biological fish swimming data, fish key points are extracted to obtain a biologically inspired experience database; Obtain the position of each joint of the robotic fish, and use the dynamic trajectory method to determine the swimming action sequence of the fish based on the position of each joint of the robotic fish; Pre-training a pre-built intelligent agent using the biologically inspired experience database to perform offline learning based on biological data to obtain a pre-trained intelligent agent; Training the pre-trained agent based on the fish swimming motion sequence and performing online learning to obtain a robotic fish swimming motion learning agent; wherein different state vectors and action vectors are used in the online learning stage and the offline learning stage; Based on the swimming action of the robotic fish, an intelligent agent is learned to control the swimming of the robotic fish.

2. The method for learning the bio-inspired robotic fish swimming motion according to claim 1, wherein: The fish key points are extracted based on the collected biological fish swimming data to obtain a biologically inspired experience database, including: Extracting key points of the fish from the collected biological fish swimming data using a pre-built skeleton extraction network to obtain coordinates of multiple key points of the fish; Extracting the fish swimming characteristics based on the coordinates of the multiple key points, and calculating the heading angle and joint angle at each moment; In each swing cycle, the swing amplitude of the heading angle and the maximum swing amplitude and instantaneous movement speed of each joint are determined according to the heading angle and the joint angles, thereby obtaining the bio-inspired experience database.

3. The method for learning the bio-inspired robotic fish swimming motion according to claim 1, wherein: The position of each joint of the robotic fish includes the current position of each joint and the position of the connecting rod at the previous moment; The method of determining the swimming motion sequence of the fish by using a dynamic trajectory method according to the positions of the joints of the robotic fish includes: Determine the angle of the joint at the current time step based on the current position of the joint and the joints before and after it; Get the instantaneous angular velocity of the joint, and determine the acceleration of the joint based on the joint and its front and rear joint connection positions and the instantaneous angular velocity of the joint; Determine the angle of the joint relative to the heading based on the angle of each joint; For the connecting rod composed of each joint, for the position point on the connecting rod, a relative coordinate system is established in the relative coordinate system of the connecting rod, and the velocity component of the position point on the connecting rod in the current connecting rod coordinate system is determined according to the angle of the joint relative to the heading, the position point on the connecting rod, the connecting rod position of the joint, and the connecting rod position of the next joint of the joint; the angle, acceleration, angle relative to the heading of each joint, and the velocity component of the position point on the connecting rod constitute the fish swimming action sequence.

4. The method for learning the bio-inspired robotic fish swimming motion according to claim 1, wherein: The motion vector in the offline learning stage includes the swing amplitude of each joint in a flapping cycle and the average value of the joint angular velocity in a flapping cycle, and the state vector includes the speed at the current moment, the bow pitch amplitude in the current cycle and the acceleration at the next moment; The action vector in the online learning stage includes the joint limit angles of each joint and the swing speed of the active joint, and the state vector includes the current robot fish speed, the bow swing amplitude in the current cycle, and the difference between the current robot fish speed and the target speed.

5. The method for learning the bio-inspired robotic fish swimming motion according to claim 1, wherein: The reward function in the offline learning phase is set as: in, represents the reward function in the offline learning phase, and are the coefficients for evaluating acceleration and pitch amplitude, represents the acceleration at the next moment, Indicates the heading amplitude in the current cycle. Indicates the allowable range of yaw.

6. The method for learning the bio-inspired robotic fish swimming motion according to claim 1, wherein: The reward function in the online learning phase is set as: in, represents the reward function in the online learning phase, and are the coefficients for evaluating speed and yaw amplitude, represents the target speed, Indicates the current speed of the robot fish. Indicates the heading amplitude in the current cycle. Indicates the allowable range of yaw.

7. A bio-inspired robotic fish swimming motion learning device, characterized in that: include: A key point extraction module is used to extract key points of fish based on the collected biological fish swimming data to obtain a biologically inspired experience database; The trajectory extraction module is used to obtain the position of each joint of the robotic fish and determine the swimming action sequence of the fish using the dynamic trajectory method based on the position of each joint of the robotic fish; An offline learning module, configured to pre-train a pre-built intelligent agent using the biologically inspired experience database, and perform offline learning based on biological data to obtain a pre-trained intelligent agent; An online learning module is used to train the pre-trained agent based on the fish swimming motion sequence, perform online learning, and obtain a robotic fish swimming motion learning agent; wherein different state vectors and action vectors are used in the online learning stage and the offline learning stage; A swimming control module is used to control the swimming of the robotic fish based on the swimming action learning agent of the robotic fish.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the bio-inspired robotic fish swimming motion learning method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the bio-inspired robotic fish swimming motion learning method according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the bio-inspired robotic fish swimming motion learning method according to any one of claims 1 to 6 is implemented.

Images